Annotate and manage training/test datasets, run model training, evaluate model performance, perform manual QA and error analysis, and refine annotation guidelines for NLP/ML projects.
If you have a Candidate Login already, but have forgotten your password please use the steps to reset your password. If you have forgotten your email login, please contact [email protected] subject Workday Candidate Login
When creating your Workday account and entering personal information like name, address, please do not use ALL CAPS.
Thank you!
NOTICE: For Privacy Policy please review here
Job Responsibilities:
The ideal candidate will have a foundational understanding of machine learning, data annotation, quality assurance, and natural language processing. They will play a pivotal role in updating our machine learning models and ensuring their efficacy.MAIN TASKS & RESPONSIBILITIES
Machine Learning Model Updates:
- Update training and test model databases with new or amended synthetic textual and image data.
- Modify and refine machine learning data creation, annotation, and rating guidelines.
Model Training and Evaluation:
- Initiate model training processes using internal tools and command-line interfaces.
- Evaluate the performance of trained models to gauge their efficacy and readiness for deployment.
Data Management and Annotation:
- Design and develop test and training datasets as per the criteria provided by the project manager and other full-time employees.
- Handle data efficiently, ensuring its integrity throughout the workflow.
- Engage in data relevance tasks, ensuring data sets are aligned with project goals.
- Annotate data accurately, ensuring it adheres to set guidelines.
Quality Assurance and Analysis:
- Conduct manual quality analysis of model results.
- Recognize error patterns and report anomalies for further investigation.
- Deliver detailed reports on findings, including aspects such as utterance quality, LLM evaluation, ASR bug tracking, and customer pain points to be reviewed by the User Experience Research team.
- Implement basic quality control measures and ensure the reliability of processed data.
- Utilize intermediate data analysis techniques to extract insights and inform decision-making.
- Arbitrate discrepancies effectively, ensuring consistent data quality.
Linguistic and NLP Tasks:
- Apply basic knowledge of natural language processing and linguistics to data processing tasks.
- Ensure linguistic accuracy in all processed and annotated data.
REQUIREMENTS
Preferred Qualifications:
- Bachelor's degree in Computer Science, Data Science, Linguistics or Computational Linguistics or a related field.
Experience:
- Ability to work in a fast-paced, collaborative environment.
- Excellent communication skills
Skills & Knowledge:
- Familiarity with command-line tools and interfaces.
- Strong analytical skills with the ability to identify patterns and anomalies.
Additional Information:
This role primarily focuses on English US data sets; however, familiarity with translation or multi-lingual data sets can be a plus for future projects.
Additional Job Details:
Similar Jobs
One Month Ago
Artificial Intelligence • Machine Learning • Natural Language Processing • Professional Services
Evaluate AI model outputs, perform audit-based reviews, apply and refine guidelines, identify performance trends and failure modes, document findings, and collaborate with cross-functional teams to improve model evaluation and data quality.
Top Skills:
Ai SystemsGenerative Ai
One Month Ago
Artificial Intelligence • Machine Learning • Natural Language Processing • Professional Services
Evaluate AI model outputs, perform audit-style reviews to find patterns and failure modes, apply and refine labeling/evaluation guidelines, document findings, communicate insights to leads and stakeholders, and participate in team calibrations to improve model evaluation and data quality.
One Month Ago
Artificial Intelligence • Machine Learning • Natural Language Processing • Professional Services
Evaluate AI model outputs, perform audit-based reviews, apply and improve guidelines, identify trends and failure modes, document findings, and collaborate with leads and cross-functional teams to refine quality standards.
What you need to know about the Vancouver Tech Scene
Raincouver, Vancity, The Big Smoke — Vancouver is known by many names, and in recent years, it has gained a reputation as a growing hub for both tech and sustainability. Renowned for its natural beauty, the city has become a magnet for professionals eager to create environmental solutions, and with an emphasis on clean technology, renewable energy and environmental innovation, it's attracted companies across various industries, all working toward a shared goal: advancing clean technology.
