At Crossing Hurdles, we work as a referral partner. We refer candidates to Mercor that collaborates with the world’s leading AI research labs to build and train cutting-edge AI models.
Organization: Mercor
Position: Data Scientist
Referral Partner: Crossing Hurdles
Type: Hourly contract
Compensation: $100-$120 per hour
Location: Remote
Duration: 3–4 weeks
Commitment: 10-40 hours/week, flexible and asynchronous
Role Responsibilities (Training support will be provided)
* Conduct statistical failure analysis across finance-sector AI tasks.
* Identify patterns in AI agent performance failures across task components (e.g., prompts, rubrics, templates).
* Perform root cause analysis to determine if failures stem from task design, rubric clarity, or agent limitations.
* Analyze performance variations across finance sub-domains, file types, and task categories.
* Create dashboards and reports highlighting failure clusters, edge cases, and areas for improvement.
* Recommend improvements to task design, rubric structure, and evaluation criteria.
* Communicate insights to data labeling experts and technical teams.
Requirements
* Strong foundation in statistical analysis, hypothesis testing, and pattern recognition.
* Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analysis.
* Experience with exploratory data analysis and deriving actionable insights from complex datasets.
* Familiarity with LLM evaluation methods and quality metrics.
* Comfortable working with Excel, data visualization tools (Tableau/Looker), and SQL.
* Strong relevant experience.
Application Process (Takes 20 min)
* Upload resume.
* AI interview based on your resume (15 min).
* Submit form.