Content Review Specialist (Preference Ranking Model Evaluator)
Mercor connects elite creative and technical talent with leading AI research labs. Position: Preference Ranking Model Evaluator, Type: Freelance, Location: Remote, Commitment: 10–20 hours/week.
Base pay range: $25.00/hr – $35.00/hr.
Role Responsibilities
* Evaluate and compare AI-generated responses based on quality, coherence, and helpfulness.
* Assign preference rankings to pairs or sets of model outputs.
* Follow detailed labeling guidelines and adjust based on evolving criteria.
* Provide brief written explanations for ranking decisions when required.
* Flag edge cases or inconsistencies in task design or model output.
* Work independently and asynchronously to meet deadlines while improving AI model performance.
Qualifications
* Prior experience in data labeling, content moderation, or preference ranking tasks.
* Excellent critical thinking and reading comprehension skills.
* Comfort working with evolving guidelines and ambiguity.
* Strong attention to detail and consistency across repetitive tasks.
* Availability for regular part-time work on a weekly basis.
Compensation & Legal
* Payments issued weekly via Stripe Connect.
* Classified as an independent contractor.
Application Process
* Submit your resume.
* Complete a short form to highlight your relevant experience.
* You may be asked to complete a brief assessment to evaluate task fit.
Resources & Support
* For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome
* For any help or support, reach out to: support@mercor.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Seniority level: Not Applicable
Employment type: Part‑time
Job function: Quality Assurance
#J-18808-Ljbffr