Computing and ICT
Active Posting
SWE-Bench Task Auditor
Mercor
Remote, United States of America
Contract
Mid Level
Job Description
Evaluate the quality, correctness, and reproducibility of software-engineering benchmark tasks used to train and evaluate a frontier AI lab's models.
Duties & Responsibilities
Assess repository-level tasks, reference patches, test harnesses, and grading integrity — and provide clear, rubric-based written feedback.
3+ years professional software engineering, Real open-source contribution or maintainer experience, Strong ability to audit reference patches, test runners, and Docker isolation, and to detect answer leakage / reward hacking.
Required Skills
Python, Java, Go, TypeScript, C++
Benefits & Perks
$70-$90 / hr, Hourly contract, Remote — United States, Weekly payments on Stripe or Wise
How to Apply
https://mercor.com
AI Custom Cover Letter Assistant
Draft a personalized, high-converting cover letter tailored specifically to this position using your selected Career Profile and AI NLP orchestration.
Sign in to generate a custom AI cover letter tailored to this job!
Sign In / RegisterInstant Match % Analysis
Sign in and set up your Career Profile to calculate your match compatibility score for this position.
Sign In to Calculate MatchJob Summary
- Posted Date Sep 12, 2026
- Expiration Date Open until filled
- Category Computing and ICT
- Job Type Contract
- Level Mid Level