AI Developer Trace Task Auditor

What you'll do

A frontier AI lab records whole software-development sessions carried out with AI-assisted developer tools, and uses them as training and evaluation material. Your job is to audit those traces.

That means reading a multi-step coding trajectory from start to finish and judging three things:

• Correctness: does the final code work, and did intermediate steps introduce bugs that were later papered over?
• Workflow soundness: was the sequence of actions sensible, the way a competent engineer using an agent would work?
• Reasoning: did the stated reasoning actually justify the edits made?

You then write clear, rubric-based feedback. The skill being bought is the one you use when reviewing a colleague's pull request that was mostly written by an agent: spotting where the tool went confidently in the wrong direction.

Who fits

Basic qualifications:

• 3+ years of professional software development
• Hands-on experience with AI-assisted coding and agentic or spec-driven workflows (Cursor, GitHub Copilot, Claude Code or similar)
• Strong code-reading and debugging across full-stack or backend systems
• The ability to evaluate multi-step coding trajectories for correctness and best practice

Preferred: experience with Kiro or Amazon CodeCatalyst, prior grading of AI-generated code, and contributions to developer tooling. The Kiro and CodeCatalyst mention hints at which tool ecosystem the traces come from, though the ad does not say so directly.

What it pays

$70–90 per hour, paid weekly via Stripe or Wise. H-1B and STEM OPT candidates cannot be supported. Weekly hours and duration are not published.

This rate is shared by a family of Mercor auditor listings posted at the same time (SWE-Bench, ML challenge, Kubernetes, AWS serverless, GPU kernels, CVE and others). This one has the lowest specialist bar of the group: general full-stack experience plus real agent use.

Worth knowing

Good:

• Most accessible of the $70–90 auditor family for a generalist engineer
• Your day-to-day use of coding agents is directly the qualification
• Asynchronous, on your own schedule

Less good:

• US-only, with the visa exclusion
• Long traces are slow to audit thoroughly, and the ad says nothing about expected throughput
• No hours or project length stated
• Independent contractor; projects can be extended, shortened or ended early

About this listing

Posted by Mercor as a remote hourly contract for US residents, confirmed open on 24 September 2026. Pay and qualifications are the ad's own. See Mercor.