Software Engineers: Paid Code Review for AI Agent Evaluation

What you'll do

Terac's study looks at the coding environments used to benchmark AI agents. In a remote session you review a series of coding tasks and their evaluation harnesses and judge whether each challenge reflects a realistic software engineering problem. You verify the logic, test cases and overall structure, and give feedback through a guided conversation and screen-sharing exercises.

So the output is spoken and shown technical judgment, not a pile of written reviews. It is quality control on the tests that decide how good an AI agent looks.

Who fits

• Practicing software engineers experienced in building and testing complex systems
• Direct experience writing test suites, evaluation harnesses or thorough code reviews
• Backend, full-stack, machine learning engineers and software architects are named as welcome

No location or language is stated beyond the worldwide listing; sessions are conversational, so fluent English is likely needed (the ad does not say).

What it pays

$65 per hour, as stated. The ad does not say how long a session lasts or whether it is one session or several, so confirm the time commitment after you are matched. A sister study for engineers based in South Asia pays $40 per hour.

Worth knowing

Good:

• Solid hourly rate for work close to day-to-day engineering
• Remote, open worldwide
• Useful look at how agent benchmarks are built

Less good:

• Session length and number of sessions are not stated
• Screen sharing is part of the format
• Registering does not guarantee a match
• The AI company behind the benchmarks is not named

About this listing

The Apply button opens this study on Terac; you sign in or create a Terac account there to apply. The listing runs until 1 December 2026. See Terac.