Senior Software Engineer: AI Evaluation / Coding Agents ($100-150/hr)

What you'll do

You treat a coding agent as another engineer whose pull requests you are reviewing. Did it understand the task, choose a sensible approach, and write code that is correct, robust and maintainable? Your written verdicts and labels become the signal used to improve the model.

The ad lists:

• Designing and iterating on rubrics and evaluation criteria for preference data
• Reviewing and labelling data to a high quality bar, catching subtle issues
• Building and maintaining pipelines that support data generation, collection and evaluation
• Writing up findings and recommendations for the team
• Working with researchers and engineers to turn qualitative judgment into scalable processes

Who fits

• 5+ years of hands-on software engineering
• Strong Python, TypeScript/JavaScript, Go or another major production language
• Experience in substantial real-world codebases and strong code-review skills
• Clear written communication, and the ability to explain why code is right or wrong
• Experience with modern LLMs or AI coding tools

LLM evaluation, coding agents, RLHF, preference data, rubric design or post-training experience is a plus, not a requirement.

What it pays

$100-150/hour. The ad asks you to state a specific hourly rate expectation when applying, so where you land in the band is partly your own ask.

Worth knowing

Good:

• A published, high hourly band
• Review work rather than building production apps
• The ad says the practical exercise tests reviewing AI code, not algorithm puzzles

Less good:

• Open to North America only
• 20-40 hours a week, with at least 6 hours of Pacific Time overlap each day
• About 3 months as an independent contractor
• Three interview steps before you start

How hiring works

An AI interview (about 25 minutes), a practical code and AI evaluation exercise (about 30 minutes), then a hiring manager interview (about 20 minutes).

Location

The ad says North America (only) but does not list countries, so check eligibility before applying.

About this listing

Posted by Turing on 9 September 2026 and read on 18 October 2026. Rate, hours, duration and interview steps are Turing's own; the framing is ours. See Turing.