Senior Software Engineer: LLM Evaluation

What you'll do

You help build the datasets used to train and benchmark large language models, working closely with researchers. In practice that means curating code examples, writing precise solutions and correcting code in Python, JavaScript (including ReactJS), C/C++, Java, Rust and Go, and then evaluating and refining AI-generated code for efficiency, scalability and reliability.

The role goes a step beyond reviewing. You are asked to build agents that verify code quality and spot error patterns, to reason about stages of the software lifecycle (prototyping, architecture, API design, production implementation, launch, experiments, monitoring, maintenance) and test model capability on each, and to design mechanisms that automatically verify a solution to an engineering task.

Who qualifies

• Several years of software engineering, including 2+ years of continuous full-time work at a top-tier product company (the ad names Google, Stripe, Amazon, Apple, Meta, Netflix, Microsoft, Datadog, Dropbox, Shopify, PayPal and IBM Research as examples)
• Strong full-stack skills and experience deploying scalable production software
• Deep understanding of architecture, design, debugging and code review
• Clear, structured written and spoken communication for evaluation rationales

That second bullet is the unusual bar: it is a pedigree requirement, not just a skills one.

What it pays

The ad does not publish a rate. Turing sets rates per project; see Turing.

Worth knowing

Good:

• Flexible load: a minimum of 10 hours a week and up to 40
• Engineering-heavy work (verifiers, agents) rather than just rating answers
• Possible extensions based on performance and fit

Less good:

• The initial duration is 1 month, with an extension not guaranteed
• Partial PST overlap is required
• Contractor engagement with no medical or paid leave
• No pay figure
• The top-tier-company experience requirement excludes many strong engineers

The ad states no assessment. Turing's help centre describes skill tests and, for some roles, a live interview.

About this listing

Read from Turing's job board on 18 October 2026. The role was first posted in June 2025 and is still open. Duties and terms are Turing's; the commentary is ours. See Turing.