Senior Software Engineer: Task Ideation

What you'll do

The work is building command-line tasks that frontier AI agents can be scored on. You start from something that happened to you (a production incident, a performance problem, a fix that looked right and was not), tell it in an AI-guided interview, and then turn it into a task: a Docker environment, an instruction, a reference solution and tests. Some engineers pair with scientists, operations professionals or hardware engineers to convert their expertise into working tasks. Reviewers do the opposite job: they try to pass the tests without solving the problem, which is how weak tasks get caught.

Who fits

The ad lists seven profiles:

• Core task engineer: senior generalist, 5+ years, production Python plus another language, Docker, pytest, advanced Linux
• Systems and storage specialist: 6+ years inside engines such as Postgres, RocksDB, Kafka, Spark or the Linux kernel
• ML systems and research specialist: ML infrastructure (GPUs, distributed training, serving), MSc/PhD or 3–5 years
• Security researcher: AppSec, cryptography, forensics or firmware; 3+ years of CTF or 4+ in industry
• Red-team / QA reviewer: attacker mindset, 4+ years in SWE, SRE or security, or 3+ years of CTF
• Benchmark infra engineer: DevOps/SRE, 5+ years; builds the environments, does not write tasks
• Calibration lead: 7+ years leading review or evaluation; sets the quality bar. Very few positions

Everyone needs strong coding, advanced Linux, concrete evidence of depth (commits to a major system, published challenges or CVEs, a postmortem you owned) and comfortable English. Test design, large codebases and AI coding tool experience are nice to have.

What it pays

$20–100 per approved task (unit: per package). The ad does not say which profile or task earns more, how long a task takes or whether the infra and calibration roles are paid the same way, so you cannot compute an hourly rate from it. Weekly payouts are shown on the listing.

Worth knowing

Good:

• Real engineering depth is the point; the ad says it is not about writing more code
• Seven distinct profiles, including review and infrastructure
• Remote

Less good:

• Compensation is per approved task, so rejected tasks pay nothing
• A wide range with no stated criteria
• Calibration lead places are very few
• Evidence of depth is demanded up front

About this listing

Vetto posted this on 15 October 2026 and we read it on 23 October 2026. Work happens in Vetto Arena. Pay and requirements are Vetto's; the gaps noted are ours. See Vetto.