Medical Evaluation Specialist (students, residents, physicians)

What you'll do

You write medical exam questions designed to break frontier models. Each one is an original, high-difficulty question-and-answer pair on diagnosis, mechanisms, pharmacology, guidelines or clinical judgement, and it has to require synthesis and genuine clinical reasoning rather than recall.

For each pair you source and verify the answer from primary literature, clinical guidelines and authoritative references, documenting the rationale and citations. Then you test it: you run it against AI models, and if they answer it too easily, you refine it to make it harder while keeping it clinically accurate and unambiguous. Reviewer feedback and evolving guidelines feed back into later rounds.

The practical skill here is knowing where models are weak: questions whose answer turns on a recent guideline change, a subtle contraindication, or two findings that only mean something together. A question that a textbook answers in one line will not survive the testing step.

Who fits

• Current medical training or practice as a student, resident or physician, or equivalent biomedical expertise
• The ability to find, interpret and synthesise primary literature and guidelines
• Close attention to detail and strong written English (native fluency is explicitly not required)
• A knack for building well-structured, clinically nuanced questions
• Self-direction and reliability working remotely

Question-writing, peer review or evaluation experience helps; AI familiarity is a plus but not essential.

How pay actually works

The headline is $40–90/hour, but the ad's compensation section says something different in substance:

Compensation is output-based; experts are paid per task that meets the project specifications.

So you are paid for accepted questions, not time. The hourly figure is best read as what a productive contributor might average. The ad also says time per task varies with experience and workflow, and that there is a minimum weekly submission requirement, though the actual number is missing from the text ("a minimum of tasks per week"). Ask what it is before you commit.

Onboarding is fast: roles are typically filled within 48 hours, and selected experts are expected to start tasks within 24 to 48 hours of onboarding.

Worth knowing

Good:

• Medical students are explicitly eligible, which is rare at this pay level
• Non-native English speakers welcome
• Quick start
• Intellectually engaging work that rewards clinical reasoning

Less good:

• Per-task pay: rejected or too-easy questions earn nothing, and the hourly band is not guaranteed
• A weekly minimum whose number the ad omits
• Only five openings
• Making questions hard enough to beat frontier models gets harder over time
• Contractor engagement: no benefits, no notice period, your own tax to manage

About this listing

Posted by micro1 on its own jobs portal and read on 24 September 2026. The $40–90/hour band, the five openings, the output-based pay paragraph and the 24 to 48 hour start expectation are its own published text; the missing weekly minimum is reproduced as found. Location is given only as remote with no country restriction, so eligibility is left open. See micro1.