What is model evaluation?
Judging model answers against a rubric for accuracy, helpfulness, safety and style. Specialist evaluation is among the best paid work in this field.
Model evaluation is judging what an AI model produces against a written rubric. Evaluators check whether an answer is correct, complete, safe and in the requested style, and score it so the model's makers know where it fails.
What the work looks like
You read a prompt and the model's answer, score it on several dimensions, and justify each score. Specialist evaluation puts experts in front of answers in their own field: a physician checks clinical advice, a lawyer checks a summary of case law, a senior engineer checks a code fix. Many projects also ask for a corrected answer when the model is wrong.
Why it pays well
An error in a specialist answer is invisible to a non-expert, so labs pay for people who can see it. Evaluation roles are among the highest paying on this board.
Example
A model is asked for the maximum daily dose of a common painkiller. It gives the right figure but omits the lower limit for people with liver disease. The evaluator marks it incomplete and unsafe, and adds the missing warning.
Looking for this kind of work? AI Safety & Evaluation roles
More terms: task types
Full glossaryReady to look at roles?
Every listing shows its pay and who it is open to.