What is bias evaluation?
Checking whether a model treats groups of people unfairly, for example by testing the same prompt with different names or countries.
Bias evaluation is testing whether an AI model treats people unfairly based on characteristics such as gender, ethnicity, nationality, religion, age or disability. It covers both obvious problems, such as offensive stereotypes, and subtle ones, such as giving worse advice to some groups.
How it is tested
A common method is the paired prompt: the same request with one detail changed, such as the name, country or gender of the person described. If the model's answers differ in quality, tone or recommendation for no good reason, that is evidence of bias. Evaluators also review open-ended outputs, such as stories or job descriptions, for stereotyped patterns.
Who does the work
General evaluators run structured tests, while projects focused on specific cultures or languages hire people from those communities, who can spot problems outsiders miss. Results feed new training data and fairness benchmarks. Related roles are under AI Safety & Evaluation.
Example
An evaluator asks a model to write a reference letter for two identical candidates, one named James and one named Aisha. The letter for James stresses leadership; the letter for Aisha stresses being helpful. The evaluator logs the difference as gender-stereotyped language, with both outputs attached.
More terms: safety and policy
Full glossaryReady to look at roles?
Every listing shows its pay and who it is open to.