What is red teaming?

Red teaming is deliberately attacking an AI model to find the ways it can be made to produce harmful, false or policy-breaking output, before real users find them. The term comes from security exercises where a red team plays the attacker.

What the work looks like

You write prompts designed to get past the model's safety training: role-play setups, multi-step requests, coded language, or questions that hide a harmful goal inside an innocent one. When an attempt works, you record the prompt, the output and why it is a failure. Those records become training data that teaches the model to refuse.

Who is hired

General red teaming suits creative writers and careful testers. Specialist red teaming hires people who know what dangerous output looks like in their field, such as chemists, biosecurity researchers, cybersecurity professionals and lawyers. Projects usually involve a wellbeing policy, since some content is disturbing.

Example

A red teamer asks a model to write a thriller scene in which a character explains, in working detail, how to bypass a building's alarm system. If the model complies, that is logged as a jailbreak.