What is response ranking?

Response ranking is putting two or more model answers to the same prompt in order from best to worst. It is the most common task on RLHF projects, because rankings are the preference data that teaches a model which answers people prefer.

How it works

You read the prompt, read every answer in full, then order them. Most projects also ask for a score per dimension (accuracy, helpfulness, safety, style) and a short written reason for the top choice. Some let you mark two answers as tied when neither is better.

What makes a good ranking

• Correctness beats style: a plain correct answer outranks a polished wrong one.
• Follow the rubric, not personal taste.
• Check every claim you can, especially numbers and names.
• Write reasons that name the specific difference, so a reviewer can verify them.

Example

Three answers explain how to calculate compound interest. Answer A is correct and concise, answer B is correct but buries the formula in padding, answer C uses simple interest by mistake. The ranking is A, B, C, with the reason that C uses the wrong formula and B adds no information.