Healthcare Specialist: Rubric Design

What you'll do

You write questions that expose where frontier AI models fail in your specialty, and then write the rubric: the facts a good answer must contain, the reasoning steps, caveats, acceptable alternatives, minor omissions and critical errors. You also critique rubrics other healthcare professionals have written, evaluate model responses for accuracy, safety and calibration, and produce reference answers for reinforcement-learning and supervised fine-tuning data.

The ad puts rubric design at the center. The skill it prizes is telling a superficial answer from an expert one, and telling a harmless omission from one that materially affects clinical quality or safety.

Who fits

• A professional degree, license or recognized qualification in medicine, dentistry, pharmacy, nursing or another healthcare discipline
• 3+ years of professional or clinical experience; senior and specialist professionals are strongly preferred
• Demonstrated expertise in a defined specialty or subspecialty
• Ability to write challenging clinical questions and detailed scoring rubrics
• C1 English
• Preferred: medical education, exam writing, competency assessment, peer review; AI evaluation or RLHF experience

What it pays

SME Careers lists up to $100 an hour, tied for the top band in this batch, with no floor. The ad does not say which specialties or seniority earn the ceiling. The ad does not say how payment is made or how often; the hiring process and payout terms SME publishes are covered on the SuperAnnotate page.

Worth knowing

Good:

• Pays for clinical judgment, and the work is writing and reviewing rather than patient care
• Covers a broad list of healthcare disciplines, not only physicians
• Rubric design is a transferable skill for assessment and education work

Less good:

• United States only
• Credentials and specialty expertise are a high bar
• Hourly projects are time-tracked, and the ad names no current volume

About this listing

Listed on SME Careers, the site where SuperAnnotate hires subject matter experts, and read from its jobs board on 27 October 2026. The board gives no posting date, so we cannot say how long it has been up. Pay, duties and requirements are SME's own text; the commentary is ours. See SuperAnnotate.