.jpg)
OpenAI has released MentalHealthBench, a public test of how AI responds to people discussing mental health. Its 1,215 synthetic conversations range from everyday stress to emergencies, and licensed clinicians wrote the scoring rules for each situation. The scores show whether a reply meets those rules; they do not measure whether it improves care.
The release addresses a gap in tests that concentrate on crisis responses or apply the same broad checklist to every exchange. Each MentalHealthBench conversation ends with a user message. The AI must answer that message in context, whether the simulated user is an adult, a teenager, a caregiver or a clinician.
Clinicians wrote rules for cases at different levels of urgency
More than 80 licensed mental-health experts from 22 countries reviewed the conversations. The released dataset contains 5,262 criteria, awarding points for helpful actions and subtracting them for harmful ones. At least three experts reviewed each conversation, and OpenAI retained criteria supported by at least two without a third expert's objection. An automated AI grader then checks model replies against those rules.
The cases were selected to cover different levels of urgency, not to reproduce the frequency of mental-health topics in ChatGPT. The benchmark includes non-acute conversations, more serious distress and situations requiring urgent real-world support. Its scores therefore describe performance on this designed set of cases.
What the score rewards, and what users value
One comparison in OpenAI's paper shows why the scoring method matters. A set of clinician-written reference replies averaged 38.5% under the benchmark's scoring system, while an AI shown the criteria it would be graded on reached 99.0%. The paper says the clinicians' typically short replies earned fewer positive points but also drew fewer penalties. Those figures compare checklist coverage under different response conditions, not the quality of a human therapy session against AI care.
A separate review by 44 adults who had used AI for emotional support covered only non-acute conversations. Those participants placed more weight on practical next steps and tone than the experts did; the experts emphasized gathering context and handling ambiguity. Their ratings did not change the benchmark's final criteria.
Other researchers can now run the public test against their own models. OpenAI says ChatGPT is not a substitute for therapy or professional care.
Sources
- OpenAI: Introducing MentalHealthBench (September 23, 2026)
- MentalHealthBench research paper, especially Sections 2, 4.5 and 4.6
- Public benchmark dataset
Image: An unrelated human telepsychiatry session photographed by Lance Cheung for the U.S. Department of Agriculture, public domain via Wikimedia Commons. The photograph does not depict an AI test or a MentalHealthBench participant.