Introducing MentalHealthBench | OpenAI

OpenAI released MentalHealthBench, an open evaluation tool designed to measure how AI systems handle mental health conversations. They built it with over 80 licensed mental health professionals from 22 countries speaking 19 languages across nearly 20 subspecialties. The benchmark assesses whether AI responses demonstrate key behaviors like safety, gathering context, respecting user autonomy, and offering practical guidance when suitable. They're releasing it publicly so other researchers can run their own tests and build on the work, framing this as a way to measure progress toward AI that responds with actual empathy rather than just mimicking it. The benchmark covers realistic scenarios with adults, teens, caregivers, and clinicians across multiple languages and regions. It spans a spectrum from everyday conversations with emotional components to high-acuity situations involving serious mental health concerns or significant distress—but not immediate emergencies. This range matters because it lets researchers see how models perform on routine situations versus more urgent ones that might need real-world support like crisis hotlines. The design acknowledges that mental health conversations vary wildly in topic, urgency, and cultural context, so a useful benchmark has to capture that breadth. Performance data shows recent frontier models improving steadily on MentalHealthBench, though the article doesn't specify which models scored where or by how much. OpenAI emphasizes that ChatGPT isn't a substitute for actual therapy or professional care—it's a tool meant to help people find their way toward real support. They position this benchmark as part of a broader effort to set higher standards for how AI handles mental health situations, acknowledging upfront that no single benchmark captures everything that matters in a personal conversation.