Click any tag below to further narrow down your results
Links
OpenAI is pitching agents that operate EHRs and payer portals by clicking and typing, so they can handle tasks like MRI authorization that APIs don't reach. The post shares evaluation results on helpfulness, safety, and cost, plus early deployments with Cigna, CVS Health, Abridge, and Ambience.
- A demo agent takes a patient's message and visit notes, pulls prior imaging and PT records from the EHR, and drafts an MRI order, encounter note, and patient reply for clinician review.
- CVS Health reports an agent cleared a months-long document-sending backlog over a weekend and handles system steps for tens of thousands of consults a day, with validation checks and human oversight.
- Cigna case managers previously had to navigate more than 10 separate systems to gather member context; computer use now pulls that together in seconds.
@muratcan on X
The post reports an experiment where Jev, an AI model, analyzed 2,029 real phone calls handled by an AI receptionist, using only call structure (turns, tool calls, workflow stages, timing) and never the audio or transcripts. Jev predicted which calls would end in a booking with an AUC of 0.78 at the halfway point, rising to 94% correct ranking near the end, at a cost of about $3.
- Jev never heard the calls; it worked only from structural signals like turns, tool calls, and workflow stages.
- It made 38,012 turn-level forecasts at a 118 ms median latency and reviewed all calls in 26 seconds.
- Zero-shot, with no fine-tuning, it reached an AUC of 0.78 mid-run and ranked calls correctly 94% of the time by the end, though it overweighted visible errors the agent usually recovers from.
OpenAI created MentalHealthBench, an open-source benchmark developed with 80+ licensed mental health experts to measure how well AI systems handle mental health conversations. The benchmark tests whether AI responds safely, preserves user autonomy, and appropriately directs people to professional help.
- Built with 80+ licensed mental health professionals across 22 countries speaking 19 languages, covering nearly 20 subspecialties
- Evaluates AI on specific behaviors: safety, seeking context, preserving user agency, and providing actionable guidance
- Covers the full spectrum from everyday emotional conversations to high-acuity scenarios requiring crisis intervention
- Results show steady improvement in frontier models, though the benchmark acknowledges AI is not a substitute for actual therapy