Researchers at UT Southwestern Medical Center say an AI system took the first scoring pass on a medical-school clinical exam in Fall 2025, while human reviewers checked students whose marks fell in the lowest ranges. The prospective deployment covered 222 second-year students and used written notes, recorded conversations and examination videos.
In this kind of clinical-skills exam, students rotate through simulated-patient encounters and are assessed against detailed rubrics. Each student completed five stations. Their notes, conversations and physical-exam performance produced separate items for scoring.
The system, called MAPLES, used a rubric to decide whether each item should be scored from a note, audio recording or synchronized video. The study used Gemini 2.5 Pro for scoring and Gemini 2.5 Flash to isolate physical-exam segments in the videos. It was a first-pass workflow: the students routed for review had their items independently rescored by standardized-patient evaluators (SPEs), the human assessors in the examination.
The team retained 72,907 item scores for analysis. It routed students in the bottom 5% for note scores and bottom 10% for audio and video scores, sending 28 students for review. Evaluators rescored 4,973 items. Their scores exactly matched the AI on 72.0% of reviewed items; when scores one level apart on rubric items with multiple levels were also counted as agreement, the rate rose to 87.7% overall, ranging from 85.7% for notes to 92.4% for video.
Physicians then reviewed 616 selected disagreements with both scores and their source visible. Their scores matched the AI on 76.0% of items, the evaluator on 19.0%, and neither on 5.0%. Because those physicians could see which score came from which source, and the cases were selected disagreements from a low-score review group, the result does not show that AI graded the whole class more accurately.
The authors counted 5,616 human scoring passes in the AI-assisted workflow, compared with 72,907 passes in a modeled system where one person scored every item, a 92.3% reduction. That figure counts scoring passes, not measured staff hours or total operating cost; the estimate excludes system development, integration and maintenance. The study covers one medical center. Human evaluators did not rescore students above the review thresholds, so the study provides no direct human comparison for those students. Faculty retained authority over pass/fail, progression and remediation decisions.
Sources
- Published paper in npj Digital Medicine
- Authors’ full manuscript and methods
- Illustrative photo by Sushil Dawka, via Wikimedia Commons, CC BY-SA 4.0. It shows a student practicing an eye examination on a dummy; it is not from the MAPLES study.
