Researchers trained an AI math tutor with Sherpa, a multi-turn reinforcement-learning method. On an evaluation across seven simulated learner profiles, students tutored by the Qwen3-8B model reached 69.0% average accuracy, compared with 48.6% for the untrained model, a new preprint reports. The learners were language-model simulations, not people.
Solving a problem does not by itself make a model a good teacher. Sherpa rewards the tutor when a simulated learner answers a math problem more accurately after a tutoring conversation than the same learner model does without tutoring.
The team trained Qwen3-8B to tutor simulated students built on smaller Qwen3-1.7B models. Each round moves from preparation to a multi-turn lesson and a test. One gate blocks instructions that reveal the final answer; another checks whether the tutoring matches the learner's assigned preference and sends feedback when it does not. The simulated students represent six preferences, including diagnosing mistakes, breaking work into subgoals and comparing methods; a seventh has no preference.
For evaluation, the researchers selected 528 MATH problems that the teacher model could solve but the simulated learner could not answer alone; the filtered training set contained 759 problems. Three of the six preference profiles were held out from training. The abstract summarizes Sherpa's average gain as 20.5 percentage points, but the main table's displayed scores (69.0% and 48.6%) differ by 20.4 points. The preprint does not explain the tenth-point gap.
On MathTutorBench, Sherpa's model scored 79.2% for the average of four teaching-response tasks, compared with 52.5% for the untrained model and 63.2% for PedagogicalRL, another training method. But the trained tutor scored lower on mistake-location accuracy (30.3% versus 37.1%) and Socratic questioning (21.9% versus 25.3%).
A separate study asked 96 high-school teachers to judge 320 pairs of tutor responses, for 960 judgments in total. Sherpa's response won 728 judgments against 187 for the base model, with 45 ties; its 79.6% preference rate excludes ties. These were judgments about generated answers, not tests of what students learned. Some evaluators said Sherpa's explanations could be overwhelming.
The October 6 paper is a preprint, and its learning results come from simulated language-model students working on math problems. Whether the method helps real students retain concepts or solve new problems remains untested.
Sources
Image credit: Preply.com Images via Wikimedia Commons, CC BY 2.0; unmodified. The photo is illustrative and is not from the Sherpa study.