A new training method lets a separate AI model search for varied math solutions, then passes only screened work to the model being trained. In a preprint, Columbia researchers report higher math-benchmark scores for three tested language models trained this way.
In reinforcement learning with verifiable rewards (RLVR), a program checks each answer and training reinforces solutions that pass. Adding a novelty bonus can prompt the same model to try different reasoning strategies, but the authors found that it could also reduce accuracy and other capabilities.
Exploration-Distillation (ExpDis) gives exploration and optimization separate roles. An “explorer” model receives a novelty bonus. A filter keeps only solutions with a correct final answer that stay within a token limit and avoid repetition loops. A separate “student” learns from those examples, then undergoes RLVR training using only correctness rewards. The student gets examples from the explorer rather than its novelty-driven parameter updates.
For Qwen3-1.7B, ExpDis reached a 51.81% mean pass@1 across five math benchmarks: AIME24, AIME25, AIME26, MATH500 and Minerva-Math. Pass@1 estimates the chance that one sampled answer to a problem is correct. Standard DAPO scored 45.63% with the same 300 RL updates; DAPO with four times as many updates (1,200) scored 47.28%. ExpDis also outscored standard DAPO on the study’s Qwen3-4B and Ministral-3-3B-Instruct models.
The paper reports tests on seven math benchmarks but leaves AMC23 and GSM8K out of the five-benchmark average because scores were already near their maximum for the tested models (on GSM8K) and Qwen3 models (on AMC23). These are the preprint’s own experiments on three base models, not an independent replication or evidence of gains on non-math work. Under the tested compute budget, the authors also saw gains plateau and begin to regress beyond three parallel explorers and four training rounds.