A new study found that an AI pipeline can lose accuracy when later stages see only the previous stage’s output rather than the original problem. In one four-stage test on math questions, accuracy rose from 15.0% to 55.5% when the same pipeline could show every stage the original question.
The 15.0% versus 55.5% comparison came from Gemma 3 12B on MATH-500. The September 26 arXiv preprint tested 21 open-weight models on two math benchmarks, using 200 paired questions per comparison. Within each pair, the model, stages, prompts and output-token cap stayed fixed; only access to the original problem changed.
Earlier work had compared decomposed questions answered in one context with subquestions handled in separate contexts, but those methods also changed the call structure. The new setup isolates what happens when later stages can still refer to the original question.
In the test where information was lost between the first and second stages, giving the original question to the second stage produced higher accuracy than giving it only to the first. That pattern held in all seven models tested on each benchmark. Builders can use the result as a diagnostic: after a stage compresses a task into one message, test whether the stage receiving that message needs the full task again.
Restoring context still did not beat a single model call: on the same MATH-500 case, the one-call reference scored 79.5%. The full-context pipeline used a median 1.17 times as many input tokens as the pipeline in which stages saw only the previous stage’s output; a token-heavy placeholder that carried almost none of the problem did not improve GSM-Hard scores, but the study did not compare equal compute spent on extra reasoning. Its main results are mathematical, and a small non-math check covered only two models, so the paper does not establish that the effect transfers to coding or tool-use pipelines.
Sources
- The Decomposition Tax (2026 preprint). The opening chart is reproduced unmodified from the paper by Tianqi Bu, YuXuan Peng, Junteng Tu and Henghui Xiao under CC BY 4.0.
- Question Decomposition Improves the Faithfulness of Model-Generated Reasoning (2023), the earlier comparison discussed above.