A coding agent improved its results on two software-repair benchmarks after researchers trained it to replace old search and tool history with a short task summary. The AutoCompact method raised pass rates over the same Qwen3-Coder-30B-A3B-Instruct system by 9.2 percentage points on SWE-bench Verified and 5.0 points on SWE-PolyBench Verified, according to an Oct. 1 arXiv preprint.
Repository-level coding agents inspect files, search for bugs, edit code and run tests. Their history fills with failed attempts and long tool outputs that may stop helping before the context window is full. AutoCompact gives the agent a compact() action: after resolving one stage of a task, it can replace earlier interaction history with a summary of its findings, relevant code and workspace state, and the next steps. The original task remains available.
The training method covers three connected decisions: when to compact, what information the summary must preserve, and how the agent should continue. A judge model corrected these choices in coding trajectories, and the environment executed each correction before the trajectory moved on. The researchers used 1,052 such trajectories to fine-tune the model, then used reinforcement learning that rewarded patches only when they passed task tests. The judge was not used during evaluation.
In the full-run comparison, both versions used the same base model and terminal tool scaffold. AutoCompact passed 39.6% of SWE-bench Verified tasks, versus 30.4% for the base model; on SWE-PolyBench Verified, it passed 24.5%, versus 19.5%. The scores average three runs. On SWE-bench Verified, the gain also appeared with a 256,000-token context window that none of the runs filled, suggesting that replacing stale history can help even when context overflow is not forcing a summary.
The result is limited to one Qwen-based agent, one terminal scaffold and two static benchmarks. The study is a preprint, not a production test of existing coding assistants. Its reinforcement-learning training used 32,000-token sequences, while the long-context evaluation used a 256,000-token window; integrating learned compaction into other agent systems remains future work.
Sources
- AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents (arXiv preprint)
- SWE-bench Verified: dataset and evaluation description (OpenAI)
- SWE-PolyBench: A Multi-Language Benchmark for Repository-Level Evaluation of Coding Agents (arXiv)
- Figure credit: Figure 1 from the AutoCompact preprint, arXiv:2610.02163v1, CC BY 4.0.