Researchers from UC Berkeley and Sakana AI used work produced by coordinated AI agents to train Qwen3.6-27B in two successive rounds. In an Oct. 8 arXiv preprint, mean scores on four research benchmarks rose, while pass rates on two coding tests fell.
Under Multi-Agent Self-Supervision (MASS), the same Qwen model solves tasks, designs a workflow for an agent team and evaluates the team's output. First, without changing the model itself, it tries different instructions for a team: which subtasks a lead agent delegates to supporting agents and what information each returns. It then trains the shared model further on selected records from those runs. The updated model starts another cycle.
On three held-out synthetic research tasks spanning finance, robotics and pharmacy, the second-round model beat the original in 69.9% of pairwise comparisons, up from 53.9% after one round. GPT-5.5 and Claude Opus 4.8 supplied six judgments per comparison; ties did not count. The researchers generated 12 tasks with GPT-5.5: nine were designated for training and three held out. One training task was dropped because no workflow beat its reference, leaving eight to produce training data.
In the paper's public-benchmark results, mean ScienceAgentBench success rose from 28.8% to 31.7%, MLR-Bench's overall score from 1.58 to 2.40 out of 10, AstaBench's rubric score from 0.053 to 0.067, and DSBench's normalized score from 0.473 to 0.482. Coding results moved the other way: Terminal-Bench 2.0 pass rate fell from 37.5% to 35.2%, and SWE-bench Verified from 62.6% to 61.1%. The paper reports means over three trials for these benchmarks, except MLR-Bench, which uses five trials on a 12-task subset.
In that MLR-Bench subset, Qwen delivered 31 of 60 papers before training and 50 of 60 after two cycles. Missing papers receive zero in the overall score, so higher delivery can lift the aggregate even when scores among submitted papers fall; that mean declined from 3.07 to 2.88. The submitted sets differed, and a paired test across the 12 task means did not establish a consistent gain.
The authors' repository separately lists a 107-task, three-run MLR-Bench result: 1.71 to 2.25 out of 10, with paper delivery rising from 57% to 76%. This is a different setup from the preprint's 12-task, five-run result; the repository says the 107-task configuration is not included in its packaged code.
The study stops after two cycles on one 27-billion-parameter model. Whether another model or another cycle would produce the same pattern remains an open question.
Sources
- Lee et al., “Recursive Self-Improvement through Multi-Agent Self-Supervision,” arXiv:2610.12176v1, October 8, 2026
- Sakana AI’s MASS repository and benchmark table
- Figure 1 from the preprint, reused without modification under CC BY 4.0.