Microsoft's Agent Lightning recipe reports Qwen3.5-35B-A3B rising 13.8 percentage points on SWE-bench Verified. The benchmark gives an agent a real GitHub issue and repository, then tests whether its code patch resolves the issue. The model scored 47.8% (239 of 500 tasks) before reinforcement learning and 61.6% (308) after 1,792 training examples. Microsoft published the mixture-of-experts recipe on September 29.
Agent Lightning trains the model through the same coding-agent harness used to run it. A harness is the software that supplies context, calls tools and controls the agent's steps. Here, mini-SWE-agent still handles repository tasks while a language-model proxy records its requests and responses for reinforcement-learning updates. That lets the training system observe what the model did without rebuilding the agent's tool loop inside the trainer.
The model is a mixture of experts, which routes each token through selected internal components. During a rollout, vLLM records which experts handled the model's responses. Microsoft's R3 method passes those routes through the training pipeline so Megatron replays them during the update. The documented setup uses four B200 GPUs.
At 2,816 training examples, the table reports 60.6%, below the 61.6% score after 1,792 examples. The project-reported result covers one benchmark; it does not show how reliable the agent would be in software teams.
Sources
- Microsoft's Qwen MoE coding-agent example, with training setup and all 500 SWE-bench Verified results
- SWE-bench FAQ on real GitHub issues, patches and benchmark evaluation
- Agent Lightning repository and September release notes
- Server-rack photo by Derrick Coetzee, Wikimedia Commons, CC0 1.0; illustrative, not the experiment
