An open-source audit of 341,054 public coding-agent runs finds that the software surrounding a model can be as important as the model when tool output swells. In one trace collection, observations longer than 20,000 characters appeared in none of the mini-swe-agent runs, half of the OpenHands runs and 55% of the SWE-agent runs. The result turns output bloat from a vague complaint into a behaviour that teams can measure in their own agent logs. [1] [2] [3]
The Agent Waste Index reads a trajectory, the recorded sequence of model messages, tool calls and tool responses, then applies simple detectors. It marks an identical action repeated at least three times without a state change as a loop, an immediate repeat after an error as a blind retry, and any tool observation above 20,000 characters as context bloat. Its pipeline normalizes nine public data formats before calculating each finding. The edition covers 11 datasets, 29 dataset-model-scaffold groups and four agent framework families, with ingestion capped at the first 20,000 rows of each dataset configuration and split. [2] [3]
The clearest internal contrast comes from NVIDIA's Open-SWE-Traces version 1.1. All three groups are software-engineering trajectories from the same trace release, yet the share containing oversized output ranges from 0% for mini-swe-agent to 50% for OpenHands and 55% for SWE-agent. That comparison does not isolate the scaffold in a controlled experiment: task mixes, model settings and collection details can differ, and the trace rows do not carry a single model field that supports a clean attribution. The audit therefore labels these groups by trace version and warns readers to quote the paired cost-and-success result rather than the scaffold figure alone. [2] [3] [4]
Across the report, loops, blind retries and excess context account for 3.96% of estimated spend. That is an internal estimate, not a bill: the code approximates tokens as characters divided by four, assumes the full context is resent at every step and prices it at a common Sonnet-class rate. The underlying datasets are also heterogeneous head samples, six of the 29 groups lack success labels, and two sources partly resample populations represented elsewhere. The authors explicitly retract an earlier broad claim that any detector finding predicts failure; in the larger edition, flagged runs do better in five of the 20 groups then carrying outcome labels. [2] [3]
The practical result is a diagnostic method rather than a universal waste rate. A team can run the published detectors against its own trajectories, change one tool wrapper or scaffold policy, and compare output size, retries, cost and task success on the same workload. The next convincing evidence would be a paired evaluation that holds tasks and model settings constant while changing only the scaffold, with billed token counts replacing the character estimate. That test would show which apparent savings survive without lowering the agent's ability to finish the work. [2]