TokenCast updates forecasts of an AI agent’s token use as it works. In an offline replay, a forecast-driven controller used 21.3% fewer tokens on average across seven budget levels while matching a fixed-cap policy’s rate of runs reaching their recorded terminal state at each level. That state can mark completion, failure or a run limit, so the replay does not measure task success.
An agent’s later steps depend on what earlier model calls and tools return. Each new call may also include the conversation built up so far, so a growing context raises the input-token cost of subsequent calls. Retries and additional tool cycles add further uncertainty.
TokenCast breaks a run into linked segments. For each segment, it estimates how many calls it contains, how much the input context grows, and tokens consumed by generated output and within-segment context growth. When it combines segments, it carries earlier context growth forward because later calls reread the larger history. It refreshes forecasts as calls finish, using observed outputs and tool results without extra language-model calls.
Across 96 combinations of benchmark suite, model and forecast point, TokenCast’s forecasts had 14.5% lower average error than the strongest comparator in each combination. It trailed the best comparator in 24 cases, all at the start of a task or call; it did not trail at later forecast points, when more execution evidence was available.
The token-saving test replayed 288 recorded GPT-5.4 runs from 144 SWE-bench Verified tasks, not a live agent service. A fixed-cap policy stopped each replay at a preset token limit; TokenCast’s controller added tokens already used to its forecast of remaining consumption and stopped when the total would exceed the same cap. For each budget, the researchers selected forecast cutoffs to match the fixed-cap policy’s rate of runs reaching their recorded terminal state.
The predictors learn from recorded traces, and transfer to new task types or models benefits from sample traces from the target setting. Integrating the forecasts into a live runtime controller remains future work. The authors’ linked GitHub repository was empty as of Sept. 29.
Sources
- TokenCast paper, arXiv:2609.35760v1
- Authors’ TokenCast code repository (empty when checked Sept. 29)
- Cover image: CSIRO server racks via Wikimedia Commons, CC BY 3.0. Illustrative image, not TokenCast’s test setup.
