A new benchmark found a wide gap in how closely three AI agent setups matched requested session durations. Across 222 coding, computer-use and research tasks, GPT-6 Astra in Codex ended within 5% of the requested session duration in 63% of counted runs; GPT-5.6 Sol in Codex did so in 39%, and Fable 5.1 in Claude Code in 4%.
AgentTime, a preprint by Michael Ofengenden and Maksym Andriushchenko, tests whether an agent can control how long its session runs. The authors first posted the paper to arXiv on Oct. 7 and revised it on Oct. 8. The benchmark draws 222 tasks from 18 sources, including coding, computer-use, agent-work and automated-research tests.
For each task, the researchers added a requested duration to the prompt and started a fresh agent session. Requests ranged from about a minute to 60 hours, though only four tasks had requested durations beyond 8.3 hours. A separate timer started when each task was delivered and stopped when the agent session ended; the benchmark's usual grader scored the result. The agent could stop early or keep running because the requested duration was an instruction, not a forced cutoff.
The paper counted a run as on time if the session lasted between 95% and 105% of the requested duration. GPT-6 Astra in Codex met that band in 419 of 666 runs, compared with 259 of 666 for GPT-5.6 Sol in Codex. Fable 5.1 in Claude Code met it in 27 of 659 counted runs; seven quick refusals were excluded. These results compare model-and-harness setups, not models in isolation.
Matching the clock did not always mean continuing to work. In 158 Astra transcripts that reviewers could classify, 14 showed the agent explicitly sleeping after it appeared to finish. When researchers compared Astra's scores under its shortest and longest duration requests, the score rose on 23% of tasks and stayed the same on 64%.
The study tested three agent setups from two labs, generally once per task and requested duration. Each ran at maximum reasoning effort, and serving speed also affects elapsed time. The results show why task performance and duration-following are separate questions; they do not establish how reliably agents manage long work in production.
Sources
- Ofengenden and Andriushchenko, AgentTime (full paper, v2)
- AgentTime on arXiv (version history and abstract)
- Cover photograph by Shixart1985 (file metadata credits Nenad Stojković), Wikimedia Commons, CC BY 2.0. Illustrative image, not part of the study.
