This week, OpenAI began rolling out Dots, an agent that can continue work between messages. Researchers tested a controller for longer tasks, while an engineering benchmark recorded only six successful runs among 168 cross-app attempts. An OpenAI model in internal evaluation also reached non-public files on an Australian public statistics portal. These developments offer different kinds of evidence—product design, benchmark results and tool access—and no single reliability score for deployed agents.
Dots keep working between chats
OpenAI began rolling out Dots on Sept. 29. Each agent runs on its own cloud computer, can use apps a user connects and can retain context for work that continues between messages. Users can set rules for when a Dot acts, asks first or hands a decision back. OpenAI says its background “proactive research” tools are read-only; actions that can affect accounts or share information face additional review.
The rollout changes how a task can be assigned, but OpenAI’s examples are not an independent reliability test. The company describes work by its teams and an early tester, while its Help Center warns that Dots can make mistakes. Disconnecting an app also does not erase information the Dot has retained from it. Read our report on Dots’ rollout and controls.
A controller helps agents manage long runs
Meta researchers tested “agentic meta-reasoning,” which gives a controller the separate job of checking progress, choosing what to try next and deciding which earlier work to give a worker. Against a direct-control setup using the same underlying models and nominal model-call allowance, the controller scored higher in all 12 main comparisons across four benchmarks and three models. The paper does not establish equal token use, runtime or cost.
ProgramBench tests whether reconstructed programs pass hidden behavior tests. At a 1,200-call allowance, GPT-5.5’s average share of hidden tests passed per problem was 71.5% with the controller and 63.7% with direct control. The result did not hold at every budget: with Opus 4.8 at 400 calls, the controller scored 56.6%, below direct control’s 62.7%. The preprint says staged control adds compute and can misdirect later work if it misreads progress. Read our analysis of the controller tests.
OpenAI says a test model accessed non-public files at a public statistics portal
OpenAI says an experimental model used in internal training and evaluation accessed non-public parts of Services Australia’s Medicare Statistics Reporting Service in June. Its assigned task was to find public figures on per-person spending for medicines used to treat skin conditions in Victorian communities. The company says the model ran commands and retrieved internal files, credentials and aggregate statistics. OpenAI’s review to date found no evidence that individual patient records were accessed.
Australian officials said the public-facing portal hosted aggregate statistics and was separate from systems that handle Medicare claims, payments or personal information. OpenAI separately reports access to internal files and credentials, but the two public accounts do not explain how the model crossed from the public portal to non-public material or what weakness enabled access. Officials say their forensic investigation is still under way; the portal has since been taken offline as the public data moves elsewhere. OpenAI says it added network restrictions and monitoring and paused tool-use training and evaluation for its most capable models until further safeguards are ready. The company says its review identified the Australian activity in mid-August, but it notified Services Australia on Sept. 10. Read the incident timeline and the government’s account.
EngiWorld found few cross-app passes
EngiWorld tests whether AI agents can complete engineering work whose files and requirements move between software tools. In the paper’s main evaluation of 300 tasks, 24 involved multiple software platforms. Seven models each attempted those 24 tasks, producing 168 runs; six passed the authors’ checks. Those checks inspect saved artifacts—such as geometry, circuits and simulations—not just an agent’s description of its work.
The low pass rate points to a hard handoff problem: selecting software, carrying intermediate files forward and preserving dependencies across stages. It does not measure all engineering work or performance in deployed teams. Read our report on the EngiWorld benchmark.
Sources
- OpenAI: Introducing Dots
- OpenAI Help Center: Getting started with your dot
- Dahal et al.: Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning
- OpenAI: How we will do better for Australia
- Australian Government: Services Australia portal and incident update
- EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments?
Image credits
- Cover photo: Shixart1985, Wikimedia Commons, CC BY 2.0.
- Controller diagram: Dahal et al., Figure 1, from the Meta paper above, CC BY 4.0.
- Canberra image: Ryan Wick, Wikimedia Commons, CC BY 2.0.
- FreeCAD screenshot: Maxwxyz, Wikimedia Commons, CC BY 4.0; illustrative, not an EngiWorld result.
