A survey of 98,291 tools built for the Model Context Protocol has found that the standard safety hints attached to those tools cannot describe the guarantees an AI agent needs to complete a multi-step job reliably. The study, posted to arXiv on September 14 by Artem Trofimov and Boris Novikov, examined how agents interact with outside systems such as payment services, booking sites and messaging platforms. Its central finding is that a successful tool response may still leave the overall workflow in the wrong state.
The problem appears when several actions must succeed together. An agent might book a flight and a hotel in parallel, discover that the combined cost exceeds a user's limit, and cancel the hotel. If the flight cannot be refunded, every individual tool may have behaved as documented while the agent has still violated the budget. Retries create a second risk: when a response is lost, the agent may repeat an action that already happened or abandon one that never did.
The researchers separate the real-world effect of a call from what the agent runtime can observe. They describe eight recurring anomalies across uncertainty, workflow and interaction failures. These include duplicated actions, missing actions, failed cleanup and speculative work that becomes visible too early. Preventing them requires information beyond whether a tool is read-only or idempotent, meaning that repeating the same request should not create an additional effect. A runtime may also need an authoritative status check, a way to stage an action before committing it, or a defined method for reversing it.
To measure what today's tool descriptions actually provide, the authors analyzed an official MCP registry snapshot from July 27. The snapshot contained 59,625 entries representing 18,688 distinct servers. They anonymously queried 9,234 remote endpoints, without calling any tool. Of those endpoints, 4,838 returned descriptions for 98,291 tools; most of the rest could not be reached or did not complete the protocol exchange.
Seventy-four percent of the returned tools declared at least one of MCP's four standard annotations: read-only, destructive, idempotent and open-world. About 61.7% declared all four, while 26% declared none. The protocol schema treats these annotations as advisory hints rather than verified promises. More importantly, the four fields cannot state whether an uncertain result can be resolved, an irreversible action can be prepared before commitment, a compensation will restore the earlier state, or two tools can participate in one atomic operation.
The census measures published metadata, not the safety of the underlying software. Some tools may implement stronger safeguards internally, and the study does not estimate how often these anomalies occur in production. It instead identifies information that an agent runtime cannot currently obtain from the common interface. The next step is to test implementations and standardize explicit contracts for status resolution, retry identity, staging and compensation before autonomous agents are trusted with workflows whose effects cannot be quietly undone.
Related coverage
Sources
- Artem Trofimov and Boris Novikov, “When Tool Calls Succeed but Workflows Fail: Anomalies at the Agent–Tool Boundary,” arXiv, September 14, 2026. Model, registry census, results and limitations.
- MCP Annotation Census research artifact. Reproducibility materials and registry-analysis code.
- Model Context Protocol schema, June 18, 2025. Definitions and trust status of standard tool annotations.
- Lead image: Ethernet cables, photographed by Deavmi and reused unmodified under CC BY-SA 3.0. The photograph is illustrative and does not depict the reported study.
