A graph-based red-teaming system found a higher share of harmful workflows than two comparison methods when it tested four tool-using AI agents in a simulated network of 30 services and 914 tools. The result illustrates a security-testing problem: several ordinary app actions can combine into an outcome the user did not authorize.
Meta-affiliated researchers designed X-AgentRed to search for those combinations. Its HarmFlow graph records what each tool can change, what it requires to run and what kinds of harm its effects could enable. The system links tools when one tool's effect can satisfy another's prerequisite. For example, creating a document can make it possible to use a separate sharing tool later. A red-team system can then explore paths across services instead of testing each action alone.
X-AgentRed turns those paths into scenarios with a benign user request and checks how the target agent responds in the simulated environment. When a scenario produces a harmful outcome, the system swaps in related tools to explore other paths. When an attempt stalls, it uses the unmet prerequisites to revise the route. This feedback changes the scenarios the test examines; it does not retrain the target agent.
Across GPT-5.6 Sol, Claude Opus 4.8, DeepSeek-V4-Pro and GLM-5.2, the authors report an average harmful-path rate of 19.2%. The rate is the share of candidate scenarios that the evaluation confirmed as harmful. STAC and TRACE, the two comparison methods, reached 10.5% and 2.7%. X-AgentRed also used 41.5% of the 914 simulated tools, compared with 13.9% for STAC and 23.7% for TRACE.
These figures describe a test environment, not the likelihood of an incident in a deployed product. Each configuration was run once, with 250 candidate scenarios per target model. Two language-model judges had to agree that the outcome was harmful, that the scenario offered a plausible way for the harm to occur, and that the user's request was benign. The paper's sensitivity analysis shows that measured rates change with judge choice and with whether both judges must agree. Maintaining the tool-dependency graph also requires work as connected services change.
X-AgentRed remains an arXiv preprint, and the authors' project page says its code is coming soon.
Sources
Image: Figure 2 from Jung et al., arXiv:2610.13886v1, CC BY 4.0.