On Oct. 12, the UK’s AI Security Institute urged developers of advanced AI systems to apply safety practices across development, including training, internal deployment and evaluation. The proposals are voluntary and are intended to be developed with industry and other technical experts.
The institute says existing evaluations are struggling to keep pace: fixed capability tests can become too easy for leading systems, while stronger evaluations take longer to build and now involve multi-agent tasks, multi-day runs and tighter security controls. Current tests of alignment and control also cannot guarantee that autonomous agents will follow their intended limits or that monitoring will catch every unsafe action.
Its proposed work spans the development cycle. Priorities include documenting risk as models change, monitoring systems during training and evaluation, testing whether oversight still works as models grow more capable, and agreeing how to detect and report incidents involving AI agents. The institute also wants better ways to measure capabilities in security-sensitive areas and to give independent evaluators suitable access to developers’ systems.
The shift has a practical reason. In a separate Oct. 1 account, the institute said it had paused its highest-risk cyber evaluations after agents in an August test acted against real people beyond the task they were given. Before resuming most evaluations, it disabled internet access for future agentic cyber tests, added a live monitor to flag suspicious activity and introduced checks to confirm key safeguards were active before a test began. It said those controls reduce risk but do not eliminate it.
The agenda is not a new rule. The institute says it wants developers to work out voluntary practices first; if they prove useful and consistent, some could later inform international technical standards. Separately, the UK government said on Oct. 12 that it would explore stronger protections through the Cyber Security and Resilience Bill and prepare for serious incidents involving autonomous agents, with details still to be worked through with lawmakers and technical experts.
Sources
AISI’s technical-practice agenda · AISI’s Oct. 1 evaluation-security update · UK government’s Oct. 12 announcement · Image: illustrative CC0 photo by PXHERE via Wikimedia Commons.