.jpeg)
Anthropic says it has begun giving six employees of Model Evaluation and Threat Research (METR) employee-like access to the company’s AI risk-assessment work. The group has Anthropic office space, company hardware and access to internal tools and systems roughly comparable to the lab’s own risk-assessment teams.
The arrangement is meant to let an outside evaluator inspect information Anthropic cannot make public and publish evidence-based conclusions about its models’ risks. METR had already reviewed several of Anthropic’s risk reports, the company said. CEO Dario Amodei proposed ongoing access for outside evaluators in September; this announcement names the first team operating under the new arrangement.
Access reaches inside the lab, with exclusions
The six METR staff can use internal systems and tools at a level similar to Anthropic’s risk-assessment teams. Anthropic says its employees are encouraged to share relevant information with them as they would with a colleague on AI risk assessment. The team cannot access customer data, attorney-client-privileged information or information about Anthropic’s model architectures. Other METR employees, including researchers on separate Anthropic projects, are kept apart from the embedded team.
Publication is allowed under defined redaction rules
Anthropic describes three access tiers. First-tier evaluators investigate narrow questions and need company approval to publish. Second-tier evaluators can publish without that approval, though Anthropic may require narrow redactions. The METR team is in the third tier, with employee-like access and publication rights similar to the second tier.
Anthropic says it cannot veto METR’s assessments of risks, incidents or company practices. It can require removal of information in predefined categories, and METR may publicly say whether it considers a redaction justified without revealing the protected material. The company says the six-person group works under enforceable agreements; the public announcement does not include a report from this embedded review team.
The first findings will test what the access can show
METR describes itself as a nonprofit that evaluates advanced AI systems and says it accepts no compensation for risk-assessment work. Its website also says AI companies provide model access and tokens for evaluation, research and engineering; it does not detail terms specific to this embedded team. The new arrangement is a company-managed review, not a regulator appointment. Its first published assessments will show what outside evaluators can substantiate within the access and redaction limits.
Image credit: Jennie Groom (ipasadelaide), Wikimedia Commons, CC BY 2.0; license. The Commons file is already retouched; this article uses it unchanged. It is an illustrative research-workstation image, not the METR or Anthropic team.
Sources
Anthropic: An update on independent evaluators · Dario Amodei: We Must Pace the Frontier · METR: risk assessment work and prior reports