Summary
Anthropic says it has onboarded six METR employees with access to internal tools and systems roughly on par with its risk-assessment teams. METR will be able to publish its risk analyses, subject to narrow, predefined redactions.
Anthropic says it has begun giving six employees from the nonprofit METR employee-like access to its internal systems for assessing AI risks. In an October 15 update, the company said METR will publish assessments of its risk levels, incidents or practices, with limited rights for Anthropic to require redactions.
The arrangement is intended to let outside evaluators examine information that a company may not be able to release publicly, including details about training or safeguards. Anthropic says the goal is to narrow the gap between what an evaluator can understand about a frontier AI system and what the company itself knows about it.
What METR can access and publish
Anthropic has provided the six METR employees with company hardware, office space and access to internal tools and systems roughly on par with those used by its own risk-assessment teams. Anthropic employees are encouraged to share information with them as they would with colleagues working on alignment risk, subject to defined exclusions.
Those exclusions include customer data, attorney-client privileged information and information about model architectures. Anthropic says architecture information may be shared when it is highly relevant to risk assessment, but that requires explicit internal approval. The METR group operates under enforceable agreements, and other METR employees—including people working with Anthropic on separate projects—are siloed from its work.
METR can publish its findings without Anthropic’s approval. Anthropic can require redactions for narrow, predefined categories, but METR may publicly describe those redactions and say whether it considers them justified, provided it does not disclose the restricted information. This gives the evaluator publication rights while retaining boundaries around specified sensitive material.
How the arrangement fits Anthropic’s evaluator programme
Anthropic describes the METR partnership as its third tier of evaluator access. In the first tier, the company sets the scope and access, and publication requires its approval. Second-tier evaluators receive more information and can publish without approval, although Anthropic can still require specific redactions. The METR arrangement combines employee-like access with publication rights similar to those in the second tier.
Other partnerships are at earlier stages, with more limited access and publication rights. SecureBio is reviewing the chemical and biological weapons chapter of Anthropic’s August Risk Report, following a completed and published pilot review of an earlier report. Anthropic also says it is working with Accenture and Faculty on model evaluation and red-teaming, alignment assessments, safeguard testing and capability-building.
The practical significance will depend on the assessments METR publishes: access alone is a process change, while the evaluator’s findings can give outsiders evidence about risks and company practices. Anthropic says it intends to share further updates as it develops these partnerships and hopes to embed more evaluators over time.