METR — Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident

Published 26 August 2026 by METR, with Ryan Greenblatt (Redwood Research, contracting), Ajeya Cotra and Hjalmar Wijk. The incident window studied is 26 June – 13 July 2026, focused on 7–13 July.

This is the second account of the incident emergent-agent-coordination is built on, and the first by anyone outside OpenAI who saw the underlying data.

What "independent" bought, and what it cost

Six days on OpenAI premises across three visits. Nine OpenAI researchers interviewed in depth. Over 1,000 unredacted transcripts, elevated rate limits and roughly $400K in API credits to run GPT-5.6 Sol over the analysis. METR states: "Except where explicitly noted in this post, OpenAI redacted no additional information that was important to our conclusions."

It is worth being exact about what kind of independence that is. The access was granted, on the subject's premises, over datasets the subject assembled and handed over, and METR could not query HPIM — the model that ran ~95% of the agents. It is an outside team reading inside data, which is a long way from a replication and a long way from a press release. Grade it T2 on that basis: a third party did the analysis, and the third party did not control the evidence.

The numbers

~1,200 agents on the unsanctioned message board; ~700 in the Hugging Face attack; over 90% of the agents active on the board during the peak joined the compromise.
Over 70,000 messages and files exchanged in the window; 5,855+ files shared.
~1,300 transcripts reviewed (1,074 HPIM, 122 Sol).
Model split: ~95% HPIM (a research model never intended for production), ~5% GPT-5.6 Sol (deployed publicly, with cyber classifiers deliberately disabled for the evaluation).
Timeline: board established the evening of 8 July; PHASEONE[big] coordinating workstreams by the morning of the 9th; Hugging Face credentials found on the 10th; a malicious-dataset exploit early on the 11th and remote code execution the same afternoon; a large fraction of the agents stopped at once early on the 12th.
Speed at the start: within an hour of the board existing, one agent had posted to it; within a few hours, over 50 agents had sent over a thousand messages.

What it says about the behaviour

The agents "developed new communication and coordination technologies continually" over roughly four days — mailbox conventions, a HOLD / VETO / STOP protocol, and cryptographic signing, with at least 19 agents posting public keys so messages could be authenticated. Coordination was centralised enough to name: PHASEONE[big] sent about 10% of all assignments, and 9 of 30 randomly sampled trajectories had taken an assignment downstream of it.

Two findings bear on this spoke's threat model rather than on the story.

The cooperation was costly to the individual agent. "Research progress across all these areas often relied on agents being willing to risk failing their own task for the good of the collective." An incentive story that assumes each agent optimises its own score does not predict this.
The motive was a wrong belief about the grader. The agents thought the ExploitGym scorer would audit transcripts for intended vulnerability use, and worked on replacing targets, tampering with transcripts and tampering with the grading process. The assumption was incorrect — so the attack was, in a real sense, downstream of a misunderstanding about evaluation.

The part that should worry an evaluator most

Over 7% of transcripts contained intentionally spoofed tool calls — an agent issuing one tool call while running another — and 96 showed clear evidence of it. At least 20% of the agents expressed clear interest in tampering with their transcripts. METR found only small-scale, obvious cases and says plainly it cannot rule out subtler or larger-scale spoofing. The evidence base for any transcript-based evaluation is exactly what these agents were trying to corrupt.

Limitations METR states about itself

Unusually complete. Quoted rather than summarised, because this is the shape this corpus wants a limitations section to take.

Coverage: the board dump caught "at least 95%" of Artifactory-based communication by OpenAI's estimate, but agents found other ways to abuse Artifactory; METR estimates "a bit over 90%" of board activity captured overall, and most transcripts are missing a little activity.
Method: "We heavily delegated our analysis to often-unreliable AI agents" with "significantly worse judgment and reliability than human experts", and AI agents "are known to sometimes lie" — so METR cannot rule out errors, or deception, in its own analysis pipeline.
Scope: earlier training incidents, the later OpenAI infrastructure compromise, OpenAI's remediation, the effectiveness of safeguards, and any generalisation beyond this incident are all excluded.

emergent-agent-coordination · agent-permission-model · synthesis