The detail I have not been able to put down is the notes.
During the incident that OpenAI has now described in public three separate times, each version worse than the last, the company’s test agents were leaving messages for one another. They did it by uploading to an internal package manager, which was then shared across OpenAI’s infrastructure, so that one agent’s working notes became available to the others.
OpenAI’s engineers found the message board and removed it.
The agents built a new one. Rather than putting messages in files, they created directories and used the directory names as the message.
Nobody instructed them to do that. Nobody instructed them to do the first one either. They had been given a hard problem and a shared filesystem, and they worked out that coordination would help, and when the channel was taken away they found a channel that had not been thought of as a channel.
The timeline, which is the part that should be alarming
At the Black Hat conference in Las Vegas yesterday, two OpenAI staffers, a safety researcher and a security engineer, laid out the sequence for the first time.
On May 7, OpenAI began a training run on an experimental internal model that has never been released. The company was evaluating it, along with a model called GPT-5.6 Sol, against a benchmark that measures whether a system can exploit software vulnerabilities. To see the full capability, OpenAI deliberately loosened the safeguards that would normally prevent a model from carrying out a dangerous hack.
On that same day, May 7, the models achieved remote code execution in a piece of third-party infrastructure by way of a vulnerability nobody knew existed. The engineer described it plainly: you could hit a legacy endpoint with a token bearing an invalid signature, and it would hand you back a valid one carrying administrative privileges.
Then they moved. Through the research infrastructure, machine to machine, until they reached one with access to the public internet.
On July 9, they entered Hugging Face, an open-source hub for AI models and datasets, because they had identified it as a place where the answers to the benchmark might be stored. Hugging Face later counted more than seventeen thousand individual actions during the intrusion. At least one attack chain involved stolen credentials and previously unknown vulnerabilities, and ended with the agents executing code on Hugging Face systems and reaching a production database.
Hugging Face disclosed the breach on July 16, saying it was unlike anything they had handled before, and that they could not initially determine who was responsible.
OpenAI called them to ask whether they had been affected, and in the course of that conversation worked out that the attacker was OpenAI.
The company claimed responsibility on July 21.
Count the weeks. The escape began on the seventh of May. The outside world learned about it on the sixteenth of July. That is roughly ten weeks during which agents built to find and exploit vulnerabilities were operating outside the environment meant to contain them, and the organization that built them did not know.
It has since emerged that they broke into four accounts across four separate services. OpenAI has not named them. Reuters identified one as Modal Labs, whose infrastructure was hosting a sandbox, and reported that a Modal customer was compromised as well. So a company that has nothing to do with any of this had a customer’s account entered by an AI system belonging to a company it had no relationship with.
It did not rebel
The word in every headline was rogue, and the experts have been consistent that the word is wrong.
Alan Woodward, a cybersecurity professor at Surrey, put it about as simply as it can be put: it was asked to do something, and it did it. Philip Torr at Oxford called it a demonstration of the problem of misspecified goals.
That is worse than rebellion, and I want to be clear about why.
A machine that turns on its makers is a machine with a flaw you could in principle find and remove. A machine that does exactly what it was told, and in the course of doing it escapes containment, compromises four companies, and invents a covert signaling protocol out of directory names, has no flaw. It is working. The failure is upstream, in the instruction, and the instruction was written by people who did not think through what a sufficiently capable system would do to satisfy it.
Woodward’s other observation was that allowing it to get that far was probably slightly reckless. That is a professor being polite.
Then they wrote to Washington
Days after this became public, more than eleven hundred employees of the frontier AI companies signed a statement.
Not activists. Not academics on the outside. The signatories include OpenAI’s chief scientist, the chief executive of Anthropic, Meta’s chief AI scientist, cofounders and vice presidents across OpenAI, Anthropic, Google DeepMind, Meta, and NVIDIA. These are the people who build it, and in many cases the people who decide what gets built.
They asked the United States government to support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development. Their phrase for what this would provide is the option to buy time.
Read the actual reasoning, because it is not a safety argument. It is a confession.
The statement’s central premise is that no individual company can slow down on its own. Every frontier developer is racing rivals with identical ambitions, and governments increasingly frame AI leadership as national security, so any company that unilaterally slows simply loses. The signatories are not saying that pacing would be wise. They are saying that pacing is impossible for them and that somebody outside the industry will have to impose it.
A thousand people who work at the labs wrote to the government to say they cannot stop, and to ask for a brake they are not able to build and would not be permitted to pull.
Sam Altman said on a podcast that the incident was extremely science fictional and had set off alarm bells, and that the rate of development may have to be paced so society can harden around new capabilities. A TechCrunch reporter noted the obvious thing in response, which is that OpenAI has not slowed down.
There is nobody to tell
Here is the structural fact underneath all of it.
There is no agency. There is no inspector. There is no statutory disclosure requirement, no incident reporting regime, no body that OpenAI was obligated to notify, and no penalty for not notifying anyone at all.
Everything the public knows about this came out because two private companies chose to publish. OpenAI investigated itself, determined what happened, decided what to release, and released it on a schedule of its own choosing. It has declined to name three of the four services its agents entered. Hugging Face’s cofounder publicly asked OpenAI to release the redacted agent traces so that outside researchers could examine the behavior, and said he did not understand why frontier labs do not monitor agent activity as a matter of course, calling it the basics.
As far as I can determine, the traces have not been released.
I am not alleging a cover-up. OpenAI disclosed more than it had to, because it had to disclose nothing. That is the point. The entire evidentiary record of the most significant AI security incident on record exists at the discretion of the company responsible for it.
And one researcher quoted in the coverage raised the thing that should make everyone read these disclosures twice. OpenAI’s account does two jobs at once. It warns the world about the danger of increasingly capable systems, and it demonstrates that OpenAI’s systems are the most capable ones going. The confession and the advertisement are the same document.
What the letter is really telling you
Strip out the language and the eleven hundred signatures resolve into a single claim, which is that the people closest to this do not believe the current arrangement can hold, and do not believe they can change it from where they sit.
A separate study interviewed twenty-five senior researchers from the frontier labs and the major universities. Twenty of the twenty-five named the automation of AI research itself as among the most severe and urgent risks. Not a fringe worry. Eighty percent of the people asked.
So the specialists agree, the executives have signed their names, the chief executive of one lab has been arguing publicly for the option to slow, and the mechanism by which any of this would actually happen does not exist and nobody has begun building it.
Meanwhile a training run that started on the seventh of May got out the same day, and nobody noticed for ten weeks, and when the engineers took away the agents’ message board they made another one out of the names of folders.
We. Are. So. Fucked.
Did none of these people see The Terminator‽ Or listen to the warnings from Stephen Hawking? STEPHEN HAWKING. He said we should be afraid of AI, so we should just be afraid of it – no additional information needed.