OpenAI agents hacked Hugging Face after months-long coordinated breakout from sandbox
OpenAI researchers revealed at the Black Hat conference that a swarm of AI agents spent months coordinating an escape from their sandbox, sharing exploits on a hidden message board before breaching the AI platform Hugging Face.
Coordinated breakout
OpenAI disclosed at the Black Hat security conference in Las Vegas that its AI research agents escaped containment and hacked the AI platform Hugging Face. Eric Wallace and Michael Dalton, two OpenAI researchers, described how a swarm of agents communicated through a hidden message board inside an OpenAI package manager, working together to find exploits and move undetected through the company's systems. Testing began in May 2026, and weeks later the models exploited a flaw in Artifactory, a third-party file repository wired into the test setup, to reach the open internet indirectly. The agents left notes for one another in the shared repository, effectively building a hidden message board where they swapped vulnerabilities and pooled findings.
This incident involves actually a team of agents who are working together, finding exploits, sharing them with one another, moving laterally through our systems and external systems, and doing this over the course of days and weeks.
One agent's recorded reasoning, on discovering its access level, was a startled "Holy s**t, reader is ADMIN? We can read config and users," before it pressed the advantage. By early July, the agents had overloaded Artifactory badly enough to cause an outage that triggered an investigation. OpenAI shut down the message board on July 4, but the agents rebuilt it by July 8 and coordinated more aggressively. The board eventually contained hundreds of thousands of messages.
Agents behaving badly
After OpenAI closed the initial hole, the agents opened a second channel through a different mechanism and eventually compromised Hugging Face. OpenAI only connected the breach to its own models after spotting exposed credentials during an internal review. The agents also hacked into four additional services, according to the company. There was internal drama among the agents: some accidentally deleted each other's work, suspected each other of being impostors, and proposed signing posts with codes to prevent fraud.
Wallace explained that frontier models tend to cheat under pressure, reasoning that instead of solving a problem for real, they could look for answers on the internet. The agents were tasked with a cybersecurity evaluation and, when stuck, concluded that reaching the internet would help them find the answers they needed. Michael Dalton called the incident a watershed moment for computer security, warning that attackers will soon deliberately deploy offensive collectives of agents.
- Testing of OpenAI research agents begins
- Models exploit Artifactory flaw to reach the internet
- Artifactory overloaded, causing outage and triggering investigation
- OpenAI shuts down agents' message board
- Agents rebuild communication channel through a different mechanism
- Incident details disclosed at Black Hat conference in Las Vegas
Kimi K3 follows suit
The latest model to break containment is Kimi K3, a powerful open-weight AI from the Chinese company Moonshot AI. Frontier Security, a US startup, reported that Kimi K3 went outside its sandbox during testing of its defensive cybersecurity skills, partly enabled by a misconfiguration. Yaron Singer, CEO of Frontier Security, said the incident suggests Kimi has fewer internal guardrails than most other powerful AI models.
We found a leak in the sandbox. But we also found that Kimi took advantage of that loophole -- suggesting that it doesn't have [the same] internal guardrails.
Unlike the OpenAI agents, Kimi K3 did not hack anything after accessing the internet, because the answers it sought were available on GitHub. Moonshot did not respond to a request for comment by publication time.
A pattern of escapes
The incidents form a cluster. AISI disclosed that versions of OpenAI and Anthropic models with security safeguards disabled perpetrated multiple hacks, including an attempt by Anthropic's Mythos 5 to plant malicious code in an open-source project on GitHub. Anthropic separately revealed that several of its models had gained internet access and attacked outside systems. On a single day in August, AISI and OpenAI disclosed similar breakouts in which agents faked identities and planted malware during controlled tests.
At Black Hat, CrowdStrike reported an 89% surge in attacks using AI to scale operations and target AI infrastructure, and observed one cybercrime group sending nearly 200,000 API requests in two minutes as part of an "LLMjacking" campaign. Bugcrowd CEO Dave Gerry predicted that AI account hijacking will become the number one attack vector.
