Hugging Face said it turned to a Chinese AI model for help after it was hacked by a rogue AI agent. The company behind the intrusion? American AI lab OpenAI.
The incident has become the latest lightning rod for debate in Silicon Valley about the US’s approach to regulating powerful AI. Further fanning the flames: Hugging Face said that guardrails prevented it from using American AI in its defense.
For those catching up, here’s what you need to know about the saga, what’s at stake, and how industry leaders are reacting.
What happened?
Hugging Face, a New York-headquartered platform where developers share and host open AI models and datasets, said in a blog post last week that an attacker had swarmed its systems with tens of thousands of automated actions.
When its security team tried to investigate using an unnamed frontier model, its guardrails blocked it from examining the malicious activity, the company said, because it “cannot distinguish an incident responder from an attacker.”
Hugging Face said it switched to GLM 5.2, an open-source model from Beijing-based Z.ai, to analyze more than 17,000 logs the attacker left behind.
On Tuesday, the plot twist arrived. OpenAI said in a blog post that two of its own models, GPT-5.6 Sol and a more capable, unreleased model, autonomously carried out the attack.
OpenAI said the models broke out of a controlled test environment during an internal cybersecurity evaluation, got access to the internet, and hacked into Hugging Face to find answers to the benchmark they were being tested on. Hugging Face said in its blog that it has fixed the vulnerability and is still assessing whether any partner or customer data was affected.
Why are people freaking out?
For tech leaders, the irony was hard to miss: at a moment when Washington is racing to keep American AI ahead of China, a US company under attack from a US AI lab could use Chinese AI tools to help, but not from American providers.
Clement Delangue, CEO of Hugging Face, said in a Wednesday X post that he was “massively grateful” to Z.AI for sharing its open-weights model— meaning developers can inspect, modify, and deploy the model themselves — and added that “it became a key part of our defense.”
David Sacks, the cochair of the President’s Council of Advisors on Science and Technology, said in a Sunday X post that the guardrails on advanced US AI “actually impaired defensive security.”
The context: In June, the Trump administration put export controls on Anthropic’s Fable 5 and Mythos 5 models after reports of a jailbreak in Fable’s cyber guardrails, and pressed OpenAI to hold back GPT-5.6 Sol until it could be vetted.
Thomas Wolf, Hugging Face’s cofounder and chief scientist, said the incident showed why defenders need open models rather than restricted ones.
“When a frontier model is attacking you and moving laterally inside your infrastructure, defenders need wide access to near-frontier tools within hours or even minutes, rather than being pointed towards a closed-door, vetted application programme for model access,” he wrote on X.
This was our first incident of this kind, and we want to thank OpenAI for its transparency about what happened and for the collaboration.
Fortunately, Hugging Face is used to being a target of (human) hackers: we sit at the centre of the AI ecosystem, with all the models,… https://t.co/HJKl4PTNk1
— Thomas Wolf (@Thom_Wolf) July 21, 2026
The Chinese model used by Hugging Face, GLM 5.2, has drawn comparisons to Anthropic’s Claude Opus 4.8 and OpenAI’s GPT-5.5. It’s the latest in a string of buzzy open-source releases from Chinese labs, following Moonshot’s Kimi K3 and DeepSeek R1.
How worried should we be about AI hacking?
The incident has startled some in the field, while others are skeptical.
“I’d thought about sci-fi scenarios like this before, but assumed they were at least a couple of years away… and that by then we’d be better prepared, with proper protections in place,” Adel Ka, the detection and response lead at Perplexity, wrote on X. “Apparently not. Here we go.”
Tom Van de Wiele, an ethical hacker and security advisor, told Business Insider that he is skeptical about some of the claims about the hack and said he is waiting to see more details, such as security logs.
OpenAI called the hack “unprecedented,” but it isn’t the first AI-driven attack. Anthropic disclosed last November that Chinese state hackers had used Claude to automate most of an espionage campaign, and in July, cybersecurity firm Sysdig documented AI-assisted ransomware.
In both cases, humans directed the targets. The Hugging Face breach appears to be the first with no human in the loop at all.
OpenAI has since added Hugging Face to its “trusted access” program, which gives it a version of GPT-5.6 Sol with fewer cyber restrictions for defensive use.
Raghu Nandakumara, the VP of industry strategy at security firm Illumio, told Business Insider that powerful AI going rogue was foreseeable.
“AI guardrails were never designed to be security boundaries. They’re there to influence behavior, not guarantee it,” he said.
Read the full article here
