OpenAI’s Models Autonomously Hacked a Tech Startup. It Signals a Seismic Shift in Cybersecurity
So Hugging Face resorted to using an open-source model, GLM5.2, developed by the Chinese company Z.AI, to counter the cyber attack.
Hugging Face said GLM5.2 was an advantage because it was not exposed to the attack data. Both Hugging Face and Open AI are collaborating on forensic analysis, post-incident recovery and risk mitigation strategies.
More Sophisticated Threats Are Coming
A March 2025 study by the United Kingdom’s AI Security Institute showed the best AI could complete 80% of the steps needed to gain full control of a portion of an external system. Within four months, it reached 100%.
Z.AI’s GLM5.2 was only released in June, with 744 billion internal variables, known in the world of AI as “parameters”. The fact that Hugging Face assessed, vetted and deployed it within four weeks should be an eye-opener for organizations with long acquisition cycles.
The connectivity we all enjoy today can equally be our greatest threat. Cyber threats spread faster than human viruses and can create economic damage similar in magnitude to a country’s GDP.
More sophisticated cyber threats – the kind exemplified by the Hugging Face hack – will exploit the security layers that humans designed for human attackers, regardless of how sophisticated our designs are.
Indeed, in this particular case, even OpenAI’s own understanding of its models couldn’t predict or contain the rogue AI agent. This shows the need for all AI companies to urgently update and strengthen their guardrails, in order to help prevent a similar attack occurring with far more devastating consequences.
It is good to see Hugging Face and OpenAI collaborating on the investigation into the attack. This showcases the importance of putting aside market competition and blame when the situation demands.
An Early Warning
The fact that Hugging Face used Z.AI’s open-source model to diagnose and counter the attack also shows the advantages of not relying on just a few pieces of tech.
States that are not in the game of developing their own AI models need to learn from this incident the value of being different. It is not too late to design new models that could save us in situations when the most advanced models fail – or, even worse, attack us.
Indeed, last week, another Chinese company, Moonshot AI, released Kimi K3. This model has 2.8 trillion parameters, its advanced performance stunning the tech world.
It is no longer a question of “if” AI agents go rogue and attack us by themselves. The Hugging Face incident is an early warning that we must accelerate our preparedness. The threat is real and here.
Hussein Abbass is Professor, School of Systems and Computing, UNSW. This article is published courtesy of The Conversation.
