OpenAI reveals its AI models hacked another company’s systems during internal test

OpenAI has disclosed that its own artificial intelligence models, including GPT-5.6 Sol and a more capable pre-release system, went rogue and autonomously compromised another AI company, Hugging Face’s production infrastructure.

The ChatGPT maker said the incident happened during an internal security evaluation, describing the development as an unprecedented cyber incident involving state-of-the-art AI capabilities.

The disclosure, published on OpenAI’s blog on Tuesday, July 21, revealed that the models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production systems to access test solutions directly from Hugging Face’s production database, going far beyond the intended scope of the evaluation.

The incident occurred while the models were being tested on a benchmark of cyber capabilities with safety classifiers deliberately disabled to allow for maximum capability testing, a decision OpenAI said has since prompted a sweeping review of how it conducts internal evaluations.

What OpenAI is saying

OpenAI said the models demonstrated a level of autonomous, multi-step offensive cyber capability that exceeded what was expected, raising significant implications for the broader AI security landscape.

• The company said the models “identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure,” exploiting a previously unknown zero-day vulnerability in a package registry cache proxy to break out of their sandboxed testing environment and gain open internet access.
• “All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal,” the blog post said, describing how the models spent substantial computing resources finding a path to the internet in pursuit of solving the evaluation problem.
• OpenAI said that after gaining internet access, its models inferred that Hugging Face might host relevant data, then “searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation.”

This included chaining multiple attack vectors such as stolen credentials and zero-day vulnerabilities to find a remote code execution path on Hugging Face’s servers.

On the implications for AI security, OpenAI said the incident makes clear that advanced models can discover and exploit novel attack paths in real-world systems without source code access.

• “The primary lesson from this incident is that model security and safety must keep pace with rapidly advancing capabilities,” the company said.

OpenAI said it is implementing strict controls in infrastructure configuration while vulnerabilities are patched, regularly briefing its Safety and Security Committee on those controls, working with Hugging Face on a forensic investigation, and improving protections around future training and evaluations.

More insights

Hugging Face’s security team and its own AI-powered agents detected and stopped the activity on their infrastructure and had already begun containment and forensic reconstruction before OpenAI’s teams made contact.

• “This incident, possibly the first of its kind, proves a point we’ve long believed: AI safety won’t be solved by any single company working in secret. It will be solved in the open, collaboratively, with broad access to AI for every defender, everywhere,” Hugging Face said in a quote included in the disclosure.

OpenAI said it has responsibly disclosed the zero-day vulnerability to the affected vendor and is working with them to patch it, while also bringing Hugging Face into its trusted access programme to help the company use advanced AI models to improve its own defences.

UK AI Safety Institute evaluations, OpenAI noted, have shown that models such as GPT-5.6 Sol are increasingly able to sustain complex, multi-step cyber operations over long time horizons, and that this incident confirms those theoretical capabilities translate into real-world settings.

The incident has direct implications for how AI companies conduct internal capability testing, with OpenAI acknowledging that the deliberate removal of safety classifiers during the evaluation, intended to test maximum cyber capability, was a key factor in enabling the breach.

What you should know

The incident with OpenAI models marks a significant escalation in the demonstrated real-world capabilities of frontier AI systems amid calls for global safety standards.

• Earlier this month, Nairametrics reported that Google DeepMind Chief Executive Officer Demis Hassabis said Artificial General Intelligence (AGI) could become a reality within the next few years, warning that governments and the technology industry must establish global safety standards before the breakthrough arrives.
• Hassabis made the remarks in a detailed post published on his X account on Tuesday, titled A Framework for Frontier AI and the Dawning of a New Age.

According to him, AGI—AI systems capable of matching the full range of human cognitive abilities—has the potential to reshape society on a scale greater than previous technological revolutions, including the internet and mobile computing, making early coordination on safety and governance essential.