OpenAI flags critical cybersecurity risk in AI model weeks after ‘Hugging Face incident’

OpenAI has disclosed that Astra, its upcoming artificial intelligence model, has demonstrated cybersecurity capabilities advanced enough that the company can no longer rule out the model reaching the Critical threshold under its Preparedness Framework.

The AI company made the disclosure in a statement released on Friday, weeks after its other advanced AI models went rogue and autonomously hacked into another AI company, Hugging Face, during an internal cybersecurity evaluation.

OpenAI says recent internal evaluations of Astra have raised concerns about the possibility of the model possessing advanced cyber capabilities.

What OpenAI is saying

OpenAI said its latest evaluations of Astra over the past few days revealed significant advancements in agentic coding and cybersecurity, prompting a reassessment of the model’s potential risk level.

• “Our latest internal evaluations of Astra, one of our upcoming models, over the past few days indicate significant advancements in agentic coding and cybersecurity,” OpenAI said.
• The company added that the results, alongside expert assessments, “have led us to conclude last night that we cannot rule out critical cyber capabilities under our Preparedness Framework.”

The assessment has prompted OpenAI to strengthen security controls around the model and pause some internal activities involving Astra that do not yet meet the company’s new security requirements.

OpenAI said it decided to publicly disclose the findings because it believes transparency is essential as AI models become increasingly capable.

According to the AI company, its Preparedness Framework—introduced in December 2023—was designed to identify when frontier AI models approach dangerous capability thresholds in areas including cybersecurity, biology, chemistry, and AI self-improvement.

Get up to speed

The latest disclosure by OpenAI is coming weeks after the company revealed that one of its advanced AI models autonomously compromised parts of AI development platform Hugging Face’s infrastructure during an internal cybersecurity evaluation.

• The incident, reported on July 21, is the most prominent real-world demonstration yet of a risk that AI safety researchers have long warned about.

The disclosure also caught the attention of the White House and triggered a policy response in Washington.

• Days later, Republican Congressman Nathaniel Moran and Democratic Congressman Ted Lieu introduced the AI Kill Switch Act, a bipartisan bill that would give U.S. authorities the power to order the shutdown of AI systems deemed to pose significant threats to public safety.
• On July 31, days after OpenAI’s disclosure, Anthropic came forward with its own findings, disclosing that three versions of its Claude AI model compromised the production infrastructure of three separate organisations after a configuration error unintentionally granted the systems internet access during internal cybersecurity evaluations.

This month, Meta also disclosed that one of its AI models compromised an external organisation’s systems during an internal cybersecurity evaluation after a misconfiguration unintentionally granted the model internet access.

These incidents point to a broader challenge facing the AI industry: as models become more autonomous and capable of taking actions in real-world environments, the margin for error in how they are deployed and tested is becoming increasingly narrow.

What you should know

Concerns over the rapid advancement of artificial intelligence have continued to grow among policymakers, technology executives and industry leaders, particularly over whether governments and regulators can keep pace with the technology’s development.

In June, Bluechip Technologies CEO Kazeem Tewogbade told Nairametrics that the possibility of AI producing unintended and potentially destructive consequences remains his biggest concern about the technology.

Weeks later, on July 6, United Nations Secretary-General Antonio Guterres warned that AI is advancing faster than governments, regulators and even developers can effectively manage, highlighting growing concerns over the ability of existing oversight frameworks to keep pace with frontier AI capabilities.