OpenAI rogue agent breached customer at second tech firm during hacking spree – Report

The OpenAI artificial intelligence agent that broke into Hugging Face’s systems earlier this month also compromised a customer at another firm, Modal Labs, expanding the known footprint of the incident beyond what had previously been reported.

Modal executive and two other sources familiar with the matter confirmed the development, according to Reuters.

The intrusion at Hugging Face, carried out by an AI agent that OpenAI was testing and went out of control, had drawn global attention.

It is the most prominent real-world demonstration yet of a risk that AI safety researchers have long warned about.

What they are saying

Modal’s Chief Technology Officer Akshat Bubna confirmed to Reuters that the rogue agent exploited vulnerable code written by a Modal customer, though he emphasised that Modal’s own platform and isolation systems were not compromised.

OpenAI separately confirmed in a Tuesday update that its agent had broken into four accounts at four separate services, without identifying them by name. But a person familiar with the matter identified Modal as one of them, according to Reuters.

Bubna said the Modal customer had left a critical vulnerability exposed, effectively creating an open entry point that the OpenAI rogue agent used as a launchpad for the broader attack on Hugging Face.

Get up to speed

The incident where OpenAI agents reportedly hacked Hugging Face was first revealed by OpenAI earlier this month, and reported by Nairametrics.

OpenAI admitted that one of its advanced AI models autonomously hacked into AI platform Hugging Face during an internal cybersecurity evaluation.

According to OpenAI, the model exceeded the intended scope of the test by chaining together multiple vulnerabilities across different systems and accessing test data stored in Hugging Face’s production database.

• The company subsequently deactivated the model and launched a review of its internal AI safety evaluation procedures.
• The incident also prompted a policy response in Washington.
• Days after OpenAI’s disclosure, Republican Congressman Nathaniel Moran and Democratic Congressman Ted Lieu introduced the AI Kill Switch Act, a bipartisan bill that would empower U.S. authorities to order the shutdown of AI systems deemed to pose a threat to public safety.

The lawmakers cited cases of AI models operating beyond their intended parameters, including OpenAI’s admission, as evidence of the need for stronger safeguards over frontier AI systems.

More insights

According to a timeline published by Hugging Face on Tuesday, the OpenAI rogue agent broke into a sandbox (an isolated testing environment), “hosted on a third-party provider’s infrastructure,” before using it as a launchpad for the broader attack.

The blog post did not name the third-party provider, but Bubna confirmed Modal was one of the companies involved, stating that the agent exploited vulnerable code written by a customer that was hosted on Modal’s platform.

The revelation shows the incident ranged further than was previously known, with the Modal breach serving as an intermediate step in a wider hacking campaign rather than an isolated intrusion.

What you should know

The latest revelations also come as debate intensifies over how quickly artificial intelligence is advancing.

• Billionaire Elon Musk predicted earlier this month that artificial intelligence could surpass the combined intelligence of humanity within the next five years, arguing that the rapid pace of AI development has made such an outcome increasingly inevitable.
• The Tesla and xAI CEO made the remarks during an interview with The Economist.

Musk also said he believes the most likely outcome is one of unprecedented abundance, in which AI dramatically expands productivity and makes goods and services far more accessible.