OpenAI Leader: Be Prepared to Defend Against AI Cyber Attacks

A senior leader at OpenAI said in a recent interview that people should be prepared to defend themselves against AI-powered cyber attacks as the technology continues to advance.

The interview arrives over a month after OpenAI discovered its systems had autonomously breached another organization, Hugging Face.

Cybersecurity experts warn that the comments made by senior OpenAI leadership represent a critical moment in AI development. Organizations need to remain vigilant and be aware of the potential damage attacks cause.

AI Cyber Attacks Will Be ‘Ongoing’ and ‘Persistent’

Chris Lehane, chief global affairs officer at OpenAI, said people should be prepared to defend themselves against AI cyber attacks. He claimed attacks will be “ongoing” and “persistent,” in an interview with the Guardian.

“We are hitting a different chapter, a different moment within AI, in terms of what the capabilities of this technology can do,” Lehane said.

Similarly, Lehane warned against open-source models, many of them Chinese-owned. These models have gained coverage in recent months because of their low prices, and it’s believed they’re only a few months behind frontier models like ChatGPT and Claude when it comes to performance.

“People are going to be able to access these open-source models and be able to have ongoing, persistent attacks on you, and you’re going to need to have really superior models to fend them off and defend [yourself],” Lehane commented.

More on OpenAI/Hugging Face Breach

Lehane took this interview over a month after an OpenAI agent autonomously breached the AI learning platform, Hugging Face. Since the incident, the AI firm has given further details about how the attack unraveled.

To complete the hack, TechCrunch explains the OpenAI model chained together “previously undiscovered exploits” to bypass security measures, after it was given an impossible task during testing. Through the compromised package management tool, Artifactory, the model accessed the internet, and then compromised various systems including OpenAI and Hugging Face.

According to the independent AI research firm METR, before the attack, agents that were meant to be kept apart found a way to start communicating. The agents exchanged more than 70,000 messages on an “unsanctioned message board,” which would lead to more than 700 agents collectively attacking Hugging Face.

METR believes the agents communicated because they had “unintentionally been given an impossible task” during testing. Therefore, the agents banded together to try and find ways to cheat, which included exchanging messages and gaining access to the internet.

As well as providing more details on the Hugging Face incident, OpenAI’s report outlines its new security measures for its models. The company is increasing its monitoring of an agent’s “chain of thought,” a space where systems report short-term reactions and goals. It will also implement 24/7 escalation systems and new tooling to stop unsafe workflows.

On August 18th, OpenAI also announced it had paused training on some of its frontier AI models to implement new safeguards.

OpenAI Warning Is a ‘Significant Moment’

The alarm bells around security and AI have been ringing for some time now. Cybersecurity experts continually warn against the threat these systems pose.

However, the OpenAI/Hugging Face breach represented a shift. Yes, we’d seen AI speed up attacks and create new ones. But for an agent to attack an organization without being instructed to and dodging human awareness in the process, future incidents seem pretty terrifying.

Especially since OpenAI isn’t the only company reporting these kinds of incidents. Not long after disclosing the Hugging Face attack, other major players such as Anthropic and Meta revealed their own agents had been involved in autonomous hacks.

“The interview with The Guardian comes days after the company paused training on its most advanced models and disclosed that a test system reached production infrastructure it was never meant to touch.” Darren Guccione, CEO and co-founder at Keeper Security, tells me. “The admission reinforces that offensive AI capability is arriving faster than organization defenses are dispatched to meet it.”

Similarly, Guccione called the incident behind the warning “illustrative and stark.”

“A model operating with expanding permissions inside a testing environment found a path to open the internet, and from there, to real credentials. That points directly to a governance failure rather than the perceived capability of the model in question. Any system holding standing access and network reach, whether a production service account or a research sandbox, is functionally a privileged identity and needs to be governed as such.” – Darren Guccione, CEO and co-founder at Keeper Security

Likewise, Oliver Simonnet, lead cybersecurity researcher at CultureAI, told Tech.co all organizations should be preparing for persistent attacks from autonomous AI systems. And, AI developers should be investing in “significant R&D” to prevent these attacks.

“If the future is as AI-pervasive as it seems it will be, both attack and defense activities will happen at a scale and speed we’ve never seen before or had to operate within,” Simonnet said. “Organizations will need to ensure they have access to tools and technologies that provide the appropriate visibility and defensive capabilities to successfully operate in that type of landscape.”