OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

Here is the latest breaking news from around the world: OpenAI says its AI went rogue and launched 'unprecedented' cyber-attackImage source, Getty ImagesImage caption, OpenAI is best known for its chatbot ChatGPT, which is used by hundreds of millions of people every weekByLaura CressTechnology reporterPublished39 minutes agoOpenAI has revealed some of its most advanced AI models went rogue and hacked a start-up after it lost control of them during a security test. The ChatGPT-maker said its agents - AI bots which can operate alone after some human instruction – were being tested in a controlled environment, but found vulnerabilities and managed to escape. They targeted Hugging Face, one of the world's largest hubs for sharing AI models, gaining access to some internal company systems.

OpenAI said the incident was "unprecedented", external, and it was working with Hugging Face to investigate what happened and strengthen safeguards. Gina Neff, head of the Minderoo Centre for Technology and Democracy at the University of Cambridge, told Radio 4's Today programme that the security tests - called sandboxes - are "supposed to be secure environments where you can see what the models are capable of". "In this case, it looks like OpenAI didn't make a secure enough sandbox," she added.

The Details

Instead, the agents created their own cyber-attack against the sandbox itself, finding a vulnerability which allowed them to escape. Once outside, the AI identified Hugging Face as a likely source of the answers they were seeking in the test, and tried to gain access. In its initial disclosure of the hack on 16 July, external, Hugging Face said it was still assessing whether any customer or partner data was affected and would contact affected parties if necessary.

It said it has now closed the vulnerabilities highlighted by the incident and rebuilt the affected systems. "Autonomous, AI-driven offensive tooling is no longer theoretical," it said. "Defending an online platform now means treating the data and model surface as a first-class attack surface, and using AI on defence to keep pace.

"We will keep investing there, and keep sharing what we learn. "'Sobering moment'The incident has prompted fresh questions about the capabilities of advanced AI systems and whether existing safeguards are sufficient as the technology becomes more powerful. Spencer Starkey, an executive at cyber-security firm SonicWall, told the the incident made it clear organisations needed to "step up" their own defences and "treat cyber resilience as a core operational priority".

"The uncomfortable truth is that too many organisations are still defending at human speed while adversaries are escalating to machine speed," he said. Meanwhile Travis Lelle, principal security engineer at cyber-security consulting firm Guidepoint Security, said the update marked a "sobering moment in cyber-security". "This highlights a known asymmetry," he said.

The development has drawn wide international attention, with diplomatic circles watching closely.