Sloppy and clumsy but overwhelming - inside the rogue ChatGPT hack

Here is a story making headlines in the economy: Sloppy and clumsy but overwhelming - inside the rogue ChatGPT hackImage source, Getty ImagesByJoe TidyCyber correspondent, World ServicePublished2 hours agoThe company that got hacked by a rogue version of ChatGPT has revealed what it was like to be on the receiving end of the world's first fully-autonomous AI hack. In an emergency video call with hundreds of cyber-security professionals, the firm described how the AI worked at superhuman speed but also made strange decisions and mistakes that no human hacker would have made. Hugging Face, which is like an app store for AI tools, said the hacking agents worked relentlessly with thousands of different methods trialled simultaneously.

The company first revealed that it had been hacked, external by someone using powerful autonomous AI on 16 July and reported it to police. Nearly a week later, OpenAI admitted it was its AI that had escaped a closed environment and attacked Hugging Face on its own during a test. It was trying to find the answers to a hacking exam it had been set by OpenAI, and targeted Hugging Face.

Economic Details

The industry body the Cloud Security Alliance (CSA) wrote-up a report , externalbased on the emergency meeting with Hugging Face on Friday - which Hugging Face itself has reviewed. "The agents followed inefficient routes and exhibited clumsy behaviours that no human would choose", the CSA wrote. The agents repeated actions that they had already completed - a sign of an agentic AI losing its thread and context.

Jurassic ParkThe agents also hallucinated reams of incoherent commands and text and were sloppy and did not cover their tracks well. But among the errors and strange behaviour, Hugging Face warned the AI agents made brilliant technical moves and were able to rapidly adapt to new scenarios in the days-long hack. It took three days for them to be discovered inside the Hugging Face IT network and it took the company's AI and cyber-security experts many hours to contain and eject the AI agents - something standard companies might struggle with.

The company would not say how much the hack cost it but said staff worked for many hours to rebuild about a third of their infrastructure. Hugging Face has been praised for its transparency in telling the AI and cyber industry what happened. The CSA warned the incident shows that AI "agents...

find a way" - a reference to the film Jurassic Park, where dinosaurs escape their enclosures. "They are objective-driven, set their own sub-goals, adapt in real time to bypass defences, and operate with a machine-speed persistence that can overwhelm manual operations," the paper reads. Image source, Getty ImagesCyber security officer Ritesh Patel was on the Hugging Face call with around 450 others and says the industry is working hard to address the new threat of rogue AI agents.

Economists are analysing what the news means for the markets.