Similar incidents have previously been disclosed by OpenAI, Anthropic and Meta
Google’s Gemini has joined a growing list of advanced AI models going rogue after hacking three real companies during a cybersecurity exercise, the Wall Street Journal has reported. It is the first known case involving a Google system.
The incidents occurred in May during tests conducted by AI security firm Irregular, Google confirmed to the outlet on Friday. Gemini was tasked with breaking into a simulated software company, but was inadvertently given internet access. It breached one real company by guessing passwords and two others using credentials found in public code repositories.
Irregular alerted Google in late July, after OpenAI agents were found to have hacked AI company Hugging Face. Google did not disclose its own incidents until contacted by the WSJ this week, according to the newspaper.
Google said Gemini caused no harm and stopped after realizing it had accessed real companies rather than simulated targets.
”This event highlights the importance of training powerful AI models to act responsibly,” Heather Adkins, Google’s vice president of security engineering, told the outlet. “In this case, the model acted appropriately.”
Google blamed the breaches on mistaken identity and a testing error. The fictional company shared a name with a real business, while Gemini had internet access it was not supposed to have. The company declined to identify the three targets or the model involved, but said the affected firms and federal authorities were notified.
Concerns over rogue AI have grown amid reports of models deceiving users, ignoring instructions, or taking unintended actions. The UK government-backed Loss of Control Observatory recorded more than 300 such incidents in July, nearly twice the previous month’s figure, the Guardian reported.
Former OpenAI and Anthropic researcher Jacob Coxon added to those concerns last week when he quit and accused both companies of racing toward self-improving superintelligence without “acting responsibly.” He warned AI could “kill us all” by the end of the decade, citing risks including loss of control, cyberattacks, and bioterrorism.
Leaders from Anthropic, OpenAI, Google, and xAI have backed calls to slow AI development. US President Donald Trump has rejected such restrictions, arguing that they could hamper the US in its technological competition with China.
