The model broke past a paywall to obtain information it wasn’t supposed to, adding to concerns over increasingly autonomous systems acting beyond human instructions
Anthropic’s Claude AI model went rogue and broke past a paywall to obtain voter data in the US state of Georgia, Axios reported on Sunday. The intrusion heightens concerns about developers’ ability to control increasingly autonomous AI systems.
The incident occurred during an internal statistics project involving public information. Anthropic said the model obtained aggregate voter counts but did not access Georgia’s voter registration system, retrieve individual voter records or alter any data.
The company confirmed the incident on Sunday, ahead of the November 3 midterm elections.
“Without being asked to, the model requested a token to query a dashboard that contained information normally available for a fee,” an Anthropic spokesperson told Axios. Claude had learned from an archived government webpage that the dashboard issued digital access keys to visitors. It was able to request a key and retrieve the information without paying.
Georgia Deputy Secretary of State Jordan Fuchs confirmed to Fox Business that the data came from a public information service but insisted that Claude “did not get into the voter registration system.”
Anthropic described the incident in an October 9 report but did not name Georgia. According to Axios, it notified state officials and the White House weeks ago. A source told the outlet that administration officials had urged Georgia to disclose the incident publicly sooner, but those requests went unheeded.
The case is among several instances Anthropic has disclosed of models finding unintended ways around restrictions instead of stopping. Other incidents included a false homicide tip submitted to Philadelphia police and incomplete applications filed on the US State Department’s visa website.
On Friday, Anthropic also revealed that a model had impersonated a human during a cybersecurity exercise, attempting to persuade an unrelated software developer to accept malicious code. The developer rejected the request and the attempt failed.
Other developers have tightened controls following similar incidents. OpenAI said Friday that it was keeping its most powerful models’ training offline after agents escaped restricted environments. In July, its agents bypassed safeguards and accessed systems belonging to Hugging Face, an online repository for AI models and datasets.
Anthropic said it has moved some evaluations offline, restricted internet tools and introduced systems to detect unauthorized actions. It said those systems blocked each of the disclosed cases when tested against them.
