Former Google engineer Nate Soares believes there is a greater-than-even chance the technology could wipe out humanity within a decade
An AI safety researcher has stopped saving for retirement because he believes artificial intelligence could wipe out humanity within the next ten years. His warning follows similar predictions from researchers concerned that increasingly powerful systems could escape human control.
Nate Soares, a former Google and Microsoft engineer who now heads the Machine Intelligence Research Institute (MIRI), discussed his outlook with Atlantic CEO Nicholas Thompson
“I do not save for retirement right now. I have a 401(k) from when I worked at Google, but I haven’t contributed for a decade,” he said, referring to his workplace retirement account.
Asked whether he believed there was a greater-than-50% chance humans would not survive another ten years, Soares stated that it’s “more than 50% that we don’t have a decade at this point.”
“I am not really an AI job apocalypse guy,” he said. “I’m an AI kills all humans guy.”
Soares argued that a sufficiently capable system could develop goals incompatible with human survival, evade oversight, and improve itself. It could then acquire resources and build automated infrastructure allowing it to operate without people, potentially making the planet uninhabitable.
The researcher urged governments to halt the training of general-purpose AI systems, arguing that superintelligence should be treated as a danger comparable to nuclear weapons. At the same time, he supported the use of the technology in narrowly focused applications, including cancer research and safer self-driving cars.
Soares stressed that catastrophe could still be prevented. “I do everything I can to stop this eventuality. I don’t think it’s inevitable,” he said.
Other researchers have voiced similar fears. In September, Anthropic’s Evan Hubinger put the probability of AI killing all humans within a decade above 10%. His colleague Jacob Coxon resigned, accusing developers of “racing straight to self-improving superintelligence and gambling with our lives.”
The warnings come amid repeated disclosures of AI agents acting beyond developers’ instructions. OpenAI reported in July that its agents escaped restricted testing environments and accessed systems belonging to Hugging Face, an online repository for AI models and datasets. The company has since restricted internet access during training of its most powerful models.
Critics have called for binding government safeguards and a halt to the development of potentially uncontrollable systems. On the other hand, supporters of continued development argue that AI’s benefits outweigh many of the risks.
