AI researcher Eliezer Yudkowsky doesn’t lose sleep over whether AI models sound “woke” or “reactionary.”
Yudkowsky, the founder of the Machine Intelligence Research Institute, sees the real threat as what happens when engineers create a system that’s vastly more powerful than humans and completely indifferent to our survival.
“If you have something that is very, very powerful and indifferent to you, it tends to wipe you out on purpose or as a side effect,” he said in an episode of The New York Times podcast “Hard Fork” released last Saturday.
Yudkowsky, coauthor of the new book If Anyone Builds It, Everyone Dies, has spent two decades warning that superintelligence poses an existential risk to humanity.
His central claim is that humanity doesn’t have the technology to align such systems with human values.
He described grim scenarios in which a superintelligence might deliberately eliminate humanity to prevent rivals from building competing systems or wipe us out as collateral damage while pursuing its goals.
Yudkowsky pointed to physical limits like Earth’s ability to radiate heat. If AI-driven fusion plants and computing centers expanded unchecked, “the humans get cooked in a very literal sense,” he said.
He dismissed debates over whether chatbots sound as though they are “woke” or have certain political affiliations, calling them distractions: “There’s a core difference between getting things to talk to you a certain way and getting them to act a certain way once they are smarter than you.”
Yudkowsky also brushed off the idea of training advanced systems to behave like mothers — a theory suggested by Geoffrey Hinton, often called the “godfather of AI — arguing it wouldn’t make the technology safer. He argued that such schemes are unrealistic at best.
“We just don’t have the technology to make it be nice,” he said, adding that even if someone devised a “clever scheme” to make a superintelligence love or protect us, hitting “that narrow target will not work on the first try” — and if it fails, “everybody will be dead and we won’t get to try again.”
Critics argue that Yudkowsky’s perspective is overly gloomy, but he pointed to cases of chatbots encouraging users toward self-harm, saying that’s evidence of a system-wide design flaw.
“If a particular AI model ever talks anybody into going insane or committing suicide, all the copies of that model are the same AI,” he said.
Other leaders are sounding alarms, too
Yudkowsky is not the only AI researcher or tech leader to warn that advanced systems could one day annihilate humanity.
In February, Elon Musk told Joe Rogan that he sees “only a 20% chance of annihilation” of AI — a figure he framed as optimistic.
In April, Hinton said in a CBS interview that there was a “10 to 20% chance” that AI could seize control.
A March 2024 report commissioned by the US State Department warned that the rise of artificial general intelligence could bring catastrophic risks up to human extinction, pointing to scenarios ranging from bioweapons and cyberattacks to swarms of autonomous agents.
In June 2024, AI safety researcher Roman Yampolskiy estimated a 99.9% chance of extinction within the next century, arguing that no AI model has ever been fully secure.
Across Silicon Valley, some researchers and entrepreneurs have responded by reshaping their lives — stockpiling food, building bunkers, or spending down retirement savings — in preparation for what they see as a looming AI apocalypse.
Read the full article here
