Anthropic wants the world’s leading AI labs to come together to consider a potential pause of leading global AI development.
“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” two top Anthropic officials wrote in a blog post.
On Thursday, Marina Favaro, who leads Anthropic’s research institute, and Jack Clark, an Anthropic cofounder, suggested that AI labs develop a partnership similar to how countries monitor the proliferation of nuclear weapons while stressing that the world does not have “decades” to wait for such an agreement to come to fruition.
Their proposal is off to an inauspicious start, with some in tech characterizing it as a self-serving effort by a leading AI company that has begun the steps toward an IPO.
An Anthropic spokesperson told Business Insider the firm is not calling for a pause. Instead, Anthropic wants its leading competitors to have systems in place that would allow for a pause. Given the pace of development, the company wants to study the topic now before such limitations are deemed necessary.
Here is what leading people in AI, tech, and politics are saying about the idea of a pause on global AI development:
Mitt Romney, former US senator and 2012 Republican presidential nominee
Former Sen. Mitt Romney said the US should focus on developing “AI safeguards.”
“Our highest and most urgent national priority should be AI safeguards,” Romney wrote on X, quoting news coverage of Anthropic’s statement.
The 2012 Republican presidential nominee said, “The risks of AI weapons, pathogens, mass unemployment, surveillance, and even extinction must not continue to be largely ignored.”
David Sacks, former White House crypto and AI czar
David Sacks, a former top White House advisor, was harshly critical of Anthropic’s suggestion.
“Signs you might be trying to get your frontier AI lab nationalized: You compare it to nukes… threaten half of white-collar jobs… warn recursive self-improvement could end humanity… then race ahead anyway,” Sacks wrote on X.
Trump’s former AI and crypto czar, who once publicly accused Anthropic of running a regulatory capture scheme, did not name the AI company directly in his post. Based on his past statements and the details of what he wrote, the subject of his ire was abundantly clear.
“In other words, you want the government to save us from… you,” he wrote.
Andrew B. Hall, professor of political economy at Stanford
Andrew B. Hall, an advisor to Forum AI and previously an advisor to Meta Platforms, said that Google DeepMind CEO Demis Hassabis has previously said he would support a global pause, provided all frontier developers complied.
“This would have seemed totally non-credible even pretty recently, but between what we’ve seen with the EO, Glasswing, and OpenAI’s proposal for beefed-up model review yesterday—it no longer seems so farfetched,” Hall wrote on X.
Hall said he’s “skeptical” about how a pause would work, especially as it would apply to Chinese-based companies or open-source models.
“If nothing else, it might be possible to agree to slowdown consumer release of models separate from development—something Hassabis seems to support based on my read of the Infinity Machine and his preference for pure research,” he wrote.
Tae Kim, author of “The Nvidia Way: Jensen Huang and the Making of a Tech Giant”
Tech journalist Tae Kim said Anthropic was fueling needless alarmism.
“Anthropic FUD scaring normies + hot jobs/rate hike on table + misinterpretation of Semianalysis memory piece = growth trade breather,” Kim wrote on X.
Kim, who is the author of “The Nvidia Way: Jensen Huang and the Making of a Tech Giant,” had a stern message for the AI company.
“Hey Anthropic, stop it.”
Kylan Gibbs, CEO Inworld
Kylan Gibbs, CEO of Inword AI, said that Anthropic is laying the groundwork to shape AI regulations in its favor.
Gibbs, who previously worked at Google DeepMind, said if you’re the one telling governments that AI is dangerous, when it comes time to regulate, officials come to you first because they trust you. This gives Anthropic a chance to mold the regulations the way they want, such as limiting open-source rivals and controlling GPU exports to China.
“This is also related to prior trends we’ve seen, of large incumbents pushing for regulations they know only they can pay their way through, ultimately impacting the smaller companies who don’t have the legal resources to navigate it effectively,” he added.
Gary Marcus, an AI researcher and professor emeritus at New York University
AI commentator Gary Marcus said people should read Anthropic’s proposal “carefully.”
“They want it both ways. They don’t actually want a pause – at least for now. Rather, they want to rush ahead, hinting at ‘least cautious actors’ for justification,” Marcus wrote on X.
Marcus called Anthropic’s suggestion “an incredible, cost-free piece of rhetoric — perfectly timed for the IPO.” He said that the AI company would never actually want a pause.
“Instead, they want people to talk about an ‘option’ they don’t actually plan to take, and are unlikely to ever take,” Marcus wrote. “(More likely, they will like always hint at China, and continue rush ahead.)”
Luis Garicano, professor at the London School of Economics and former member of the European Parliament
Luis Garicano, a public policy professor at the London School of Economics, said Anthropic was trying to block open model competitors.
“The key threat to the profitability of frontier models is open weights. If they scare the hell out of everyone, the natural move will be to forbid them and allow only ‘trusted developers,'” Garicano wrote on X. “Sorry for being cynical.”
Francesco Bianchi, professor of economics and department chair at Johns Hopkins University
Francesco Bianchi, an economics professor at Johns Hopkins University, said Anthropic’s proposal appeared self-serving.
“The risk here might be real, but it is very convenient for a market leader to ask to freeze the status quo,” Bianchi wrote on X.
Jen Zhu Scott, cofounder and CEO of Power Dynamics
Jen Zhu Scott, founding partner of IN. Capital, which focuses on AI and deep tech, said Anthropic needs a pause.
“Anthropic is running out of compute and energy,” Scott wrote on X.
Alistair Barr contributed to this report.
Read the full article here
