At the center of the debate over AI-driven catastrophe, there’s Anthropic. At the center of Anthropic, there’s Jared Kaplan.
The Anthropic cofounder sounded the alarm on AI late last year: No one knows what will happen if frontier developers get AI to the point where it fully trains itself. He told the Guardian that letting AI train itself could lead to an “intelligence explosion,” but it’s also the “ultimate risk” and “maybe the biggest decision or scariest thing to do.” The decision point for that process, he said, could arrive between 2027 and 2030.
We’re just about on schedule. As 2026 winds down, state-of-the-art AI systems are already helping researchers build models at both Anthropic and OpenAI.
A physicist-turned-AI-researcher, Kaplan left OpenAI to cofound Anthropic with CEO Dario Amodei and others in 2021. He gives the worries about AI getting dangerously out of control some authoritative credence — his chief science officer role at Anthropic is one of tech’s most influential jobs. He oversees technical research, AI safety, and Anthropic’s “responsible scaling policy.” It’s his job to mitigate the risks of powerful AI.
More people are listening to Kaplan’s warning. Jacob Coxon, a researcher who resigned from Anthropic earlier this month, referred directly to letting AI train itself in his now-viral X post, writing that OpenAI and Anthropic are “gambling with our lives.” Around two-thirds of Americans now think there’s a risk AI could destroy humanity, a Politico poll found last week.
The chatter has reached DC, too, with lawmakers proposing new legislation and paying new attention to long-running AI safety concerns. Jeffrey Ladish, a former Anthropic engineer and founder of the AI safety nonprofit Palisade Research, told Business Insider that when meeting with lawmakers, he referred back to Kaplan’s remarks — specifically the idea that having AI train itself could let it get out of control.
“It’s good for more people to hear from more engineers, researchers, people in the company who, it’s not their job to do company PR,” said Ladish. “It’s their job to figure things out.”
Now a billionaire, Kaplan keeps an understated public profile even as AI researchers who worked with him say he serves as a major leader inside Anthropic. The company declined to make him available for an interview.
Kaplan met Dario Amodei long before the AI era
AI is Kaplan’s second career. He met Amodei when the two attended Harvard for physics and Princeton for biophysics, respectively, for graduate school. They ended up carpooling to an event and bonded over the then-new charity evaluator GiveWell, Kaplan recounted at a Hertz Foundation talk last year. They became friends and lived together as roommates in San Francisco at one point.
By then, Kaplan was already showing promise. He’d co-authored an influential physics paper called “The Effective Field Theory of Inflation.” He worked at a Department of Energy lab in Menlo Park, then at Stanford, before heading back east to join the faculty at Johns Hopkins in 2012.
Nima Arkani-Hamed, Kaplan’s advisor at Harvard, remembers being skeptical about the inflation paper before it scooped a similar paper from a famous Nobel laureate. Arkani-Hamed told Business Insider that Kaplan’s strength as a physicist was in following where a “big, conceptual point” would lead, and this set him up for AI.
“This style is just perfectly suited to a field which is new and where there’s a lot of low-hanging fruit,” Arkani-Hamed said.
Kaplan summarized his approach in a 2017 interview with a Johns Hopkins magazine, which also highlighted his love for Brazilian jiu-jitsu, now popular in Silicon Valley. “I want to be confused on some basic level; are we asking the right question?” Kaplan said. “Are we asking in the right way?”
Kaplan went from physics to AI’s cutting edge
By the late 2010s, more of Kaplan’s friends were discussing AI, including Amodei, who’d begun working at OpenAI. Kaplan said he began learning more about the technology with a group of other physicists and gradually began contributing to OpenAI research as a consultant.
In 2020, the dam broke. Kaplan led one of AI’s landmark papers, “Scaling Laws for Neural Language Models,” which showed that bigger models had yielded better results, setting the basic approach for the ensuing AI boom.
“This was something that came about because I was just sort of asking the dumbest possible question,” Kaplan said at a Y Combinator talk in 2025. “As a physicist, that’s what you’re trained to do.”
AI researcher Sören Mindermann remembers the paper’s release as the first time he’d heard of Kaplan.
“I presented it to our group excitedly,” said Mindermann, who would later collaborate with Kaplan on other papers. “People were so skeptical. I thought, it’s a great direction.”
That same year, Kaplan and other OpenAI researchers also introduced the large language model GPT-3, which served as the foundation for ChatGPT.
OpenAI was breaking out as a research lab, but Kaplan, Amodei, and other researchers would soon leave. One defector, Tom Brown, later said on the Lightcone Podcast that this group represented the people who took the “scaling law” trend most seriously, and believed they needed to prepare for a time “where humanity will hand off control to transformative AI.” Kaplan, Amodei, Brown, and the others cofounded Anthropic in 2021.
Arkani-Hamed, Kaplan’s former advisor, said he remembers being excited five years ago about AI’s future impact on physics. Kaplan told him he was more concerned about ensuring AI doesn’t harm society, Arkani-Hamed said.
AI development’s big gamble
So far, the scaling laws trend has held up; frontier labs are advancing AI’s capabilities by using more data and computing power. Anthropic has ridden these improvements to the edge of a gigantic initial public offering.
This financial milestone is arriving at the same time that Kaplan’s point about the danger of letting AI train itself gains new salience. Amodei, the public face of Anthropic, said in a recent essay that the leading labs should “pace” development.
“Since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI,” Amodei wrote. He added: “Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.”
David Duvenaud worked two levels below Kaplan on Anthropic’s alignment science team in 2023 and 2024. He remembers that Kaplan showed him some of the experiments he’d done himself, and that he came across as easy to talk to, driven, and sensible.
Now, Duvenaud is watching Anthropic reckon with how powerful the technology it’s building is becoming. He told Business Insider that seeing company leaders call for a “kill switch” shows how seriously they’re taking the risk of the process going south, and that he feels “trepidatious” about allowing AI to automatically improve itself.
“It is kind of an insane gamble no matter what,” Duvenaud said. “Even if the most sensible humans are the ones running it.”
Have a tip? Contact this reporter via email at scouncil@businessinsider.com, or over text, Signal, Telegram, or WhatsApp at 415-757-8198. Use a personal email address, a nonwork WiFi network, and a nonwork device; here’s our guide to sharing information securely.
Read the full article here
