It’s not uncommon for AI companies to fear that Nvidia will swoop in and make their work redundant. But when it happened to Tuhin Srivastava, he was perfectly calm.
“This is the thing about AI — you gotta burn the boats,” Srivastava, the cofounder of AI inference platform Baseten, told Business Insider. He hasn’t burned his quite yet, but he’s bought the kerosene.
The story goes back to when DeepSeek took the AI world by storm at the beginning of this year. Srivastava and his team had been working with the model for weeks, but it was a struggle.
The problem was a tangle of AI jargon, but essentially, inference, the computing process that happens when AI generates outputs, needed to be scaled up to quickly run these big, complicated, reasoning models.
Multiple elements were hitting bottlenecks and slowing down delivery of the model responses, making it a lot less useful for Baseten’s customers, who were clamoring for access to the model.
Srivastava’s company has access to Nvidia’s H200 chips — the best, widely available chip that could handle the advanced model at the time — but Nvidia’s inference platform was glitching.
A software stack called Triton Inference Server was getting bogged down with all the inference required for DeepSeek’s reasoning model R1, Srivastava said. So Baseten built their own, which they still use now.
Then, in March, Jensen Huang took to the stage at the company’s massive GTC conference and launched a new inference platform: Dynamo.
Dynamo is open-source software that helps Nvidia chips handle the intensive inference used for reasoning models at scale.
“It is essentially the operating system of an AI factory,” Huang said onstage.
“This was where the puck was going,” Srivastava said. And Nvidia’s arrival wasn’t a surprise. When the juggernaut inevitably surpasses Baseten’s equivalent platform, the small team will abandon what they built and switch, Srivastava said.
He expects it will take a couple of months max.
‘Burn the boats’
It’s not just Nvidia making tools with its massive team and research and development budget to match. Machine learning is constantly evolving. Models get more complex and require more computing power and engineering genius to work at scale, and then they shrink again when those engineers find new efficiencies and the math changes. Researchers and developers are balancing cost, time, accuracy, and hardware inputs, and every change reshuffles the deck.
“You cannot get married to a particular framework or a way of doing things,” said Karl Mozurkewich, principal architect at cloud firm Valdi.
“This is my favorite thing about AI,” said Theo Brown, a YouTuber and developer whose company, Ping, builds AI software for other developers. “It makes these things that the industry has historically treated as super valuable and holy, and just makes them incredibly cheap and easy to throw away,” he told BI.
Browne spent the early years of his career coding for big companies like Twitch. When he saw a reason to start over on a coding project instead of building on top of it, he faced resistance, even when it would save time or money. Sunk cost fallacy reigned.
“I had to learn that rather than waiting for them to say, ‘No,’ do it so fast they don’t have the time to block you,” Browne said.
That’s the mindset of many bleeding-edge builders in AI.
It’s also often what sets startups apart from large enterprises.
Quinn Slack, CEO of AI coding platform Sourcegraph, frequently explains this to his customers when he meets with Fortune 500 companies that may have built their first AI round on shaky foundations.
” I would say 80% of them get there in an hourlong meeting,” he said.
The firmer ground is up the stack
Ben Miller, CEO of real estate investment platform Fundrise, is building an AI product for the industry, and he doesn’t worry too much about the latest model. If a model works for its purpose, it works, and moving up to the latest innovation is unlikely to be worth the engineers’ hours.
“I’m sticking with what works well enough for as long as I can,” he said. That’s in part because Miller has a large organization, but it’s also because he’s building things farther up the stack.
That stack consists of hardware at the bottom, usually Nvidia’s GPUs, and then layers upon layers of software. Baseten is a few layers up from Nvidia. The AI models, like R1 and GPT-4o, are a few layers up from Baseten. And Miller is just about at the top where consumers are.
“There’s no guarantee you’re going to grow your customer base or your revenue just because you’re releasing the latest bleeding-edge feature,” Mozurkewich said. “When you’re in front of the end-user, there are diminishing returns to moving fast and breaking things.”
Have a tip? Contact this reporter via email at ecosgrove@businessinsider.com or Signal at 443-333-9088. Use a personal email address and a nonwork device; here’s our guide to sharing information securely.
Read the full article here
