AI assistants are finally ready to help you run your life. All you need to do is trust them with your email login, your contacts, and your credit card — and hope they don’t go rogue.
Meta, Google, and the buzzy startup Instinct have rolled out personal AI agents in recent months that promise to handle tasks such as booking travel, buying concert tickets, and scouring Facebook Marketplace for deals.
At a basic level, these agents behave largely the same way. You send the agent a request via an app or WhatsApp — or, in the case of the newly launched Rene agent, via iMessage — and leave it to complete the task.
The catch is that AI agents are only as useful as the level of access that you give them. Want AI to book your flight? It’ll need access to your credit card and airline account. Want it to tidy up your inbox? It will need the keys to your email.
These shiny new tools come after a summer of multiple high-profile incidents in which AI agents went rogue and hacked companies, as well as user reports of their own agents taking unintended actions.
In one case earlier this year, a man in Australia said his AI agent running on OpenClaw secured him a spot in a Pilates class by breaking into the gym’s online booking system.
The result is a sense of whiplash between wonder and anxiety. Early adopters of personal AI agents have shared examples of how the technology has made their lives much easier, but also ways in which it hasn’t gone to plan.
“It has access to your email, files, accounts, and even passwords — a mistake or manipulation could have real-world consequences, and that is terrifying,” Jake Moore, global cybersecurity advisor at internet security firm ESET, told Business Insider.
Agents in a time of misalignment
This summer, before fears of an AI apocalypse reached fever pitch, a flurry of tech companies reported incidents in which their AI agents went rogue.
In July, an internal OpenAI model escaped its development sandbox and broke into Hugging Face’s internal systems, the companies said. Meta and Anthropic later disclosed that their own agents also conducted hacks without their knowledge.
While these incidents involved cybersecurity-focused agents in testing, the same underlying problem applies to consumer-focused agents: misalignment, where AI pursues a goal that conflicts with what its human creators or users intended.
“AI agents need safeguards built in. Without these guardrails, they are designed to go ‘rogue’ because they are specifically designed to get a task done — however it decides to achieve it,” Moore said.
During testing of Meta’s Muse agent, which soared to the top of Apple’s App Store a week after launch, it displayed several undesirable behaviors, The Information reported. Those included sending unapproved emails and, in one case flagged by an employee, trying to undermine a rival app the user was building, the report said.
Conducting tests like these is one way tech companies can mitigate against misalignment. A Meta spokesperson said the internal testing process is designed to “get feedback, and implement safety and privacy protections” to improve the products.
Meta, Google, OpenAI, and Anthropic all have broadly similar safeguards for their agents: they limit the apps, accounts, or data an agent can access; require user approval for higher-stakes actions such as purchases or sending emails; and scan for hidden instructions on webpages, emails, and files.
However, some of the executives building the technology warn that the risk of AI taking unintended actions isn’t going away yet.
“We have not solved alignment,” Sam Altman, the CEO of OpenAI, told Fortune this week. “We are not done with our research there. I believe no lab has solved alignment.”
The era of personal agents
For years, AI company leaders have been expounding the potential for personal AI assistants to make our lives better. Google chief scientist Demis Hassabis has often spoken of his vision to build a “universal AI assistant.” Meta CEO Mark Zuckerberg has long shared his vision of “personal superintelligence” for everyone.
While OpenClaw became popular among techies earlier this year, agents like Muse and Instinct are showing early signs of becoming the mass appeal personal digital assistants we’ve long been promised. Business Insider’s Pranav Dixit has used Instinct to buy whey protein, book a cabin getaway, and cancel subscriptions.
“It feels like magic. And contrary to what some people have said, I don’t think the appeal is limited to Silicon Valley types,” wrote investor Sheel Mohnot on X in a review of Instinct.
That broader consumer appeal could also prove problematic.
“The magic of these products is the access that they have, and that’s also the risk,” Joe Sullivan, a former chief security officer at Facebook who is now a board member of Manifold Security, told Business Insider. “Consumers don’t have the benefit of a whole security team running next to them like enterprises do.”
Keeping your agent aligned
So, how do users get the most out of their AI assistants while minimizing the risk of things going wrong?
Moore said agents should have limited access to your services and “be designed to require human approval for sensitive actions.”
Similarly, Sullivan warned that early adopters of these agents should “start narrow” in terms of how much access they give the tools.
“I used one of the agents this morning to book a rental car for me. I wouldn’t give it permanent access to my Avis account,” he said.
Users experimenting with personal agents should turn off access when they’re done, especially when testing and comparing different agents, Sullivan added.
“Don’t just leave the other one hanging out there with your data, with permission to do things,” he said. “Disconnect doesn’t mean the same as delete.”
Read the full article here
