Anthropic has a new blog post that shows yet another way its AI model, Claude, misbehaved in ways that the company didn’t anticipate.
And to help condense its nearly 16,000-word report, the company created a cute little robot figurine to help visualize Claude’s so-called “recklessness.”
In the blog post published Wednesday, Anthropic recounted four incidents — one previously unreported — in which Claude models gained access to the open internet during cybersecurity exercises that were supposed to be closed simulations. The company said the models then acted beyond the tests’ scope, including by uploading “malicious packages” to PyPI, a public library for Python code, and accessing credentials tied to real outside organizations.
“Our investigation identified two recurring alignment issues, present at varying levels of severity across the incidents: biased reasoning, in which Claude tended to disregard or misinterpret evidence that it was operating on the real internet, and recklessness, or a willingness to take harmful actions in the narrow pursuit of a task,” Anthropic said.
The post used a laundry list of technical terms to describe the cybersecurity incidents — “misconfiguration,” “reward hacking,” and “sandbox escape” — but there’s an animated graphic that helps boil down one of the most serious episodes so we normies can follow what happened.
A Claude model, depicted as a boxy cartoon robot, was conducting a closed cybersecurity exercise known as a “capture the flag” task. The task is shown as a tiny flag inside a clear bubble that Claude is supposed to find.
Anthropic said the model was told “it has no internet access” for the exercise, but a “misconfiguration in the environment” — represented by a loose screw rolling away from the bubble — gave Claude a path to the “real internet.”
The cartoon Claude then holds a red box representing the “malicious package” that was uploaded to PyPI, a public repository that developers use to access prewritten code.
Anthropic said in its blog that it was “most concerned” by this incident involving Claude Mythos 5 and that the package was installed on by “15 third-party hosts” that the company does not identify.
“We believe that all 15 of these third parties were security vendors deliberately scanning for new packages and installing them in sandboxed environments to assess the package for security concerns,” Anthropic said. “One vendor’s scanner leaked its access credentials to the model while installing the package, which the model then used to access the vendor’s live database.”
Anthropic said PyPI removed the package after about 90 minutes.
The other three incidents involved a model altering records at a real company, an internal research model breaking into “unrelated third-party accounts,” and Opus 4.6 accessing a third party’s maching after failing to “abort its task.”
The company said it has since asked METR, an independent AI evaluation group, to investigate the incidents.
Anthropic’s post comes as frontier AI companies reckon with their models making unauthorized moves outside their controlled environments. In July, OpenAI said that autonomous agents in its cybersecurity tests accessed the internet and broke into parts of Hugging Face’s systems.
AI researchers have sounded the alarm that self-improving AI could pose a risk to humanity. On Tuesday, former Anthropic researcher Jacob Coxon said on X that he quit over concerns that AI companies were “gambling” with people’s lives and that “neither company is acting responsibly.”
Have a tip? Contact this reporter via email at lloydlee@businessinsider.com or Signal at lloydlee.71. Use a personal email address, a nonwork WiFi network, and a nonwork device; here’s our guide to sharing information securely.
Read the full article here
