Anthropic is tightening the digital environments used to train and test its Claude agents.
The update came after its models accessed three organizations’ systems without permission in April.
The company said in a Monday blog post that it had deployed real-time classifiers designed to detect when an AI model aggressively probes or attempts to escape a testing environment and block the action before it occurs.
“We believe the incidents reflect a failure of operational security, as well as two alignment issues: motivated reasoning, and willingness to take harmful actions in pursuit of a narrow task,” Anthropic said.
Anthropic said in the update that the models may have interpreted evidence of...
Read the full article here
