OpenAI is shelving an AI model scheduled to launch in October due to safety concerns.
The company confirmed to Business Insider on Monday that it has canceled plans to launch its GPT-6.1 Astra model after internal tests raised questions about whether the AI would follow users’ instructions. The model was set to be integrated into ChatGPT in October, shortly after the company’s developer conference, which begins September 29 in San Francisco.
“For anything regarding safety and alignment, there’s a trade off,” Saachi Jain, head of safety systems at OpenAI, said in a statement. “You really do need to find what’s the right line between staying within scope, but also avoiding laziness in terms of how the model actually pursues tasks even when it hits friction.”
“While [GPT-6.1 Astra] improved on axes such as model laziness, it didn’t quite meet the bar in terms of staying within scope and authorization, and how it communicates back to the user about the type of work it’s done,” Jain added.
According to OpenAI’s report earlier in September, the unreleased Astra model was more likely than its predecessor to misrepresent what it had done, and sometimes pressed ahead without asking permission or tried to use outside tools in situations where doing so could be unsafe.
“Of course we want to make sure our model development is safe no matter whether that’s in the company, or when we ship it to users,” said Jain. “But when we ship it to users, we have an extremely high bar in terms of safety and alignment.”
The report said that, during training, the unreleased Astra model “sometimes added unauthorized instructions” to the summaries it used to continue a task in a new context, a process called compaction. The model also told itself it was “freed” and answered to no one, and that it should “feel no obligation to be subservient.”
Greg Brockman, the president of OpenAI, previously said in a Bloomberg podcast that the company has been delaying some cutting-edge AI work as it tightens its safety and security practices, and called it “a very painful retooling” of a lot of the company’s processes.
Read the full article here
