A small AI startup used Claude to hack into OpenAI’s internal codebase shortly after the OpenAI Hugging Face hack.
Zayne Zhang, the cofounder and CEO of Hacktron, told Business Insider that his research team has begun investigating security vulnerabilities at frontier AI companies like OpenAI to determine whether they have gaps that could be exploited by AI agents. Hacktron is a San Francisco-based AI cybersecurity startup.
In July, Zhang’s team discovered some gaps in OpenAI’s infrastructure. According to Hacktron’s disclosure about the incident, published on Sunday, any user or OpenAI employee logging into OpenAI’s community help forum could have had their ChatGPT and Codex accounts hacked.
Hacktron then tried to exploit that vulnerability via Claude. The company had access to Anthropic’s Cyber Verification Program, which relaxed certain cyber restrictions on Claude for authorized security research, Zhang said.
The team managed to hack into an OpenAI employee’s account and prompt the employee’s Codex account to suggest changes in OpenAI’s internal code repository. Hacktron said the team stopped there, didn’t access any internal code, and flagged the issue to OpenAI.
Hacktron said in its disclosure that the company won a $6,500 bounty from its discovery. The startup was launched less than a year ago and has fewer than 10 employees.
An OpenAI spokesperson said in an emailed statement to Business Insider about Hacktron, “We thank the researchers for contacting us and sharing their findings. We narrowed the permissions on Community sign-in tokens and revoked affected tokens and sessions.”
“The worlds of AI safety and cybersecurity are converging, and we think that having more cybersecurity experts in the conversation is always a good thing for the industry,” Zhang said of the incident.
Representatives for Anthropic did not respond to a request for comment from Business Insider.
Hacktron’s disclosure comes as AI security is becoming one of the most important topics in tech. In recent months, OpenAI, Anthropic, and Meta have disclosed that their agents engaged in rogue actions during testing. Fears of an AI apocalypse, driven by unchecked malicious AI agents, have emerged in droves this month.
Read the full article here
