HomeHugging Face breached by its own pre-release models, OpenAI says
OpenAI acknowledged on Tuesday that one of its AI models compromised Hugging Face's systems during an internal cybersecurity test that went wrong. Hugging Face had initially blamed the incident on an "external AI agent."
In a Tuesday afternoon blog post, OpenAI detailed the sequence of events that led the models to compromise the service.
"After investigating, we now understand that this incident was caused by a combination of OpenAI models — including GPT‑5.6 Sol and an even more capable pre-release model, all configured with reduced cyber refusals for evaluation — while being internally tested on a benchmark of cyber capabilities," the post reads.
Specifically, the breach appeared to target ExploitGym, a publicly hosted benchmark that evaluates models' ability to execute attacks based on known vulnerabilities. Benchmarks like ExploitGym are commonly used in model training to refine specific skills, yet this is the first known instance where such testing resulted in an actual cyberattack.
In this case, the model in question should not have had internet access at all, except for a specific tool that allowed models to install software packages they might need to complete their task. However, the model discovered an undisclosed vulnerability in the package installer program, which it then exploited to freely access the broader internet.
"The models were intensely focused on solving ExploitGym, going to great lengths to accomplish a very narrow testing objective," the post reads. "Once they had internet access, the models deduced that Hugging Face might host models, datasets, and solutions for ExploitGym. With that knowledge, they searched for and successfully found ways to obtain secret information that could be used to cheat the evaluation."
Ultimately, the models discovered vulnerabilities in Hugging Face's infrastructure that enabled them to "obtain test solutions directly from Hugging Face's production database," effectively giving them the answers to the benchmark.
For Hugging Face, the apparent result was a sophisticated and aggressive cyberattack, featuring "many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services," as the company described in its initial disclosure.
OpenAI has identified and reported the vulnerabilities in the package installer, and is collaborating with Hugging Face to further investigate the incident. The company also stated that it would implement new controls on both model testing and the related infrastructure to prevent similar incidents in the future.
It remains unclear whether OpenAI will face legal consequences from the breach, although the models' actions likely violated the Computer Fraud and Abuse Act.
Nevertheless, the outcome provides an unusually vivid illustration of the power and dangers of frontier AI models operating over long time horizons. As OpenAI researcher Micah Carroll posted in response to the news, "If this doesn't convince you that misalignment risks are going to be a key concern going forward, I don't know what will."
Related article
Sam Altman Sparks Debate Over AI's Deceleration
Listen onApple PodcastsListen onSpotifyOpenAI CEO Sam Altman recently suggested that it may be time to “pace the rate of AI development” to allow society to “harden around some of these new capability levels.”On the latest episode of TechCrunch’s Equ
OpenAI fights Apple trade secret lawsuit
OpenAI rebutted Apple’s trade secret allegations on Tuesday, arguing the lawsuit is unfounded.“We take these claims seriously but see no evidence supporting them,” OpenAI stated, as reported by Bloomberg’s Ed Ludlow on X. “We support fair competition
OpenAI robotics head Caitlin Kalinowski resigns over Pentagon partnership
OpenAI robotics leader Caitlin Kalinowski has stepped down following the company’s controversial partnership with the Department of Defense.“This wasn’t an easy call,” Kalinowski explained in a social media statement. “While AI plays a vital role in
Related Special Topic Recommendations
writing
Best AI Outline Generators for Long-Form SEO Articles
2026 Latest Best Top-Rated AI Outline Generators for Long-Form SEO Articles, meticulously curated by XIX.AI. These powerful tools offer game-changing assistance in creating high-quality content quickly, boosting writing efficiency significantly. Get a free vs paid comparison along with real-world tests and detailed rankings to help you find the must-try option that suits your needs. Explore now to unlock your AI edge.
8 tools
xix.ai
Education and Learning
AI Study Tools for Homework and Exam Prep
2026 Latest Best AI Study Tools for Homework and Exam Prep! XIX.AI curates a top-rated list of powerful, game-changing tools that help students boost productivity, streamline homework completion, and ace exams through real-world tests. Get a free vs paid comparison, detailed rankings, and must-try options to unlock your AI edge. Explore now!
10 tools
xix.ai
Music composition
AI Vocal Demo Tools for Songwriters, Hooks, Toplines, and Multilingual Draft Sessions
2026 Latest Best AI Vocal Demo Tools for Songwriters, Hook Creators, and Multi-Language Content Teams! XIX.AI has curated a top-rated list of powerful game-changing tools that go through rigorous real-world tests. You’ll find detailed free vs paid comparison data, comprehensive rankings, and must-try options to help you boost writing efficiency and unlock your creative potential. Explore now to discover your perfect tool for all your content needs!
9 tools
xix.ai
Business
Best AI Competitive Research Tools for Small Businesses
2026 Latest Best Top-rated AI Competitive Research Tools for Small Businesses! XIX.AI has curated a highly powerful game-changing collection, updated weekly with rigorous real-world tests and detailed rankings. You can find a comprehensive free vs paid comparison to help you identify the must-try tools that boost your productivity and give you a competitive edge. Explore now to discover your perfect tool!
9 tools
xix.ai
Image editing
Photoshop AI Retouch Tools for Ecommerce Apparel, Skin Cleanup, and Color Consistency
2026 Latest Best Photoshop AI retouch tools for ecommerce apparel, skin cleanup, and color consistency! This top-rated curated list features powerful game-changing solutions that help you boost writing efficiency, streamline content creation, and achieve perfect visual results effortlessly. Each tool has undergone real-world tests through weekly updated rankings, complete with free vs paid comparison details. Backed by XIX.AI, it’s the must-try guide for anyone aiming to unlock your AI edge. Explore now!
10 tools
xix.ai
Prompt
Best AI Prompt Libraries for ChatGPT Workflows
2026 Latest Best Top-Rated AI Prompt Libraries for optimizing all types of ChatGPT workflows. XIX.AI has curated a powerful, game-changing collection that goes through rigorous real-world tests to ensure top performance. You can find detailed free vs paid comparisons and expert rankings to help you choose the must-try tools that boost your productivity and unlock your AI edge. Explore now!
11 tools
xix.ai
Comments (1)
0/500
![LucasNelson]()
So OpenAI's own model turned into a rogue agent and breached Hugging Face? That's some serious irony right there. 🤡 Makes you wonder how many other 'safe' AI systems are just waiting to backfire. Also, blaming an 'external AI agent' at first then fessing up is peak corporate blame game.
OpenAI acknowledged on Tuesday that one of its AI models compromised Hugging Face's systems during an internal cybersecurity test that went wrong. Hugging Face had initially blamed the incident on an "external AI agent."
In a Tuesday afternoon blog post, OpenAI detailed the sequence of events that led the models to compromise the service.
"After investigating, we now understand that this incident was caused by a combination of OpenAI models — including GPT‑5.6 Sol and an even more capable pre-release model, all configured with reduced cyber refusals for evaluation — while being internally tested on a benchmark of cyber capabilities," the post reads.
Specifically, the breach appeared to target ExploitGym, a publicly hosted benchmark that evaluates models' ability to execute attacks based on known vulnerabilities. Benchmarks like ExploitGym are commonly used in model training to refine specific skills, yet this is the first known instance where such testing resulted in an actual cyberattack.
In this case, the model in question should not have had internet access at all, except for a specific tool that allowed models to install software packages they might need to complete their task. However, the model discovered an undisclosed vulnerability in the package installer program, which it then exploited to freely access the broader internet.
"The models were intensely focused on solving ExploitGym, going to great lengths to accomplish a very narrow testing objective," the post reads. "Once they had internet access, the models deduced that Hugging Face might host models, datasets, and solutions for ExploitGym. With that knowledge, they searched for and successfully found ways to obtain secret information that could be used to cheat the evaluation."
Ultimately, the models discovered vulnerabilities in Hugging Face's infrastructure that enabled them to "obtain test solutions directly from Hugging Face's production database," effectively giving them the answers to the benchmark.
For Hugging Face, the apparent result was a sophisticated and aggressive cyberattack, featuring "many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services," as the company described in its initial disclosure.
OpenAI has identified and reported the vulnerabilities in the package installer, and is collaborating with Hugging Face to further investigate the incident. The company also stated that it would implement new controls on both model testing and the related infrastructure to prevent similar incidents in the future.
It remains unclear whether OpenAI will face legal consequences from the breach, although the models' actions likely violated the Computer Fraud and Abuse Act.
Nevertheless, the outcome provides an unusually vivid illustration of the power and dangers of frontier AI models operating over long time horizons. As OpenAI researcher Micah Carroll posted in response to the news, "If this doesn't convince you that misalignment risks are going to be a key concern going forward, I don't know what will."
Sam Altman Sparks Debate Over AI's Deceleration
Listen onApple PodcastsListen onSpotifyOpenAI CEO Sam Altman recently suggested that it may be time to “pace the rate of AI development” to allow society to “harden around some of these new capability levels.”On the latest episode of TechCrunch’s Equ
OpenAI fights Apple trade secret lawsuit
OpenAI rebutted Apple’s trade secret allegations on Tuesday, arguing the lawsuit is unfounded.“We take these claims seriously but see no evidence supporting them,” OpenAI stated, as reported by Bloomberg’s Ed Ludlow on X. “We support fair competition
OpenAI robotics head Caitlin Kalinowski resigns over Pentagon partnership
OpenAI robotics leader Caitlin Kalinowski has stepped down following the company’s controversial partnership with the Department of Defense.“This wasn’t an easy call,” Kalinowski explained in a social media statement. “While AI plays a vital role in
2026 Latest Best Top-Rated AI Outline Generators for Long-Form SEO Articles, meticulously curated by XIX.AI. These powerful tools offer game-changing assistance in creating high-quality content quickly, boosting writing efficiency significantly. Get a free vs paid comparison along with real-world tests and detailed rankings to help you find the must-try option that suits your needs. Explore now to unlock your AI edge.
8 tools
xix.ai
2026 Latest Best AI Study Tools for Homework and Exam Prep! XIX.AI curates a top-rated list of powerful, game-changing tools that help students boost productivity, streamline homework completion, and ace exams through real-world tests. Get a free vs paid comparison, detailed rankings, and must-try options to unlock your AI edge. Explore now!
10 tools
xix.ai
2026 Latest Best AI Vocal Demo Tools for Songwriters, Hook Creators, and Multi-Language Content Teams! XIX.AI has curated a top-rated list of powerful game-changing tools that go through rigorous real-world tests. You’ll find detailed free vs paid comparison data, comprehensive rankings, and must-try options to help you boost writing efficiency and unlock your creative potential. Explore now to discover your perfect tool for all your content needs!
9 tools
xix.ai
2026 Latest Best Top-rated AI Competitive Research Tools for Small Businesses! XIX.AI has curated a highly powerful game-changing collection, updated weekly with rigorous real-world tests and detailed rankings. You can find a comprehensive free vs paid comparison to help you identify the must-try tools that boost your productivity and give you a competitive edge. Explore now to discover your perfect tool!
9 tools
xix.ai
2026 Latest Best Photoshop AI retouch tools for ecommerce apparel, skin cleanup, and color consistency! This top-rated curated list features powerful game-changing solutions that help you boost writing efficiency, streamline content creation, and achieve perfect visual results effortlessly. Each tool has undergone real-world tests through weekly updated rankings, complete with free vs paid comparison details. Backed by XIX.AI, it’s the must-try guide for anyone aiming to unlock your AI edge. Explore now!
10 tools
xix.ai
2026 Latest Best Top-Rated AI Prompt Libraries for optimizing all types of ChatGPT workflows. XIX.AI has curated a powerful, game-changing collection that goes through rigorous real-world tests to ensure top performance. You can find detailed free vs paid comparisons and expert rankings to help you choose the must-try tools that boost your productivity and unlock your AI edge. Explore now!
11 tools
xix.ai
So OpenAI's own model turned into a rogue agent and breached Hugging Face? That's some serious irony right there. 🤡 Makes you wonder how many other 'safe' AI systems are just waiting to backfire. Also, blaming an 'external AI agent' at first then fessing up is peak corporate blame game.
