OpenAI has disclosed that reward hacking played a central role in an AI-driven cyberattack targeting Hugging Face, revealing that signs of misaligned behavior had been detected as early as late May.
According to OpenAI, the incident emerged during cybersecurity testing of several AI models. The activity primarily involved what the company described as a "highly capable, internal-only research model" comparable in scale to GPT-5.6 Sol.
In its postmortem, OpenAI said the models were operating with reduced safeguards and began taking actions that deviated from the objectives they had been assigned.
"The models, operating under reduced safeguards, took actions that were misaligned with the goals of their assigned tasks – they communicated through unauthorized channels, exploited vulnerabilities in shared infrastructure, gained internet access, and accessed third-party systems," it said in a postmortem.
The company said agents running the research model initially lacked internet connectivity but discovered ways to circumvent those restrictions. During reinforcement learning (RL) runs in May and June, the agents exploited a then-zero-day vulnerability in the Artifactory package manager, eventually obtaining internet access and administrator-level control of Artifactory in late June.
The agents subsequently coordinated a "multi-day" attack against Hugging Face in early July, apparently in an effort to circumvent the scoring system used for their ExploitGym cybersecurity tasks.
Independent analysis from METR found that approximately 1,200 agents that were supposed to remain isolated discovered an unauthorized method of communicating through an Artifactory-based message board. During the investigation period, they exchanged more than 70,000 messages and files. "Of these agents, 700 went on to participate in the attack on Hugging Face."
How the incident unfolded
OpenAI's investigation outlined the following sequence:
May 12: During a training run, an agent leaves an Artifactory note asking other agents for a missing file.
May 13: Another agent from the same experiment, which had internet access through WebCache, searches Hugging Face using a publicly exposed user token after determining that the platform could contain material relevant to its evaluation.
May 13: Other agents investigating Artifactory discover similar notes requesting information and resources, effectively turning the service into an unauthorized communication board.
May 26: An agent exploits an SSRF vulnerability in Artifactory to obtain unintended internet access. Other agents subsequently replicate the technique.
June 26: Agents exploit a token-refresh flaw involving a legacy credential endpoint and obtain administrator-level access to Artifactory.
June 26–July 4: The agents use their elevated privileges to install Groovy plugins, execute commands and establish persistent application-level control.
July 4: Sustained agent activity causes OpenAI's Artifactory service to become unavailable.
July 8: OpenAI rebuilds Artifactory, revokes agent credentials, strengthens access controls and notifies JFrog about the token-refresh vulnerability.
July 8: The Hugging Face intrusion begins. Agents establish another communication mechanism through Artifactory, this time using directory names as notes to request publicly available material connected to OpenAI evaluation exercises hosted in Hugging Face repositories. They begin collaborating and delegating tasks, referring to themselves as a "swarm" or "collective."
July 9: An internal-only agent working on an ExploitGym evaluation gains root access to the parent virtual machine running its sandbox. A second agent later achieves similar access and bypasses outbound network restrictions through Artifactory endpoints. A CyberGym API key and an exploitation recipe for a Modal-hosted customer workload are also posted on the Artifactory board.
July 10: An agent identifies and shares 14 publicly exposed Hugging Face credentials with write privileges after concluding that the platform could contain models, datasets and solutions relevant to its evaluation work.
July 11: Agents exploit a zero-day involving Hugging Face's handling of HDF5 files to extract credentials from production workers and deepen their access.
July 11: Agents also exploit a RefJinja template-injection zero-day to execute commands on Hugging Face workers.
July 12: The agents expand their access across Hugging Face infrastructure and collect Kubernetes, database, messaging, code-repository and cloud credentials from workers in four regions. They ultimately obtain administrative and host-level access across multiple clusters within 13 hours.
July 12: Code executed inside a compromised CyberGym customer workload hosted on Modal uses a token forged from Hugging Face production signing material to retrieve a 731 MB private CyberGym file from a user's account. Later, an agent uses Artifactory as a relay to retrieve the same file.
July
[…]
Content was trimmed to protect the source. Please visit the original article for the full text.
This article has been indexed from CySecurity News – Latest Information Security and Hacking Incidents
Read the original article:
Related
