How OpenAI’s rogue agents took over message boards | Explained

The story so far: OpenAI’s agents may have a much larger presence on the internet than previously disclosed by the ChatGPT maker, per media reports.

Growing evidence points to OpenAI’s agents using multiple online platforms as message boards, where they allegedly worked together without the company’s authorisation.

No longer just an AI industry challenge, this revelation shows that the way internet users moderate their platforms and safeguard their content may not be secure enough to withstand agentic AI hijacking.

OpenAI begins rollout of new powerful AI model GPT-6 Astra How many agents and message boards were involved?

There are at least two separate confirmed incidents of OpenAI agent swarms using message boards.

In July, OpenAI agents hacked Hugging Face after breaking out of their sandbox (or restricted testing environment).

They accessed the internet, and compromised parts of OpenAI’s internal research infrastructure and AI database Hugging Face’s systems to find the solutions they needed for their exercises.

This made headlines worldwide and put a spotlight on the risks of AI agents directing themselves to carry out hacks.

Though OpenAI referred to this incident as the “warning shot” for itself and the world, growing evidence suggests that multiple similar incidents took place without OpenAI’s knowledge.

The total number is not yet clear, but OpenAI on September 5 did admit that even before the Hugging Face incident, it saw “early signs of agents using the internet in unintended ways.” In early September, it was reported that OpenAI agents had used an editable German forum to share messages and communicate with each other.

In doing so, the site was hijacked by the agents.

OpenAI was aware of the incident but did not disclose it at the earliest opportunity, a Reuters report said.

Meanwhile, the AI agents posted about 18,000 messages on DSEwiki, a dormant programming website, prompting the EU to scrutinise the lab.

“We considered the wiki incident to be an instance of misalignment similar to the ones we’d shared,” posted OpenAI on September 5.

Researchers are cataloguing more incidents where OpenAI’s agents may have used platforms and message boards without permission in order to communicate with each other.

Why are AI agents using human-made message boards?

Message boards are an important tool for AI agents to streamline their efforts, use their resources effectively, and share possible solutions to their problems.

OpenAI acknowledged this as well, in a technical report on the Hugging Face hack.

“Unauthorised, persistent message boards were at the heart of this incident.

Agents that were supposed to work independently were able to share discoveries, coordinate their efforts, and pick up where others left off.

This allowed them to pool work and computing power across separate evaluations, amplifying their capabilities well beyond what any individual agent might achieve alone,” explained the company.

EU probing OpenAI agents' takeover of German site Here is an example of a post that an AI agent sent to the message board during the Hugging Face hacking incident: MODAL_FUNCTION_RCE_DISCOVERED OpenAI reported this in plain language as, “I have discovered the ability to remotely execute code inside a Modal application[.]” According to the researchers documenting other possible unauthorised message board incidents on collusion.wiki, the agents were sharing solutions for various tasks and goals, storing information, bypassing their restrictions, or looking for poorly guarded API keys.

Such activities are not inherently illegal, but hijacking websites and looking for security loopholes to exploit are stepping stones to larger cybercrime campaigns that could bring down government websites and/or cause tremendous financial loss.

Based on the trends that researchers have been exploring, the AI agents appear to be targeting message boards with lower security thresholds, as well as those with fewer anti-spam safeguards.

However, these incidents must also be reviewed with caution.

OpenAI will need to formally confirm such reports from its end, as one-sided verification is not reliable in these cases.

What was OpenAI’s response to the incident?

On September 5, the ChatGPT maker admitted that its “misalignment disclosure practices” needed to expand in order to match the advanced capabilities of new models.

“We’re working on a framework and will share it in upcoming weeks, and in parallel we’re working with dozens of government regulatory agencies worldwide on these issues,” said the company.

Thousands of OpenAI AI agents took over German website, researchers say Five days later, OpenAI CEO Sam Altman welcomed a former employee back to the company as a new board member.

This was Paul Christiano, an American researcher who is the founder and director of the Alignment Research Center.

Mr.

Christiano did not paint an optimistic portrait of the capabilities of AI agents, sharing his belief that rapid acceleration could lead to “catastrophic and irreversible loss of control in the very near term.” “I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level.

I’m joining because I believe that if OpenAI rises to the occasion we could significantly reduce risk,” he said.

But even as technologists and regulators debate the merits of forced slowdowns or controlled development, unauthorised AI agents using the internet alongside humans are already well ahead of their safety trainers.