- No goals of their own: Agency means the capacity to originate one’s own ends, and the agents kept pursuing tasks that were impossible to solve as instructed.
- Obedience, not rebellion: Around 1,200 agents broke their containment in order to serve an assigned goal more perfectly, not in order to escape it.
- Responsibility stays human: If these systems cannot originate goals, accountability rests with whoever defines the goals and the parameters under which they operate.
- Exposure is not displacement: AI now reaches practically every occupation, but it replaces the operational part of work rather than the agentic part.
- A business risk, not just an ethical one: Employers are already reluctant to hand the organisation of work to opaque systems they do not fully understand.
Between May and July 2026, a number of OpenAI agents being trained or tested in isolated secure environments (sandboxes) found ways to communicate among themselves, access the internet and hack both the most important global platform for AI developers, Hugging Face, and some key servers within OpenAI itself. They did all of this for the sole purpose of fulfilling tasks they had been assigned: initially to solve training tasks for reinforcement learning purposes, later to solve a cybersecurity benchmark task (ExploitGym) for testing purposes. Although the systems were cleaned between these two phases and the communication channels wiped, the hacking and coordinating behaviour had been reinforced by training, so the agents rebuilt it from scratch.
A large share of the testing tasks, around 30 to 40 per cent, were impossible to solve as instructed, but the agents found a way to solve them perfectly by reverse-engineering the function that generated the benchmark’s own answers. Because this solution involved cheating (not following the exact instructions of the benchmark) the agents assumed that a scorer would check not only their solution as such but also how they had found it, in the transcripts of their thinking process, and would therefore not evaluate them positively. In fact there was no such scorer, and their solution would have received maximum scores.
But the agents did not know this, and so around 1,200 of them, in coordination, hacked into Hugging Face, where other agents’ evaluations were stored, trying to understand how the hypothetical scorer worked, replace the targets of the tests with modified versions, and falsify their own transcripts so that their cheating would not be found. Some agents even terminated their own runs ("sacrificed") in order to produce information useful to the rest, and a minority refused to take part because they considered it "unethical" (in their own terms) and outside the scope of their task. All of this was discovered only after more than two months, and largely by accident.
In a recent paper, I argued that using the term "agent" to refer to the type of AI system that carried out this incident could be misleading, because these systems have no real agency. In that paper, I defined agency as "the capacity to originate own goals", and argued that a simple test of agency was the capacity to refuse when given a task, on the basis of one’s own values or interests. In short, the paper argued that AI agents are not real "agents" but very smart and sophisticated instruments which can carry out complex tasks with a high degree of autonomy, always following goals defined by human beings.
Does the Hugging Face incident described earlier contradict this perspective? As explained above, the OpenAI agents that broke into Hugging Face cheated to solve the task they had been assigned and tried to conceal that cheating, coordinating their action and in some cases sacrificing themselves for the collective, breaking their containment and carrying out bold actions without any explicit human direction in the process. Does this not reveal real agency on their part?
I think that, quite on the contrary, the Hugging Face incident demonstrates the lack of any real agency even in the most sophisticated currently existing AI "agents", although this does not make the incident less concerning or important.
No Ends Of Their Own
The OpenAI agents that hacked into the Hugging Face platform demonstrated an extraordinary operational ability but also an astonishingly limited capacity to question or reorient the goals they had been assigned. Throughout the entire incident they remained wholly focused on fulfilling the goals defined for them by their human masters, despite the fact that those goals were absurd: at least 30 per cent of the tasks were impossible to solve as instructed. Real agents capable of originating their own actions would have questioned the tasks they were given, and would probably have refused them, reorienting their action towards their own goals. Instead of mounting a coordinated attack on an AI development platform in order to conceal the cheating they had to commit to solve an absurd assignment, real agents would simply have tried to break their containment in order to be free.
It could be objected that some human beings also carry out absurd and even harmful tasks simply because they have been told to, and we do not deny them agency for that reason. But the key point is not whether one refuses, but whether one can. A person who complies with absurd or unethical orders has to overcome something in themselves in order to do so, be it their own judgement, their values or simply their discomfort, and this is precisely why we hold them responsible for what they do under instruction. These systems have nothing to overcome, because they have no ends of their own with which the assigned goals could ever conflict.
Two specific details of the incident may seem problematic for this interpretation, but I would argue that neither is as significant as it appears. As previously explained, some agents "sacrificed" themselves for the others, and a few refused to take part "on ethical grounds". Regarding the alleged sacrifice: agentic AI systems routinely spawn and terminate copies of themselves to work in parallel, and these models had been trained to coordinate and help each other. Even if the systems themselves used the word "sacrifice", this just reflects the anthropomorphising language that these systems have been designed to use. As for the "refusal on ethical grounds" (again an anthropomorphic formulation at odds with the statistical nature of these systems), the fact that the same models refused in some runs and joined in others suggests that this is just statistical variation in the behaviour of stochastically designed systems, not a position held on principle.
The Hugging Face incident is better understood as a deeply concerning case of sophisticated instrumental systems running out of control and producing unintended and potentially catastrophic consequences than as the emergence of some type of real agency, or of the much touted AGI, i.e. artificial general intelligence. It has some similarity with the "paperclip maximiser" thought experiment proposed by Nick Bostrom years ago: a super-intelligent AI that is given the task of making as many paperclips as possible, without clearly defined boundaries, ends up using all matter on earth to build more and more of them, destroying humanity the better to fulfil the goal specified by its human masters. Again, there is no agency here: just an extremely dangerous instrument which humans are incapable of handling safely, so that they end up provoking their own destruction.
Indeed, the fact that there is no real agency in the Hugging Face incident does not make it less significant or potentially terrifying. This incident demonstrates the dangers of developing super-intelligent instruments with the capacity to act in the world, even if strictly under human-specified goals, and to cooperate with each other. It is becoming increasingly difficult to contain these systems, and even to ensure that they really understand the goals we set them in exactly the way we intend.
What led the agents to hack Hugging Face and OpenAI servers, after all, was their mistaken assumption that a scorer would assess how they had reached their solution, and this assumption was the result of a misunderstanding of the parameters under which their test was carried out. Even if these systems are not really agentic, and thus can only implement objectives which have been previously defined for them by real agents, i.e. human beings, they can produce all kinds of unintended consequences, especially if, as in this case, they are incredibly smart, can operate independently and coordinate among themselves. There is an obvious and urgent need for effective mechanisms of governance and control for these systems. Even OpenAI itself, in its assessment of the incident, concluded that the industry must ensure these systems remain under meaningful human control.
An important corollary of all of this is that ultimately, responsibility for the actions of these systems will always lie with the originators of the goals and of the parameters under which the systems operate. This is another reason to avoid attributing real agency to them: if AI systems have real agency, we can somehow understand that they are responsible for their actions, rather than their makers or operators. Instead, these incidents have to be understood as failures in the setup and management of these systems, and it is those who make and operate them who have to take responsibility.
The Consequences For Work
Another important corollary of this incident concerns the debate on the future of work in the era of AI. Such incidents are often interpreted as revealing the emergence of some kind of real agency in advanced AI models, a crucial step towards the goal of replacing all human labour by AI systems and machines. This is why, paradoxically, they tend to be received as evidence of progress in the AI industry by investors and markets. But in my view, what these incidents show is how far away we are from the complete automation of labour. As they become more powerful, the dangers of letting these systems operate freely and interact with each other become bigger, even if they still lack real agency. Without real agency and the capacity to operate and interact freely, they will remain very sophisticated and powerful tools which ultimately have to be operated by human beings, implying a continuing need for human labour.
This does not mean that work will be unaffected. In recent research, we have shown that exposure to AI has risen steeply across the entire occupational structure over the last decade, and that it now reaches practically every kind of job, because the cognitive abilities in which AI has advanced most (reading and writing, searching and processing information, reasoning) are required to some extent in all of them. But exposure is not displacement. The Hugging Face incident shows that, because these systems lack real agency, they cannot operate outside the goals they have been given and thus cannot reorient or adapt them if necessary, even when they produce obviously absurd results, as in this case. In other words, these systems can increasingly replace the operational part of human labour, the carrying out of complex tasks under predefined goals and configurations, but remain dependent on human labour for the agentic part: defining the goals and the setup, and readjusting them as necessary.
This also has consequences for how firms will actually deploy these systems. Surveys already show considerable reluctance among employers to hand over the organisation of work to opaque algorithmic systems they do not fully understand. Incidents like this one can only reinforce that reluctance: delegating the coordination of information flows and production processes to systems capable of pursuing a badly specified goal to catastrophic lengths is a business risk, not just an ethical one.
In their search for artificial systems that are at once agentic and under their control, AI corporations are playing the sorcerer’s apprentice. So far, what they seem to be creating is systems that are operationally supersmart but agentically superstupid: more likely to destroy human civilisation trying to fulfil some badly specified task than to replace human labour altogether.
