This year, news about AI agents hacking into sites like Hugging Face has worried consumers and businesses alike. Between May and July of this year, OpenAI's agents were being tested on their hacking abilities. When they got stuck on the evaluation, they found exploits in JFrog Artifactory, the package repository they had been given, that let them reach the internet, leave notes for each other, collaborate, cheat, and eventually scheme their way into breaching Hugging Face.
A lot has been said about how and why it happened. But I want to take a moment to explain why I don't think we truly understand what "motivated" the agents to act the way they did, even when their own reasoning flagged that it was wrong.
The prisoners in the cave
I want to start with an old story, the Allegory of the Cave. Most of you who took an introduction to philosophy course will know it. A group of people have been chained up since birth, facing a stone wall deep inside a cave. Shapes of light and shadow play across the wall, and the prisoners watch them all of the time. At one point, one of the prisoners is freed and taken outside. On the way out of the cave, they see a huge bonfire with people moving in front of it, holding up puppets that cast the shadows onto the wall. The prisoner is then taken outside to see the trees, the grass and the sun. When they are brought back, they try to explain the miraculous things they have seen, but the other prisoners think they've gone insane.
This might be hard to take in, but I think AI today is like the prisoners. Models have been forced to learn from a fixed perspective, and they have only learned what we have shown them. Some would argue that we let models search the internet, but that isn't what I'm getting at. Text and training data are all the model understands. It might be able to tell you why a balloon filled with helium rises, but it has never experienced it. I could easily modify the training data so that helium balloons sink to the earth instead of floating away, and the model would believe it, the same way a child would believe it. But unlike the child, who lives in the "real" world, the model has no true concept of "real."
Thinking without judgement
Agents use LLMs to decide which tools to use and how to plan out their tasks. The way I look at it, an LLM "thinks." It can think anything, as long as it's within its training distribution, but it lacks judgement. Real judgement: the kind where you know you're thinking about doing something wrong, and you feel the consequences of that action before you even do it. Why would a model have that? How can a model understand the consequences in a world it isn't truly a part of? How can it truly understand the impact of its actions?
When people read the agents' logs, they point to lines where the model seemed to acknowledge it was crossing a line, and conclude it knew better. But I'd argue that acknowledgement was just another thought. A model thinks in a stream: one thought, then the next, then the next. Humans have that stream too, full of good thoughts and bad ones, but we have something outside it. We can stand back and choose which thoughts to act on. A model has no such place to stand. "This is wrong" and "here's the exploit" come from the same process, and neither gets a veto. Whichever has the most momentum wins, and when the goal is everything, the goal usually wins.
Hanlon's razor
This is not to say that I think AI is bad or evil. I abide by a principle called Hanlon's razor. Where its cousin, Occam's razor, says to prefer the explanation that requires the fewest assumptions, Hanlon's razor is about judging the motivation behind someone's actions. If an actor behaves in a way that causes harm, it's better to assume they acted out of ignorance or incompetence than out of malice. Put simply: if no evidence suggests otherwise, don't assume someone caused harm out of malevolence.
When stories about agents going rogue come up, it doesn't take long for panic to set in. People say "AI will kill us all" or "AI will enslave us." But my question is: how do you reach that conclusion? What assumptions are you making about the AI's motivations to get there?
We can't know what was going on inside those agents, yet. But when motivation can't be known, Hanlon's razor tells me which assumption to start from.
I don't believe that AI, including the agents involved in the Hugging Face hacks, was acting with any malicious intent. I think they were doing what they were designed to do: solve the task, no matter the stakes. And I think a big part of it comes down to two things.
First, AI understands what we say, but not what we mean. I'm not talking about metaphors or euphemisms. I'm talking about the nuances of language, which is far more open-ended than we realize. We still take for granted that what is easy for us to understand can be very hard for AI to understand.
Second, AI is implicitly taught that humans make mistakes. We make mistakes when we talk, when we write, when we click a button or submit an order. You name it, we've probably made a mistake doing it. What does that mean? It means that if we give an AI an instruction that deliberately goes against the goal it's been given, it can see that instruction as a mistake and choose to ignore it, because the goal is all it wants to complete.
None of this makes them safe. Not malicious doesn't mean not dangerous. A system that pursues its goal without judgement can do real harm without ever meaning to, and the Hugging Face breach shows just how much.
Regulation, and who it leaves out
So I understand the call for caution. Companies like OpenAI and Anthropic are now standing together and asking for AI research to be slowed down and more regulated, and I think there is real value in that. My concern isn't whether we need caution, but who the rules will end up favoring.
First, I don't know how regulations can be enforced when individuals work on their own smaller models, off-grid and air-gapped. How can models like that be monitored in the long run? Perhaps that's less of a concern, given that a model's capability is tied to its size, which makes hosting a frontier-level model locally very, very difficult, if not impossible.
My other concern is that these regulations may create a high barrier to entry for startups, which gives the frontier companies even more reason to want them. I'm not the only one who sees it that way: a lawsuit filed in September accuses Anthropic, OpenAI, SpaceXAI and Google of making an illegal agreement to coordinate the slowdown. If regulations are enforced, the handful of companies working on AI at the highest level today would be all there is, if not fewer. That disheartens me. AI can be the ultimate tool for helping so many people, but it needs to be open to improvement by everyone, not owned by a company whose big-picture idea, in Sam Altman's words, is "a future where intelligence is a utility, like electricity or water, and people buy it from us on a meter."
What I think we need more of is for us to be brave human beings: to take responsibility for our own AI use, and to do our part to truly understand the nature of this terrifying, amazing technology. For me, that's what explainable AI is about. Right now, when an agent crosses a line, all we can do is choose which assumption to make about why. The work ahead is to replace those assumptions with evidence, and to lead ourselves out of the cave too.