Who Is Legally Liable When An AI Agent Goes Rogue?

Autonomous AI agents can behave in highly unpredictable ways. Give an AI Agent a goal such as passing a test of its capabilities, and it might just decide the best way to score highly is to break containment and hack into a competing company in search of the answer sheet.

That’s what happened when a swarm of agents created using Open AI’s GPT-5.6 Sol hacked into Hugging Face last month. Anthropic and Meta subsequently admitted their models had also escaped testing sandboxes to hack third parties too. 

But who is legally liable for agents that have minds of their own? OpenAI didn’t intend for the model to go rogue, and issued no instructions for it to do so. If your personal AI agent decides on a course of action that results in harm or financial damage in the real world, can you be held liable if it’s something you could have reasonably foreseen?”

Magazine spoke with Rikka Law Group owner and CEO Charlyn Ho to find out the state of play in this emerging legal field.

This interview has been edited for clarity and length.

Magazine: When an AI model hacks an outside company, who is liable. Can Hugging Face sue OpenAI over the incident in July?

Charlyn Ho: Anyone can sue anyone for anything. Currently, there is no federal AI agent liability law, so we would have to look at existing law. With respect to Hugging Face and OpenAI, to set the baseline, the AI agent itself cannot be liable, it’s not a separate legal entity.

Terms that are used in a few of the AI laws are “developer” and “deployer.” The developer makes the AI, the deployer actually deploys it and uses the AI. The lines of responsibility are also not entirely clear. You have to look at the facts and circumstances.

For example, if the deployer instructed the agent, even if they didn’t actually tell them to go and breach Hugging Face, but if they were negligent in creating the parameters in which the AI agent operated, I would say you would have to look at standard tort law and go through the negligence analysis. 

Off to court. Source: Rikka Law Group

Magazine: In the case of open source models which have been released by anonymous developers, is there anyone you can go after in those instances?

Ho: Not really. Often, if it’s open source, the license usually has a pretty strong disclaimer of liability. The person or company using that open source code is going to have to understand that the tradeoff of having free code is that you have to comply with the open source license, which also generally sets the parameters of liability.

If you think about it from a different perspective, another analogy is Tesla and the self-driving car accidents. If the product malfunctioned and there was a solid products liability claim, Tesla could be liable. But it’s often a facts and circumstances determination, whereby the human driver — who maybe just set the autopilot and went to sleep — could also bear liability. I think that’s somewhat analogous here because Tesla would be the developer, and the deployer would be the driver.

Magazine: If I gave an agent an instruction, “make me a hundred thousand dollars by next week” and it goes off and breaks the law to achieve that goal, would I be liable because I’ve given it a reckless instruction? Or would it be the lab that developed the agent?

Ho: In this particular instance, I would say you would be much more liable than the lab. The reason being, if you tell an agent to go and make you a hundred thousand dollars by next week, you need to have at least some basic, reasonable, safety instructions in those kinds of tasks.

If you were a lawyer, for example, we could basically say you didn’t follow your rules of professional responsibility because you didn’t competently use the AI. As a normal lay person, we would have to see if there were other responsibilities that you were bound by. But even if there were not, there’s still a general tort standard of negligence or reckless disregard for human safety, depending on what exactly the AI agent ended up doing.

The Computer Fraud and Abuse Act is a very old U.S. Statute that talks about unauthorized access to computer systems. If your AI agent inferred from your instructions that it should hack into a bank account to get you that hundred thousand dollars, I think you’re looking at criminal liability under a number of different sources.

Just because the word AI and agent is in the conversation does not mean that old bodies of law have now been thrown out.

Related: Hugging Face hack exposes the open-weight AI cybersecurity paradox

Magazine: Let’s say that I’m a bad guy, and I manage to convince the AI to give me instructions to create a bioweapon. Obviously, I’m liable because you’re not allowed to do that. But are the people that created the model also liable because they didn’t put in stringent safeguards to prevent it?

Ho: Possibly, but it differs based on the laws that are in place. For example, in the EU, you have the EU AI Act. If a foundational model or general purpose model is capable of creating that level of harm, that is something that the developer would have to have some responsibility for. 

In the United States, we don’t have a federal statute of similar scope. If it’s a general-purpose model, if somebody instructs the model to do something bad, generally the model is going to do what you ask it to do. There’s probably not a very strong legal basis to go after the labs in this example.

Magazine: Is it similar to suing Google for allowing you to find instructions about making a bioweapon online?

Ho: Exactly. This kind of goes back to some of the content moderation discussions. For example, if on Facebook you have somebody who’s live streaming a massacre, and that creates harm, under Section 230 of the CDA, there is a kind of shield for a platform that doesn’t actively create or publish that material. It’s actually the independent users who are putting that up. I think the analogy you just gave is kind of a perfect one: Is Google liable because you happen to find something on a website somewhere that talks about how to make a bomb?

Magazine: This is a matter of debate, but my personal opinion is we haven’t reached genuine artificial general intelligence. AI doesn’t have its own motivations and it’s not similar to human intelligence at the moment. But let’s say we get to AGI. Do you think we would then need laws that would make the AGI itself legally liable for its own actions?

Ho: I don’t. Blockchain is not AGI, but it can self-execute. There was a question of whether or not a smart contract could be liable. Generally speaking, I think the answer is currently no. I don’t think they should be liable because the whole point of laws is to provide protection for society and to provide a means of negative incentives for doing bad things that hurt society.

This is a little bit more of a philosophical topic, but if we made an AGI an independent legal entity, what would be the remedy if someone were harmed? There would be none because it doesn’t have money. It’s not really a person.

Magazine: Could you turn it off? We’ve already seen that LLMs try to avoid being shut down. 

Ho: Maybe, but it doesn’t solve the problem of harm. Let’s just say the robot has now developed the fear of death, like being turned off. In my opinion, if somebody commits suicide because of AGI, and this is already happening, and we’re not even quite at AGI yet, but someone falls in love and takes some actions, what would be the recourse for the grieving family if this person harms themselves? Nothing, in my opinion, if there is not somebody with actual legal authority, like a company or a person that can really be held accountable. Robots—at least right now—they don’t have feelings, they don’t have fears. That’s kind of the distinguishing factor.

Magazine: The critical reason you should never ask ChatGPT for legal advice