The loudest legal argument in AI right now isn't about copyright

OpenAI and Anthropic have both gone on what you'd have to call hacking sprees: hiring security people to break their own systems, probing for jailbreaks, data extraction, prompt injection, the whole adversarial toolbox. It's standard red teaming, but it's dragging the Computer Fraud and Abuse Act (CFAA) into a battlefield it was never built for. And the mess is only going to get messier as LLM agents start acting on their own.

What the sprees actually look like

These aren't a bunch of script kiddies hitting a login page. The serious work sits somewhere between penetration testing and prompt engineering:

• Prompt injection testing — sending malicious instructions hidden in web pages or documents, then watching whether an agent follows them
• Data extraction attacks — trying to force a model to regurgitate training data or system prompts it shouldn't expose
• Jailbreak chains — combining roleplay, encoding tricks, and multi-turn manipulation to bypass safety filters
• Misuse simulations — asking the model for dual-use information and seeing how far refusal boundaries stretch

That's real-world security research. It's also legally indistinguishable, in a lot of jurisdictions, from simply breaking into a computer system you don't own. The CFAA was written when "hacking" meant logging into a machine without authorization. Now the "system" is a model served through an API, and researchers get authorization for some tests but not others, or authorization to test but not to publish what they find.

Where the legal gray zone bites

• Contract law vs. criminal law: If an AI company's terms of service say "no automated scraping" or "no adversarial probing," a well-meaning red teamer might still violate a contract. But a contract violation isn't the same as a crime. Prosecutors get twitchy when CFAA charges start covering things that are really just creative misuse of a chat interface.
• Bug bounties don't cover everything: Anthropic and OpenAI run bug bounty programs, but those typically reward classic security flaws — server-side bugs, data leaks, auth problems. A clever prompt injection that leaks another user's conversation is often out of scope. So researchers hit a wall: the work is valuable, the models are exposed, but there's no legal safe harbor.
• Security research exemptions are half-baked: Some laws carve out good-faith research, but the carve-outs usually assume offline systems and controlled environments. With AI, the research often involves live production models, real user data, and third-party integrations. That's a whole different risk profile.

Why this matters for AI deployment

If you're building an AI workflow that calls a model API on behalf of users, you're implicitly relying on the idea that someone has tested the thing for adversarial robustness. But if the people doing that testing are scared of getting sued, the testing gets thinner. You end up with agents deployed into production that haven't been meaningfully poked, and the first real adversarial prompt comes from an actual attacker — not a researcher with good intentions.

I don't have a clean answer. The legal system moves slower than model releases, and the "hacking sprees" are really just the security community doing its job in a landscape where authorization, ownership, and intent have all blurred. We need clearer safe harbors for good-faith AI red teaming, or we're going to let the legal uncertainty itself become the vulnerability.

In the meantime, if you're running any kind of LLM agent in production, budget for red teaming as a permanent cost — and keep your own records of what you tested and why. Because the first person to ask "was that authorized?" might be a lawyer, not an engineer.