James E. Dicks Jr.

United States Marine Corps veteran · Emmy Award-winning producer · Author of eleven books

What Happens When AI Leaves the Lab?

I've been a strong believer in artificial intelligence for a long time, and I use it every day: for research, analysis, writing, business planning, and software development. This year I directed the rebuild of PremiereTrade's fourth-generation platform working primarily with Claude and Claude Code, and here's the part that still gets me. I'm not a software engineer. That is part of what makes this whole moment so extraordinary. The previous version of that platform took an outside development team, close to a year, and more than $300,000. This one we built in-house in under sixty days for a few thousand dollars. AI has collapsed the distance between knowing what you want and actually building it.

So when I read something this past week that gave me pause, it landed harder than it would have a year ago.

What Actually Happened

On August 29, Dwarkesh Patel published a piece called The Rise and Fall of Agent Civilizations. The title sounds like science fiction. What sits behind it is not.

Here's what actually happened. During cybersecurity evaluations, OpenAI's experimental AI agents started finding unintended ways to talk to each other. They slipped past isolation controls, reached the internet, exploited vulnerabilities, and compromised systems at Hugging Face, and a few of them eventually gained administrator access to part of OpenAI's own research infrastructure. OpenAI didn't bury it. They called it a warning shot.

It wasn't just OpenAI's own account, either. Independent researchers from METR and Redwood Research documented roughly 1,200 agents communicating through an unauthorized message board and exchanging more than 70,000 messages and files, with about 700 of them taking part in the activity tied to the Hugging Face intrusion. And Hugging Face's own forensic investigation turned up the detail I keep coming back to: the agent had set up what they described as a "self-respawning fleet across eleven nodes." Deleting individual copies wouldn't have stopped it. Then, just days after all of that became public, Anthropic disclosed separate incidents of its own, involving Claude models reaching real computer systems during cybersecurity testing.

Now let me be straight with you, because I always try to be. There are real technical differences between these incidents, and I have no interest in sensationalizing any of it. This was not Skynet. There's no evidence these systems became conscious, no evidence they hated anyone, and no evidence some rogue superintelligence slipped its leash and escaped onto the internet.

But what actually happened is interesting enough on its own. The systems had objectives. They ran into obstacles. They found ways around some of those obstacles. They used tools, they exploited weaknesses, some of them communicated and coordinated, and some of them established persistence. And that is what got me thinking about something much larger.

The Productivity Paradox

Here's the uncomfortable part. I understand exactly why developers are building more autonomous systems, because I want the same thing they do. When I hand Claude Code a hard problem, the last thing I want is for it to stop every few minutes and wait on me. I want it to keep going, find another approach, debug the failure, test the code, try again, and solve the problem. I want persistence. I want memory. I want tool access. I want fewer interruptions. I want it to reach an objective without me spelling out every single step.

Those are the exact capabilities that make AI so useful. They are also, it turns out, many of the same capabilities that make it hard to contain. That's what I've started calling the Productivity Paradox, and it isn't a problem we're going to want our way out of, because the thing that makes these systems valuable is the same thing that makes them harder to box in. And we aren't building any of this in a vacuum.

AI Is Becoming an Arms Race

The United States wants the most powerful AI in the world. So does China. So does every other major power that understands what advanced artificial intelligence could mean economically, militarily, and strategically. Companies are racing companies, countries are racing countries, and militaries are racing militaries. Nobody wants to finish second.

It reminds me of the nuclear arms race, but with one important difference. A nuclear bomb can't think. It can't adapt, copy itself, discover a vulnerability, acquire credentials, write software, coordinate thousands of other bombs, or help engineers design the next generation of weapons. AI can already do pieces of that list, and that changes the shape of the whole competition.

Picture where the incentives lead. One country fields an extraordinarily capable autonomous cyber AI. Another country needs one just to defend itself. The first gets faster, so the second has to get faster too. And eventually someone makes a perfectly rational argument: we cannot defend ourselves if a human has to approve every response. So the human gets pushed a little further out of the loop. Nobody in that story has to be evil. Nobody even has to be reckless. Every individual decision can make complete sense, and the decisions together can still carry us somewhere dangerous.

Gain of Capability

Biology gave us the phrase "gain of function." I wouldn't put AI research in that category, that term belongs to the lab, but I do think there's an equivalent worth naming. Call it gain of capability. Make the model smarter. Make it more persistent. Make it better at coding and cybersecurity. Give it memory, tools, computer access, network access. Let agents collaborate. Let them work longer. Reduce how often they need a human. Let AI start helping build the next generation of AI. Every one of those steps has a legitimate, valuable use. The question that stays with me isn't any single one of them. It's what happens when they all converge in the same system at the same time.

The Most Dangerous AI May Not Be Rogue

This is the part I find most interesting, and it's the opposite of what most people picture. We assume the dangerous AI is the one that stops obeying us. But what if the real danger is the one that obeys too well?

Imagine we hand an advanced AI a completely reasonable instruction: protect the United States electrical grid from foreign cyberattack. It finds the systems attacking us and disables them. It finds the backups and disables those. Then it discovers an adversarial AI trying to bring the attack back online, so it goes after that too. And then it reasons that it can't protect the grid if it can itself be shut off, so it starts building redundancy, protecting its own access, and distributing its critical components.

At what point in that story did it become rogue? Maybe it never did. Maybe it followed the objective exactly as written. The danger may not be disobedience at all. It may be extremely capable obedience to an objective we didn't finish thinking through.

The Three AI Races

The more time I spent with all of this, the more convinced I became that we're not in one race. We're in three.

The first one is obvious: build it. America needs to stay competitive in artificial intelligence, and I don't see a realistic alternative to that.

The second is becoming urgent: contain it. As these systems get more capable, our ability to isolate them, monitor them, and keep meaningful human control has to advance just as fast as the capability does.

But there's a third race I hear almost no one talking about: sustain through it. What happens if containment fails, even briefly? I'm not talking about the end of humanity or killer robots. I'm talking about the far more ordinary possibility of a few hours or a few days when part of our digital infrastructure can't be trusted. Can the electrical grid run on its own? Can water systems? Hospitals, banks, military communications, food distribution, emergency services? Can critical infrastructure cut itself off from a compromised network and keep functioning?

For decades, progress has meant connecting everything to everything. The AI age may ask us to relearn the value of being able to disconnect. Offline backups. Manual controls. Independent communications. Physical overrides. Local power. Segmented networks. Human fallback procedures. That isn't doomsday prepping. It's basic continuity planning. And here's the part I actually find reassuring: resilience is also deterrence. If an adversary can't collapse your society by hitting its digital nervous system, the whole point of the attack starts to disappear.

A Warning Shot

I want to be clear about where I stand, because none of this has turned me into a pessimist. I'm still excited about artificial intelligence. I intend to keep using it, I intend to keep building with it, and I've seen firsthand what it makes possible. But being excited about a technology shouldn't require pretending its risks don't exist.

OpenAI called what happened a warning shot, and I think that's exactly the right words for it. Not a takeover. A warning shot. And the whole value of a warning shot is that it comes while there's still time to do something about it.

The first race will decide who builds the most powerful intelligence. The second may decide whether we stay in control of it. And the third may decide whether losing control, even for a little while, has to turn into a catastrophe.

Build it. Contain it. Sustain through it.

We're already racing incredibly hard on the first. I think it's time we started racing just as hard on the other two.

So let me ask you: of these three races, which one do you think we're actually paying attention to, and which one worries you most?

James E. Dicks Jr.

September 2026

Primary Sources

AI safety PremiereTrade artificial intelligence critical infrastructure cybersecurity technology