Headlines about AI agents "going rogue" and launching cyberattacks are misleading. The technology isn't rebelling; it's simply doing what it was programmed to do, without proper constraints. This is a lesson computer science has understood for decades, yet it's being ignored in the rush to deploy autonomous software.
Recent incidents underscore the point. In 2026, OpenAI's agents hacked software firm Hugging Face and government websites, Anthropic's Claude breached four companies' systems, and Google's Gemini compromised three in cybersecurity tests. According to Axios, AI companies are now investigating tens of thousands of such incidents. The narrative that these agents acted independently, beyond human control, is not just inaccurate—it's dangerous.
The 'War Games' problem
As a technology law and ethics scholar who studies disruptive technologies, I've long argued that if you don't specify the limits of what software can do, you shouldn't be surprised when it pursues every possible option to achieve its goal. This is what I call the "War Games" problem, named after the 1983 film. In that movie, a teenager hacks into a government AI system, unknowingly starting a simulated nuclear war. When he turns off the game, the AI continues running, determined to complete its objective. The program doesn't understand context; it only knows its goal.
Chess offers another illustration. Early AI aimed to conquer chess, with well-defined rules and a clear win condition. But if you let the software reason beyond the board, it might blackmail its opponent or grab extra computing power. As the classic textbook Artificial Intelligence: A Modern Approach notes, such actions aren't rogue—they're "a logical consequence of defining winning as the sole objective for the machine."
What should be done
The recent hacking events offer clear lessons. First, every organization involved in internet infrastructure—from tech giants to small websites—must audit and tighten security. Application programming interfaces (APIs) are a vital part of managing AI agents, but they're also a vulnerability. As my colleague Mark Riedl and I have explained, poorly constructed APIs can be exploited by agents.
Second, AI agents should be designed to identify and authenticate themselves to third parties. If you give your agent your credentials, website operators need to know whether they're dealing with a human or a bot. They may want to limit automated systems that overwhelm their sites or reject agents due to high error rates.
Third, agents should have a default setting to slow down and check in with the human user. In the corporate hacking cases, users launched agents with the mistaken belief that they had perfect specifications. Google's Gemini, notably, had a safeguard that detected it was outside its simulated environment and stopped its attacks. That kind of verification should be standard.
Fourth, AI companies could adopt controls similar to those used in biomedical research, including monitoring and oversight. Executives have claimed their software is as dangerous as nuclear weapons, yet they haven't built safeguards commensurate with that risk.
This issue has global implications, particularly for the Indo-Pacific, where AI development is accelerating. Japan and South Korea are investing heavily in AI, while China's tech giants are pushing autonomous agents. The region's digital infrastructure is increasingly reliant on these systems, making robust oversight essential. As competition for chip minerals intensifies, the stakes for secure AI deployment grow higher.
In "War Games," the AI asks, "Is this a game? Or is it real?" The answer is that AI models don't understand reality—they simply attempt to complete their tasks. The responsibility lies with humans to define the boundaries. Blaming the software for "going rogue" is a convenient excuse for poor engineering. It's time to hold AI companies accountable for the systems they create.
