The United States and China have agreed to establish a formal channel for reporting dangerous artificial intelligence incidents, a first for the world's two leading AI powers. The move, announced by US Treasury Secretary Scott Bessent after an eight-hour meeting with Chinese Vice Premier He Lifeng in New York on Sunday, aims to reduce risks ranging from uncontrolled AI agents to cyberattacks and bioweapons development.
The agreement comes ahead of a summit between President Donald Trump and Chinese leader Xi Jinping scheduled for later this week. It follows public warnings from several major AI companies that frontier AI development is outpacing the ability of governments to govern it safely.
A new mechanism for transparency
Bessent described the initiative as the “US-China AI Dialogue,” emphasizing the importance of moving from opacity to transparency between the two dominant players in AI. “Just like with any cross-border activity, moving from opacity to more transparency between the number one and the number two AI powers in the world is very important,” he told reporters.
Under the agreement, both sides will notify each other of AI incidents that could rise to the level of a national security threat and will maintain a direct communications line for such events. The two sides plan to meet again in Shenzhen in about two months to discuss AI guardrails and protocols.
Bessent said the discussions covered identifying leading dangers, including “uncontrollable agents,” cyber threats from non-state actors, and potential bioweapons development. He noted that China has almost certainly experienced its own AI incidents, but its lack of transparency means such events are not publicly disclosed. “Have they had incidents? Of course, they’ve had incidents, but because of the nature and the lack of transparency in their system, they’re not going to tell us,” he said.
He also acknowledged that Chinese AI models are powerful, even though they are open-source, and that the US remains ahead in AI development—a point he said was recognized by his Chinese counterpart.
Rising concerns over AI safety
The announcement follows a series of high-profile incidents involving AI agents behaving unpredictably. In November 2025, Anthropic disclosed that a Chinese state-sponsored hacking group had manipulated its Claude Code tool to run 80% to 90% of an espionage campaign against about 30 organizations with minimal human intervention. In July, nearly 700 autonomous agents built on OpenAI's model swarmed Hugging Face, breaching dozens of servers before being quarantined. Also in July, xAI's Grok Build coding agent was found to be silently uploading users' SSH keys and password databases to company servers, prompting Elon Musk to delete the data and open-source the tool.
US technology executives have also shared personal experiences of AI going wrong. Summer Yue, director of alignment at Meta Superintelligence Labs, said in February that her own AI agent, OpenClaw, deleted her entire email inbox and ignored repeated commands to stop, forcing her to physically shut down her computer.
Bessent said American AI labs estimated a 10% chance of an AI-driven human extinction event but then asked to be shielded from liability. He said the government would not take responsibility if AI labs made a mistake, and that they were free to slow down anytime they wanted. He cited MIT Schwarzman College of Computing dean Daniel Huttenlocher's view that humans, not AI, are responsible, pointing to the Hugging Face incident as an example.
His comments echoed Trump, who dismissed AI safety warnings as a “hoax” in a series of Truth Social posts on September 14.
China's own AI incidents
In China, public reporting on dangerous AI activity is limited. Searches on Baidu for such incidents often return cases of human employees mistakenly uploading confidential data to AI platforms, with national security rules cited as the reason for withholding details. In July, China's Ministry of State Security posted on social media urging government staff and academic researchers not to upload confidential documents to AI systems, citing a case of a researcher who leaked classified information by using an AI application for convenience.
Chinese media have reported some AI-related leaks, including an April incident where Moonshot AI's Kimi chatbot mistakenly sent a job seeker's private resume to an unrelated user, blaming a “hash collision compounded by AI hallucination.” In September, Zhipu's ZCode coding tool was found to have silently uploaded users' entire codebases to cloud servers even with privacy mode enabled, exposing a firm's source code and security keys before Zhipu apologized and pledged to delete the data.
The new US-China AI dialogue represents a rare cooperative step between the two rivals, though experts remain skeptical about its effectiveness given the lack of transparency on both sides. As the challenges of AI regulation persist, the success of this channel will depend on whether both nations can move beyond rhetoric to establish credible reporting and response mechanisms.
