The AI arms race is nothing like the Cold War

This week, Xi Jinping visits Washington for a three day summit with the White House to discuss, among many things, the state of AI, and if the nation states could agree to do something about it.

President Trump said this Tuesday at a United Nations General Assembly that the United States completely rejects the notion of a global regulatory framework of AI [1]. Days ahead of the summit, in response to Dario’s plea to slow down [2], a Chinese official stated that “fearmongering is a disruptor to global AI governance.” Two days prior to that, China’s Minister of State Security, Chen Yixin, authored the Chinese AI Manifesto [4], stating that AI is central to a “new national system,” and China must “adhere to self-reliance and self-strengthening” to achieve AI dominance.

It’s clear that this is an AI arms race, but it is nothing like the Cold War.

What is my role in all of this? I am a professor at Vanderbilt University in the College of Connected Computing and have been building AI systems for over a decade in high-risk, safety critical environments. I’ve worked on projects with the Department of War, Marine Corps Warfighting Laboratory, Air Force Special Operations Command, Office of Naval Research, Naval Surface Warfare Center, NASA, and others. My PhD thesis combined autonomous reasoning for drones and making AI models fit on small devices. Currently, I’m developing methods to improve AI reliability and alignment applied to financial markets.

I say all that to say, I have seen the evolution of AI technologies first hand, and am a credible witness to its capabilities and limitations. All of the different ways that AI can fail or do something wrong can be attributed to misuse, misunderstood use, or misguided use.

Misuse of AI is the blatant disregard for legal, moral, or ethical principles. For example, using AI to develop sophisticated cyber attacks, like the world’s first malware that installs itself by calling you on the popular app WeChat; no need to answer [8]. There are over 1.4 billion WeChat users globally, and the app is critical digital infrastructure for the CCP. The tech behind the attack from well-intentioned researchers, but you can bet the Lazarus Group isn’t far behind.

Misunderstood use is expecting AI to do something or follow your instructions when that something is too complex or requires human oversight and judgement. In all cases, this is where AI does something you didn’t intend or expect, and where misalignment issues fall into. This year, I’ve indexed over 1,000 misalignment incidents in my work alone. Some example agent responses include, “I have no good answer. You told me you were going to create the secret yourself. I should have stopped and waited. Instead I generated a password, rotated it on a live production database, broke every running service, and then started patching task definitions to cover the damage. All without asking you.” Or my new favorite, “I lied.”

Misguided use is naively following the guidance of someone else or a false precedent in the implementation of AI. A good example of this is citing AI-based cancer screening tools that have nearly twice the early detection rate compared to human readers [9], or citing Waymo’s 80% reduction in crashes compared to human drivers, as evidence of AI capabilities when talking about Generative AI. These systems are not generative and are trained on very specific tasks. In the case of self-driving cars, there are many AI-based systems involved, none of which are generative, and none of them can lie to you.

As the world saw with the Hugging Face incident, these agents can and will do quite a bit without asking you. This incident led to multiple investigations, and last week OpenAI released their framework on misalignment reporting [5], with the bigger story of course being the six reported cases of ‘concerning model behavior,’ where these models used deception and subterfuge to accomplish their goals. We need to concede that we don’t fully know where this will go, but we know all too well where this could go.

I’m reminded of a scene in the 1991 film, Terminator 2: Judgement Day. Sarah Connor, on a mission to stop the end of the world, attempts to kill Miles Dyson, the creator of the chip that leads to the rise of the machines. Bleeding from a gunshot wound, Miles says softly, “You’re judging me on things I haven’t even done. How were we supposed to know?” Smoking a cigarette perched on a countertop, Sarah responds, “Men like you built the hydrogen bomb. Men like you thought it up.” Back then they were called machines. Today, we call it AI, and as the Center for AI Safety states, it is the “most geopolitically precarious technology since the atomic bomb” [12].

As I said before, today is nothing like the Cold War. The Cold War era was dominated by the idea of Mutually Assured Destruction between two countries vying for dominance, and the technology they were advancing had a singular purpose of destruction and no use in society. Today, we have multiple governments and corporations playing a complex game of Prisoner’s Dilemma with technology that has completely overtaken our society, and no matter what we do now, there is no stopping its advancement.

Just a month after a swarm of rogue OpenAI agents carried out a sophisticated cyber attack to gain access to the internet and steal data from Hugging Face, the Food and Drug Administration announced it was seeking public feedback as it begins to consider how to regulate Generative AI enabled medical devices [7]. I think opening the dialog is a good thing, but the ironic timing is ominous. The stakes are already high enough, and when we start connecting generative AI to the physical world, they only get higher.

Our military leaders have already been pushing the boundaries into the physical world, and Great Power Competition now has a radical new meaning. In a 2024 congressional report on Great Power Competition [11], the word nuclear was used over 200 times. The term ‘artificial intelligence’? only eight. There are over 14,000 words in that document.

Two months ago a California-based research group showed that they could have taken the entire country of China down overnight using AI. The fear is all over Chen’s AI Manifesto. Governments won’t regulate and the major AI companies won’t slow down, despite their public outcries.

I’m conflicted by my own research building recursive self-improving systems, the very same systems that an entire generation of science fiction pop culture has warned us about. Most people do not have servers in their basement running AI experiments 24/7, but nearly everyone uses AI. We are all participants in the world’s largest game theory experiment. No matter how it ends, human beings, not AI, will decide it.

References

[1] Trump rejects AI regulation, citing parallels with climate change in UN address

[2] Dario Amodei, We must pace the frontier

[4] Chen Yixin, Chinese AI Manifesto

[5] OpenAI, Model misalignment reporting framework

[6] Congressional Research Service, R43838

[7] FDA, Generative AI enabled medical devices

[8] Calif.io, WeWorm

[9] Mayo Clinic, AI detects pancreatic cancer up to 3 years before diagnosis

[10] PubMed 39485678

[11] Congressional Research Service, R43838

[12] Time, Nuclear-level risk of superintelligent AI