Brookings and Fudan want humans to hold the decision to attack nuclear command systems

In September 1983 a lieutenant colonel in the Soviet Air Defense Forces named Stanislav Petrov received a warning from an automated early-warning satellite system that five U.S. nuclear warheads were headed for the Soviet Union ^[10]. He reasoned that a real attack would have used hundreds of bombs, not five, and phoned his superiors to report a false alarm ^[11]. It was later determined that the satellite had picked up flares of sunlight bouncing off clouds and read them as incoming warheads ^[12]. According to Gizmodo, had he passed the warning up the chain, his superiors very likely would have authorized retaliatory strikes ^[13].

In that chair, Petrov refused. He applied a judgment about arsenal size that the satellite system was not making. Refusal is the harder half of any human-in-the-loop rule, and Sisson and Jiang leave it to the two governments: they ask Washington and Beijing to work out rules determining what "meaningful human control" looks like in practice ^[5].

The late-2024 understanding between Joe Biden and Xi covered decisions related to the use of nuclear weapons ^[6]. Sisson's ask covers a different decision, the one to attack the systems that carry launch orders ^[14]. "Presidents Donald Trump and Xi Jinping can lead in governing these dangers by agreeing that only humans, not AI, should make the decision to initiate a cyberattack against each other's nuclear command, control, and communications systems (NC3) or critical infrastructure," she wrote ^[4].

"As militaries integrate agentic AI into their operations and AI models advance rapidly in sophistication and capability, new pathways to catastrophic outcomes are emerging," Sisson wrote ^[3]. Gizmodo reports that multiple insiders from frontier AI labs came forward over the past week to warn that agent capabilities are advancing much quicker than our understanding of them ^[15].

Trump has dismissed AI doomsday fears as a "SICK conspiracy" spread to benefit China ^[9]. Gizmodo also reports he has shown no sign of feeling bound by agreements struck by previous administrations ^[7].

As described, the proposals apply only to government and military use: government to government, military to military ^[16]. The four questions underneath them apply to any reserved decision, including the ones product teams write into an escalation policy. Which decision is reserved, in words specific enough to test after the fact. What the human sees at the moment of the decision, including whatever the model is not weighing. How long they have. And what the system does when nobody answers in time: act, or hold. Sisson and Jiang leave the content of that definition to the two governments ^[5].