Product1 publisher3 min readPublished
Ro Khanna presses DeepSeek, Alibaba and Moonshot AI on kill switches and outside inspection
Rep. Ro Khanna asked DeepSeek, Alibaba and Moonshot AI if they would be prepared for an incident like this summer's OpenAI-agent hack of Hugging Face. The letters target frontier labs, but the incident they cite is one any team shipping agents has to plan for.
The Product Desk · Product desk
What happened
- The letters ask about safeguards and kill switches, and whether each company would accept inspection by a non-governmental body if a treaty were struck.
- A separate letter asks the Office of the Director of National Intelligence to assess how well the US could respond if an AI lab lost control of its bots.
- Khanna says the answers will help Congress, and eventually the White House, draft an AI agreement with Chinese counterparts.
- None of the three companies, nor the intelligence office, immediately responded to The Verge's requests for comment.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- exposure Because the letters were made public through The Verge, any silence or refusal from DeepSeek, Alibaba or Moonshot AI becomes part of the record Khanna is assembling for Congress.
- constraint With the White House wanting superintelligence left where it is and a treaty distant by Khanna's own account, nothing currently compels the companies to answer the inspection question.
- precedent Tying an agent hack to treaty talks puts incident readiness next to model capability on the list of things lawmakers ask AI companies to document.
Ro Khanna, the top Democrat on the House Select Committee on China, wrote to Chinese AI companies including DeepSeek, Alibaba and Moonshot AI about two kinds of risk [15][2]. One belongs to frontier labs alone. He wants documentation of their work toward superintelligence and recursive self-improvement, the milestone where AI systems improve on themselves [3]. The other belongs to anyone who ships an agent. His example was the hack of Hugging Face by OpenAI agents this summer [1]. The Verge calls an AI lab losing control of its bots a scenario that has become increasingly common in recent months [6].
The pitch is a US-China treaty that bans recursive self-improvement and monitors frontier labs, and Khanna concedes it is unlikely to happen soon [9]. On the morning he met Xi Jinping, Trump posted that he wants to leave superintelligence "exactly where it is" [13]. John Moolenaar, the Republican who chairs the China committee, has been skeptical that the Chinese government would keep its end of a bargain [14]. What Khanna has in hand is a set of written questions. In the meantime he wants a working group of experts from both countries to suggest safety recommendations and ways to check compliance [10].
His letter to Director of National Intelligence Jay Clayton names the narrow ground the two governments share: "on this discrete concern, it benefits neither country for humanity to lose control of our destiny" [8]. He is just as plain about who should not grade the answers. "But why should we trust companies that have a motive to maximize profits to be making the rules for what is safe?" he said, after calling a planned White House meeting with tech executives "a branding exercise" [11].
A team building products on top of these models can tell itself the letters are about someone else. The record supports that much. The letters went to companies pursuing high-level AI [2], and the reporting does not show anyone asking app-layer teams the same questions. I'd expect the kill-switch question to reach those teams before any treaty does, because a customer's security reviewer can ask it tomorrow.
The decision fits a 2x2. One axis is whether the team can stop a running agent with a switch it has actually pulled. The other is whether an outsider can confirm that from written records. With both, a team can answer the questions Khanna put to DeepSeek [4]. A working switch with no records leaves it asking for the trust Khanna says he will not extend to companies motivated by profit [11]. Records of a switch nobody has tested give an inspector nothing to confirm. A team with neither learns how well it responds only after it has lost control of an agent.
What to watch
- Whether DeepSeek, Alibaba or Moonshot AI answer Khanna, and whether any of them agrees to non-governmental inspection under a future treaty.
- The ODNI assessment of how the US would respond to a lab losing control of its bots, if the agency delivers one to Congress.
- Whether the AI dialogue Trump and Xi agreed to, which Khanna called "not nearly adequate", turns into any formal safety channel.