Skip to content

Science1 publisher3 min readPublished Updated

Three U.S. agencies name six China-based AI companies as distilling frontier models

The NSA, FBI and CISA say the six pulled capabilities out of Claude, GPT, Gemini and Grok variants through proxy accounts since late 2024, and they want providers watching subscription-to-usage ratios and query throughput.

The Scientist · Science desk

Illustration accompanying Three U.S. agencies name six China-based AI companies as distilling frontier models

What happened

  • The NSA, the FBI and CISA issued a joint cybersecurity advisory saying China-based AI companies are running industrial-scale knowledge distillation campaigns against U.S. frontier AI models.
  • The agencies assess that the activity narrowed the performance gap between Chinese models and U.S. systems while cutting research, development and compute costs.
  • Alongside the naming, the document lists indicators of whether a model is being distilled, the tactics and procedures involved, and recommended mitigations.

Compiled by The ScientistSomething wrong?How this is made

Why it matters

  • decision Every provider serving a frontier model through an API now has to set a numeric line on account behavior and query volume, and that line prices its own false positives against paying customers running large legitimate workloads.
  • exposure The reachable surface is the distribution chain. A provider selling through clouds and aggregators cannot see who sits behind a resold subscription, and the advisory treats that opacity as the attacker's main tool.
  • constraint Deliberately degrading outputs for suspected distillers puts a cap on the fidelity a provider can promise anyone its classifier flags, and the recommendation arrives without a measured dose or a measured cost.
  • precedent Naming six operating commercial firms, and not a malware family or an intrusion set, makes the company name itself an indicator that cloud resellers and enterprise buyers can be expected to screen against.

Distillation, in its ordinary academic sense, is a training method. A smaller student model learns from a larger teacher's outputs instead of from raw data. Nothing in that needs the teacher's weights. Queries and answers are enough, and that is why the advisory traces the adversary lifecycle from initial access through exfiltration and maps it onto MITRE ATLAS, with a pointer to complementary NIST guidance [10]. The document also describes chain-of-thought extraction and prompt injection used to surface hidden model reasoning [9].

The indicators the agencies offer are behavioral. Providers are told to watch for anomalous accounts, subscription-to-usage ratios and enterprise-scale query throughput [12]. Those are ratios without published cut points. The reporting on the advisory carries no threshold and no error rate [17]. A provider that picks a threshold is picking a false positive rate for its own paying customers, and picking it without knowing how much of its traffic is distillation in the first place.

The agencies' second recommendation covers targeted response changes, including subtly altering outputs for suspected distillation attempts so the results are worth less as training data [13]. The logic is clean: if the student is fitting the teacher's function, corrupting the labels degrades the student. What is missing is the dose response. There is no reported measure of how much perturbation is needed, how much student accuracy it removes, or what the same perturbation does to a legitimate customer flagged by mistake [17].

The claimed effect is that this activity has let Chinese models close some of the performance gap with U.S. systems while cutting research, development and compute costs [6]. No magnitude appears [18]. It doesn't tell you how much of any named model's measured performance came through the API and how much came from its own pretraining run. Separating those needs a counterfactual, and the agencies present an assessment instead: the activity was likely conducted with awareness from the Chinese government [5].

Six companies are named, DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun and Z.AI [3], and the advisory says they extracted proprietary capabilities from variants of Claude, GPT, Gemini and Grok since at least late 2024 [4]. That is four separate U.S. model families [16]. Requests were routed through native APIs, cloud platforms and third-party aggregators to obscure where they came from [7], so no single provider sees the whole pattern in its own logs. The third recommendation asks for that correlation across model providers, cloud platforms and API aggregators [14].

The part a provider controls least is the resale layer. The advisory describes a gray market of API proxies, which it calls "transfer stations," used to bypass geographic restrictions and bulk-procure premium subscriptions across teams of developers [8]. Distillation now sits at the center of several of these companies' development strategy [2]. Previous joint advisories from this trio covered Russian state-sponsored cyber threats, BlackMatter ransomware, and, with the Department of Energy, advanced persistent threat actors targeting industrial control and SCADA devices [15].

What to watch

  • Any response from the six named companies or from Beijing, and any fuller account of the evidence behind the government-awareness assessment.
  • A published experiment showing how much output perturbation degrades a student model, and what it costs users flagged by mistake.
  • Terms-of-service or contractual changes aimed at API aggregators and resellers. Such a change would show providers acting on the resale layer.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories