Product1 publisher3 min readPublished
CISA, NSA and FBI warn US AI firms of industrial-scale distillation by Chinese companies
CISA, the NSA and the FBI want US model providers to catch industrial-scale distillation hiding inside traffic that looks like a busy enterprise customer, then quietly degrade the answers, which makes false positives a product decision.
The Product Desk · Product desk
What happened
- CISA, with the NSA and the FBI, published an advisory telling American AI companies they are the targets of what it calls aggressive, malicious and targeted distillation activity at an industrial scale.
- It names DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun and Z.AI as having extracted billions of tokens across millions of requests from Claude, GPT, Gemini and Grok variants since at least late 2024.
- The recommended detection covers anomalous prompts and accounts, subscription-to-usage ratios, immediate maximum usage from new accounts, and enterprise-scale throughput patterns.
- The advisory then asks providers to subtly alter responses for suspected malicious distillation attempts, in order to reduce the payoff to the companies running the campaigns.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- cost The signals named are the profile of a legitimate power user, so the false positives are paid by the new enterprise account that hits its ceiling in week one and by the staff who have to review it.
- constraint Because the alleged traffic came through aggregators, proxies and shared subscriptions, a single provider's account-level signals describe the reseller rather than the buyer, which limits what any one vendor can act on alone.
- decision Providers now have to decide whether to ship deliberate answer degradation at all, and what they are willing to tell a paying customer who receives it.
- contradiction The same advisory calls distillation a legitimate and useful research technique while treating the named firms' use of it as extraction, leaving each provider to draw a policy line the agencies did not.
A data team stands up a new enterprise account on a Tuesday, points a backfill at it and saturates the rate limit inside an hour. That account has just tripped two of the signals the agencies tell providers to monitor: immediate maximum usage from a new account, and enterprise-scale throughput [6]. It has also done nothing except read the quickstart and mean it.
The per-request shape is where this gets genuinely hard. The advisory describes billions of tokens moved across millions of exchanges [2]. Divide the orders of magnitude and you get roughly a thousand tokens per exchange [12], an ordinary long answer to an ordinary long question. Distillation transfers a large teacher model's learnings into a smaller student model, in IBM's framing [10], and the transfer medium is answers, so the aggregate is the abuse while the individual call is just usage. The agencies also recommend hunting malicious prompts [6]; TechRadar's account of the advisory does not say what marks a prompt malicious.
Then the routing. The report lists seven distinct channels [13]: multiple accounts, different API access points, multiple cloud providers, third-party AI aggregators, proxy services, transfer stations, and premium subscriptions shared between developers [5]. Those are also the ordinary ways customers buy inference. When traffic arrives through an aggregator, the account-level signals describe the aggregator, and no single provider can see which of the six named firms [11] sits behind that key. Cross-organization intelligence sharing [8] is the only recommendation that touches the gap, and it asks providers to hand abuse signals to their competitors.
"Subtly alter responses for suspected malicious distillation attempts" [7] is the sentence that ends up in a support thread. No SLA has a clause for deliberately degraded output, and you cannot document the tripwire without shipping the evasion guide alongside it. A customer who spent two days chasing a quality regression you caused gets told nothing usable, and whoever is on support Friday cannot tell them either.
So draw two axes before anyone writes the rule. First, identity you trust: a named contract and a verified payment method, or nothing but traffic shape. Second, reversibility: holds for review, step-up verification and rate limits on one side, silently worse answers on the other. Irreversible degradation belongs only in the quadrant where you can name the account and defend the call in writing. Everywhere else the reversible controls are the honest option, and the tradeoff is real, because a rate limit tells a distiller roughly where your threshold sits. That is the price of not lying to a customer you have not yet identified.
The target list runs from GPT-4, Claude 3.7 and Gemini 2.5 Flash Preview up to GPT-5 and Claude Fable 5 [4], which makes this a standing workload across model generations rather than an incident to close out. The advisory is written for the provider's intellectual property and, in the agencies' words, systematic extraction threatening US technological leadership [14]. The bill goes to whoever staffs the abuse queue, and to the customers who get sampled into it by mistake.
What to watch
- Whether any provider whose models are named updates its acceptable-use terms to describe distillation detection or deliberate response alteration.
- Whether the cross-provider intelligence sharing the advisory asks for gets a named venue, and whether aggregators and resellers sit inside it.
- The first support thread in which a paying customer attributes degraded output to an abuse-detection rule.