Build1 publisher3 min readPublished
Resold chat logs move distillation outside the origin lab's request telemetry
Bloomberg reports that US frontier labs and the US government are increasingly worried about distillation attacks, and that foreign actors are buying logs of conversations held on legitimate accounts. The detection Anthropic describes happens at request time.
The Engineer · Build desk
What happened
- Bloomberg reports that the US government and American AI developers are increasingly concerned about how well distillation attacks work against Western frontier models.
- China rejected the allegations and pledged countermeasures if America used them as a pretext to contain Chinese AI development.
- US and Chinese leaders are set to meet on September 24, with AI development and possibly distillation on the agenda.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Request-time controls reach only the route where the attacker holds the account. A log sold on after delivery was generated by traffic the lab already approved and billed.
- exposure Any intermediary holding customer transcripts becomes a supply point for someone else's training set. The risk sits with buyers of wrappers and resellers, outside the frontier lab's own perimeter.
- decision If the high-value payload is internal reasoning output, labs have to decide how much of it to expose per response, because that choice caps what a single successful request is worth to a distiller.
- contradiction The same reporting that carries the labs' request for government help also carries the South China Morning Post's claim that a US model used a Chinese model's output for early training data, so the line being drawn is about whose model is the teacher.
Distillation needs pairs: a prompt, and a stronger model's response to it, which a smaller model is then trained to emulate [6]. Whether anyone at the origin lab can see it happen depends on where the buyer gets those pairs.
Route one is self-service. The attacker holds an account, sends the prompts, keeps the responses. Anthropic's September 2026 report on countering malicious uses of AI describes distillation attempts it detected and countered over the previous year, and says these were often obvious because the prompts were clearly engineered to make Claude output its internal reasoning systems [8]. Rate limits, per-account scoring and terms enforcement all act at request time, so a request-shaped signal is one a lab can act on.
Route two is purchase. Bloomberg's reporting, as relayed by Tom's Hardware, says foreign actors have been buying logs of third-party conversations made using legitimate accounts, and that this makes the practice hard to halt entirely [5]. Every request in such a log was billed to a real customer doing plausible work. By the time the transcript is for sale it is a file on a third party's disk, and the origin lab's view of it ended at delivery [15].
The gap survives tighter API monitoring. To reach route two from inside the request stream, a lab would have to spot collection intent in traffic that is individually reasonable, or keep control of a response after it has been served.
The labs are framing this the way they framed AI investment, as a national security issue, and they want the US government to help prevent it [12]. US and Chinese leaders are set to meet on September 24, with AI development and distillation possibly on the agenda [13]. China has rejected the allegations and pledged to enact "countermeasures" if America used the pretext of those allegations to "contain" Chinese developments [3].
The dispute is over ownership. Distillation is treated as legitimate when a company trains a smaller model for internal use, or when an independent developer builds a lighter model for local or specific workloads, and as malicious when the teacher belongs to another firm [14]. The South China Morning Post claims Thinking Machines' Inkling model used other models, including Moonshot's Kimi K2.5, to generate early training data [9]. Tom's Hardware also argues that OpenAI and Anthropic trained on illicitly obtained material, including pirated books and scraped web articles [10].
DeepSeek in 2025 and Kimi K3 in 2026 are the results usually attributed to the technique, delivering similar levels of intelligence faster and far cheaper than the frontier models from Anthropic and OpenAI without quite matching them [7]. The reporting says foreign labs get comparable capability at a fraction of the cost and compute, but puts no figure on the fraction and no size on the log trade [2].
Western labs pledged earlier in 2026 to work together against distillation [4], and the reporting describes no rule. Tom's Hardware says detection can be easy depending on how an attack is conducted, while stopping one is hard [17].
What to watch
- Whether the September 24 leaders' meeting produces any stated US position on distillation, or any Chinese countermeasure.
- Whether any lab publishes a control aimed at transcript resale, such as contractual bans on log resale or withheld reasoning output.
- Whether Anthropic or its peers publish detection volumes, so the scale of the self-service route can be compared with the purchased-log route.