Can six Chinese firms really just distill their way to a frontier model?

The NSA, CISA, and FBI just claimed that DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI have been running "industrial-scale" distillation attacks on US models like Claude, GPT, Gemini, and Grok since late 2024. The logic is simple: if you use a massive teacher model to generate high-quality synthetic data for a smaller student model, you slash your training costs and time. But calling it an "attack" feels like a stretch unless they're bypassing API rate limits or ignoring Terms of Service (ToS) on a massive scale.

How distillation actually works in practice

Distillation isn't some secret weapon; it's standard ML. You take a frontier model (the teacher), feed it a million prompts, and use its outputs to train a smaller model (the student). If these six companies are doing this at scale, they aren't just "copying"—they're optimizing.

The real technical question is how they're doing it without getting their API keys revoked. To distill a model that competes with GPT-4o, you need billions of tokens of high-quality synthetic data. If you try to pull that through a standard OpenAI or Anthropic API, you'll hit rate limits or trigger "unusual activity" flags if your patterns are too robotic. I suspect they're using massive proxy farms to rotate IP addresses and accounts to avoid detection.

Why the "industrial scale" claim is debatable

The US agencies argue this saves "billions" in development. While synthetic data reduces the need for human-labeled data, it doesn't replace the need for massive compute clusters. You still need thousands of H100s (or whatever substitutes they're using given the export bans) to actually train the student model on that distilled data.

If you're using a teacher model to generate a dataset, you run into the "model collapse" problem. If the student only learns from the teacher's synthetic output and never sees real-world, organic data, it starts to hallucinate in a very specific, repetitive way. To avoid this, you need a mix of synthetic and organic data. The claim that they're just "copying" ignores the immense engineering effort required to filter that synthetic data so the student doesn't just become a shallow mirror of the teacher.

The cost of "shortcut" development

Let's look at the numbers. Training a frontier model from scratch can cost upwards of $100 million in compute alone. If these firms are using distillation, they might reduce the data collection phase by 60-80%, but the actual training run still costs tens of millions.

The risk here isn't just "copying"; it's the quality ceiling. A distilled model almost never surpasses its teacher. If DeepSeek or Moonshot are relying solely on this, they'll hit a performance plateau. You can't "distill" a breakthrough in reasoning; you can only distill the appearance of reasoning.

Is this actually an "attack"?

Most of these companies are likely just using the APIs as intended, perhaps ignoring the "do not use our output to train competing models" clause in the ToS. In the AI world, that's not a cyberattack; it's a business strategy. Unless the NSA has evidence of actual system breaches (SQL injections, unauthorized server access), calling it an "attack" sounds more like geopolitical positioning than a technical diagnosis.

If I were building a model today, I'd distill from every single available frontier model to create a diverse synthetic set. It's the only way to catch up quickly. The real win isn't the copying—it's the cleaning and curation of that data.