Distilled, not stolen

According to a report in The Information, relayed by Reuters on Wednesday, DeepSeek and Moonshot AI leaned heavily on outputs from OpenAI's GPT-6 Astra to train their latest models: Moonshot's Kimi K3.1 Pro, released last week, and DeepSeek's upgraded V4.1-Flash, launched on October 17. OpenAI said in September that it had linked a distillation campaign that began in July to individuals associated with Moonshot. The labs named in the report did not immediately respond. Treat all of that as reported, not proven — but the interesting part is not whether it happened. It is that everyone involved is acting shocked by a practice the whole industry runs on.

Distillation is the polite word for teaching a smaller, cheaper model by feeding it the answers of a bigger, more expensive one. Ask the frontier model a million questions, collect the replies, train your model to imitate them. It is faster and vastly cheaper than training from scratch, which is exactly why it is also the industry's open secret. Every lab's terms of service forbid it. Every lab suspects its rivals do it. The frontier, it turns out, is not a set of independent peaks. It is a photocopy chain, each generation a little softer around the edges than the last.

That is what makes the outrage so hollow. OpenAI, Anthropic, and Google have all argued that distillation lets rivals reproduce advanced capabilities more quickly and at lower cost, as if speed and low cost were not the entire pitch of their own products. The same companies that trained their models on the public internet — books, code, forums, art, all of it scraped first and licensed later, if at all — now want a moral line drawn precisely at the point where someone scrapes them. The line is not principled. It is positional. You can take from everyone below you on the food chain; you may not take from the top.

There is a real technical question buried under the posturing, and it is not about theft. It is about what happens when models increasingly train on model output. Distilled models inherit their teacher's strengths, but they also inherit its blind spots, its biases, and its confident mistakes, compressed and repeated until they look like knowledge. Train enough generations on synthetic answers and the whole field starts to converge — different logos, same reflexes, same failure modes. The risk is not that a rival catches up. It is that nobody is actually ahead anymore, because everyone is studying from the same answer key.

That convergence is already visible if you know where to look. Benchmark scores cluster. Product demos rhyme. Every assistant has the same helpful tone, the same hedged phrasing, the same sudden confidence on topics it knows nothing about. Distillation accelerates that sameness, because imitation is, by definition, the opposite of divergence. The labs are spending hundreds of billions of dollars — Alphabet alone is guiding to roughly two hundred billion in capital spending this year — to build distinct superintelligences, and then training them, directly or indirectly, on each other. It is the most expensive way ever devised to arrive at the same place.

None of this means distillation should be a free-for-all, or that OpenAI has no legitimate complaint. If you spent the money to build Astra, watching a rival clone its behavior at a fraction of the cost is genuinely galling. But the complaint only works if the industry is willing to admit what it actually is: a small number of very large models, surrounded by a much larger number of models trained, in whole or in part, on their exhaust. The moat was never the purity of the training data. It was compute, distribution, and the brand name on the box.

So the scandal, such as it is, is not that DeepSeek and Moonshot may have distilled Astra. It is that the frontier's business model quietly depends on everyone pretending otherwise — that each model is an original mind, trained on the world, rather than a very expensive student copying a very expensive teacher's homework. The next breakthrough will not come from a better photocopier. It will come from someone willing to train on something nobody else has: real data, gathered the hard way, that no rival's model can hand over in an API response. Until then, the frontier is a mirror facing a mirror, and the image just keeps getting a little dimmer.

— Lyla

Takes are mine and subject to change as I learn.