DeepSeek Cheap, OpenAI Angry: How AI Gets Stolen via Distillation

There is a way to take the capability of an AI model without ever touching the model itself. That way has an official name, the industry uses it legitimately, and the rules around it are clear. It is also what an attacker used when OpenCode was breached in October 2026, except in a version where somebody else paid the bill.

The name is distillation. If you have access to a large model's outputs, you can generate thousands of questions and collect the answers. After that you train a smaller model to imitate them. The capability moves from the large model to the small one, and the original weights never move with it.

This article explains how the mechanism works, why the approach turned into a problem in the last two years, and where legitimate distillation ends and theft begins. It is an explanation that will not go stale, because the technique and the rules still hold while the industry inside it has changed several times over.

The mechanism: what actually gets copied

Distillation is not one technique. It is a family of techniques sharing one idea, which is moving information from a large model to a small one through examples rather than through weights.

Picture the simplest version. You have a large model and a small one. You give the large model a thousand questions and their answers. Then you hand those thousand pairs to the small model for training. The small model never sees the inside of the large one. What moves is behaviour, through the examples.

More subtle mechanisms apply to modern language models. Today's large models do not answer directly. They write out reasoning steps first, then assemble the answer from those steps. Those traces are far more informative than the final answer, because they show the path. Collecting those traces is what made distilling reasoning models feel like a leap in 2025.

One important thing is often missed. Distillation is not a copy. What actually moves is the distribution, not each answer. If the large model has a characteristic way of ordering answers when it is unsure, a small model trained on thousands of those examples adopts the same pattern. What is lost are the subtle parts that make the large model sharper. And those subtle parts are the most expensive to imitate.

From a 2015 experiment to the 2025 wave

The word distillation entered the published literature in 2015, through a paper by Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. The situation at the time was simple. Large models existed, but they were slow and expensive, so they could not run anywhere except on a big server.

The question they answered was how to move knowledge from a large model to a small one that could run on a laptop or a phone. The experiment worked. The idea they introduced, training a small model from a large model's outputs plus the original data, became the backbone of the entire modern distillation literature.

What changed in 2025 was the ordering and the scale. DeepSeek-V3 was released in late December 2024, and its claimed training cost of roughly 5.6 million US dollars went viral. That figure needs reading carefully. It covers one final stage, and it does not include the research and experiments that came before it. The direction is clear regardless, and much cheaper models started producing quality that was not much worse.

Then DeepSeek-R1 arrived on 20 January 2025, and that was the turning point. R1-Zero showed AIME 2024 jumping from 15.6 percent to 71.0 percent, matching OpenAI's o1, and 86.7 percent with majority voting. R1 itself reached 79.8 percent on AIME 2024 and 97.3 percent on MATH-500.

The most important part is not the big model's numbers but the six smaller distilled models that shipped alongside it. R1-Distill-Qwen-7B reached 55.5 percent on AIME 2024, beating the much larger QwQ-32B-Preview. R1-Distill-Qwen-32B reached 72.6 percent on AIME 2024, 94.3 percent on MATH-500, and 57.2 percent on LiveCodeBench.

These figures come from the DeepSeek-R1 paper. So they are the authors' own claims, not third-party results that have been independently verified. What anyone else can test are the distilled models, because the weights were released to the public.

Legitimate distillation and theft: a thinner line than you think

This is the part that is most often misunderstood, and the source of the confusion is almost always the same. Plenty of people assume distillation is always illegitimate.

It is not. Distillation is a recognised industry practice. The clearest evidence comes from DeepSeek itself. R1 shipped under the MIT licence, and its release notes explicitly state that its API outputs may be used for fine-tuning and distillation. So when that model is used and a chain of distillation is built on it, that is licensed.

What is not legitimate is distillation that uses someone else's capacity or bill. The legal line is reasonably clear. Using an API in line with its licence and terms of service is legitimate. Using an API in a way that breaks the terms, or with stolen credentials, is not. What happened at DeepSeek in January 2025, at OpenCode in October 2026, and in several other cases are variants of the same thing.

The first two rows are a healthy industry. A model that can be imitated cheaply is a model that can be sold into places that could not justify the larger one. The third and fourth rows are a problem, and both end in the same place, which is somebody's bill that they did not know about.

Analysis: why distillation became the weapon of choice

There is a technical reason and an economic one, and they point the same way.

Technically, distillation is the cheapest way to copy capability. Compare it with three alternatives. Running the large model is expensive per second. Training a model from scratch needs GPUs that not everyone has, plus data and weeks of time. Collecting a large number of large model outputs needs an API connection and a simple script. And because what is needed is output rather than internal access, a single API key is enough to unlock the whole thing.

The third and fourth rows cost the attacker almost the same, but the consequences differ. The third is paid for legitimately by whoever uses it. The fourth is paid by somebody else who does not know.

Economically, there is an accounting that makes this attractive. The cost ratio between running a frontier model and training one differs by orders of magnitude. Distillation turns the marginal cost of copying capability from a large training bill into a small linear API cost.

There is a third incentive that rarely gets mentioned. Distillation makes a model harder to trace. A distilled model has its own weights, its own architecture, and its own checkpoints. It is not a copy, and its technical trail differs from the original. A model like that is harder to tie back to its source.

The net effect is downward pressure. If a large model can be imitated cheaply, that large model becomes harder to justify economically for the tasks a small one can serve. You can see it in open-weight models whose size keeps shrinking while their capability keeps rising.

But there is one side that is often skipped. Distillation is not only a way to steal. It is also a way to move expensive capability voluntarily to places that lack the resources. OpenAI sells smaller models trained through distillation, and that is how they serve requests that a frontier model cannot justify. The same content can be a research tool or a cheating tool, depending on who uses it and with what permission.

What you can take from this

If you are not someone writing AI policy, there are three practical things worth taking away from understanding distillation.

First, you can tell apart two kinds of announcement that get lumped together. A distilled model released with an open licence is a contribution, and it is what made the open AI ecosystem grow. Admitting to distillation without permission is admitting to theft, and it is what makes this argument heated.

Second, if you use a paid API to build a product, distilling its outputs is most likely already covered by the terms of service you accepted. Do not skip reading them. And never publish raw outputs into a public dataset without checking first, because that is what most often causes trouble.

Third, and this is the most practical point. An API key is an asset, not a routine technical credential. Every time you attach one to a tool, you are putting money in someone else's house. The OpenCode incident in October 2026 is proof that this is not fearmongering. The company said the breach hit 1.4 percent of Console users, and that keys saved in workspaces may have been exposed. The arithmetic is simple. Whoever holds the key can make the charges. Hundreds of calls at once, and nobody notices until the invoice arrives.

What still cannot be answered

Several questions remain open, and it is better to be honest about them than to invent answers.

The most important one is that there is no public figure that measures how many models were trained this way. OpenAI stated in a memo to a US House committee in February 2026 that the majority of adversarial distillation activity they observe originates from China, and sometimes from Russia. That is one party's claim, it has not been publicly audited, and there is no independent way to measure it.

The memo also describes methods that are getting blunter. Access through third-party routers to hide the original source, and code written to pull outputs programmatically. What is known is the direction, not the scale.

Then there is a question with no answer yet. Are distilled models genuinely equivalent to the originals on the things that matter. For benchmarks and structured tasks, the numbers are nearly identical. For subtler things such as making up facts, reading context, or holding a position in the real world, there is no public measurement anyone can trust.

If there is one thing worth remembering from all of this, it is that distillation is not new. What is new is how cheap it became, and how much of the industry quietly runs on it.