What Is Distillation?

Distillation is a technique for compressing a large AI model into a smaller one by training the smaller "student" model to copy the outputs of the larger "teacher" model. The result is a lighter model that keeps much of the original's ability while running faster and more cheaply on less hardware.

How distillation works

Distillation works by having a small student model learn from a large teacher model rather than from raw data alone. The teacher runs over a set of examples and produces its outputs, and the student trains to reproduce them. Because the teacher's outputs carry more information than a plain right-or-wrong label, including how confident it was across the options, the student can learn more efficiently than it would from the original data by itself.

The payoff is size. A distilled model can be a fraction of the teacher's size while retaining much of its performance, which makes it practical to run where a giant model could not go, such as on a phone or under tight cost limits.

Why distillation matters for AI

Distillation helps close the gap between what large models can do and what most systems can afford to run. State-of-the-art models are powerful but expensive and slow, and distillation produces smaller versions that are cheaper to serve and quick enough for real-time use, with only a modest drop in quality. That trade-off is often worth it in production. At Custom AI Studio, techniques like distillation help us size a model to a client's actual workload, rather than paying for more model than the job needs.