What Is a Mixture of Experts (MoE)?

Mixture of experts (MoE) is a model architecture that divides a neural network into many specialized sub-networks, called experts, and activates only the few most relevant ones for each input. This lets a model hold a very large number of parameters while using only a fraction of them on any single request, which keeps it faster and cheaper to run.

How a mixture of experts works

An MoE model adds a routing layer that decides, for each input, which experts should handle it. Rather than pushing every token through the entire network, the router picks a small subset of experts best suited to that input and sends the work only to them. The rest stay idle for that request.

This is called sparse activation. A model might contain hundreds of billions of parameters spread across its experts, yet only activate a small share of them per token. The model gets the capacity of a huge network without paying the full compute cost on every call.

Why mixture of experts matters for AI

MoE matters because it breaks the tight link between how large a model is and how expensive it is to run. Teams can grow a model's total capacity, and the knowledge that comes with it, without a matching jump in the cost of each response. Many of the largest current language models use an MoE design for exactly this reason.

The trade-off is added complexity in training and serving, since the routing has to be balanced so no expert is overworked or wasted. For most users this is invisible, showing up only as a capable model that is cheaper to operate than its size would suggest.