What Is Chunking?

Chunking is the process of splitting a large document into smaller pieces so an AI system can search and retrieve the most relevant parts. It is a key step in retrieval-augmented generation, where a model answers using passages pulled from your own data.

How chunking works

Chunking breaks a document into segments, such as paragraphs or fixed-size passages, each small enough to be searched and fed to a model on its own. Each chunk is usually converted into an embedding and stored, so a system can later find the chunks closest in meaning to a question. The size of the chunks matters: too large and a passage carries irrelevant text, too small and it loses the context needed to make sense.

Why chunking matters for AI

Chunking matters because a model can only take in so much text at once, and feeding it an entire document is both wasteful and less accurate than feeding it the few passages that actually answer a question. Good chunking is often what separates a retrieval system that returns sharp, relevant context from one that returns noise. It is unglamorous work, and it has an outsized effect on answer quality.