What Is a Token in AI?

A token is the basic unit of text a language model processes, often a whole word or a piece of one. Models do not read text as letters or full words but as sequences of tokens, and they generate output one token at a time. Token counts determine how much fits in a model's context and what a request costs.

How a token works

Before a model reads text, the text is split into tokens by a process called tokenization. A token is often a common word, while longer or rarer words break into several pieces, so "cat" might be one token and "unbelievable" a few. As a rough guide, a token averages around three-quarters of a word in English, though this varies by language and content.

The model works entirely in tokens: it takes a sequence of them as input and predicts the next token, repeatedly, to produce a response. Everything a model handles, the prompt, the context, and the output, is counted in tokens, which is why token counts show up throughout working with models.

Why tokens matter for AI

Tokens matter because they are the unit behind two practical limits. A model's context window is measured in tokens, so token count decides how much text can go in at once. Pricing for model APIs is usually per token as well, for both input and output, so token count drives cost. Understanding tokens helps explain why very long inputs get truncated or expensive, and why trimming prompts can save money. Knowing the token cost of a request is a routine part of designing systems that use models efficiently.