Principles

What a Token Actually Is

The model cannot see the word strawberry. It sees a handful of number-coded fragments — and that, not stupidity, is why it miscounts the r's.

A token is the actual unit a model reads and writes. Not a letter, not reliably a word, but a chunk. A tokenizer chops text into these pieces, usually sub-word: common words are a single token, rarer ones split — unbelievable becomes un + believ + able. And each token is really just an ID number in a fixed vocabulary of a few tens of thousands. Everything is tokens: your prompt becomes tokens going in; the reply is tokens coming out, one at a time.

Here is the whole engine, and it is smaller than people expect: the model predicts the next token. Given the tokens so far, it produces a probability across the entire vocabulary for what comes next, picks one, appends it, and repeats. That loop — next-token prediction — is all of it. Everything that looks like reasoning is that loop, run well.

Why should a reader care about a detail this deep in the plumbing? Because the model's most visible traits fall straight out of it:

  • The context window is a token budget. The model can only hold so many tokens at once; run past the limit and the earliest ones fall off the back — it doesn't forget, it never had them.
  • Cost is per token — you pay by the token, in and out. Verbosity has a price.
  • The famous stumbles finally make sense. Ask how many r's are in strawberry and it fumbles partly because it never saw the letters. It saw the tokens. It isn't counting r's; it's predicting the next token about r's.

Where it breaks — the honest half:

  • The model doesn't read the way you do. No letters, no words as you know them — token IDs and probabilities.
  • Tokenization is uneven across languages. English is cheap in tokens; many other languages cost more for the same sentence — a quiet unfairness baked into the meter.

One sentence: a token is the fragment the model actually reads, and everything it does — its memory limit, its price, its blind spots — is downstream of the fact that it thinks in tokens, not words.

Anchored links: → Layer III · FOUNDATION — embeddings (tokens become vectors) · → Layer IV · ORCHESTRATION — the context window.