How models work
What is a token in AI?
Also called
- tokens
Definition
A small chunk of text, often part of a word, that language models read and write, and that providers use to measure usage and price.
Key takeaways
- A token is a small chunk of text, often part of a word, that language models read and write.
- How many characters fit in a token depends on the model and the language.
- Token counts affect what you pay on usage-based plans and how much fits in the context window.
In plain English
Models do not see words the way people do: they break text into tokens. How many characters fit in a token depends on the model and the language, so the same text can count differently on different models. Token counts affect two things you will notice: how much you pay on usage-based plans and how much fits in the context window. Example: a long contract pasted into a chat uses far more tokens than a short question.
A small business example
Picture a bookkeeper on a usage-based plan who pastes whole contracts into a chat for every small question. Each long paste uses far more tokens than a short question, so the bill grows quietly. Pasting only the clause that matters keeps costs down and leaves more room in the context window.
Questions
Why do tokens matter on a flat monthly plan?
Even on a flat plan, tokens decide how much fits in the context window, the amount of text a model can consider at once. On usage-based plans they also decide what you pay.