A token is the small text unit AI reads and writes.
AI does not swallow a sentence whole. It eats text in little nuggets called tokens.
Tokens set how much text a model can read at once. More tokens also mean a bigger API bill.
LLM
A token is the basic text unit an LLM uses for language.
Context-window
A Context-window can hold only a set number of tokens.
Inference
Inference costs more when input and output tokens increase.