A speed score for how fast an AI writes during inference.
TPS is the AI’s words-per-minute score. A fast bot sounds like an auctioneer. A slow one types like your uncle with one finger.
It shapes chat wait time and autocomplete smoothness. People use it to compare inference services.
Token
TPS counts tokens made each second.
Inference
TPS measures how fast the model generates during inference.
Continuous batching
Continuous batching cuts idle time, so TPS often goes up.
GPU
GPU power and memory speed can limit TPS.