A slowdown in an AI system’s answer speed caused by slow tokenization.
It is a theme park with one tiny ticket scanner. The roller coaster is ready, but everyone waits at the gate.
This happens with many users or very long prompts. The first word comes late, and total speed drops.
Tokenizer
The tokenizer cuts text first, so a slow one blocks the entrance.
Token
Text becomes tokens before the model can keep going.
TPS
Slow tokenization can pull TPS down before inference starts.
Inference engine
Even a fast inference engine can wait at a slow input gate.