A way for a model to predict several tokens in one step.
Old-school AI drops words like a slow gum machine. MTP turns the knob and gets a small handful.
It helps AI answer faster during inference. You meet it inside large-model speed tricks.
Token
MTP predicts several tokens at once instead of one at a time.
Inference
MTP is mainly used during inference to make generation faster.
Inference engine
MTP often needs an inference engine to run well in real systems.
Continuous batching
Continuous batching handles the waiting line, and MTP makes each step bigger.