Inference

Fact

The run step where a trained model takes input and makes output.

In Plain Words

Training is teaching a barista the menu. Inference is your latte getting made after you order.

Every chat reply, image, or model call runs inference. It spends time and computer power.

Related Concepts

• Large Language Model
• Token
• GPU Compute
• Application Programming Interface

LLM
Inference is when an LLM uses its learned skills to answer a request.

Token
Inference reads input as Tokens and writes output step by step.

GPU
GPU power affects the speed and cost of inference.

API
An API wraps inference so apps can call it.