Making a model smaller and lighter while trying to keep its skill.
Model compression is like repacking a huge camping backpack. Keep the snacks. Leave the cast-iron pan.
You meet it when AI must run on a laptop or phone. It uses less VRAM and often runs faster.
Quantization
Quantization is one of the most common ways to compress a model.
Distillation
Distillation helps move a big model’s skill into a smaller model.
Local-LLM
Model compression helps a Local-LLM run on a normal device.
VRAM
Smaller models usually use less VRAM while running.