Vision Transformer

Fact

A vision model that cuts an image into patches and uses a Transformer.

In Plain Words

A Vision Transformer treats a photo like a tray of brownies. It cuts it into squares, then judges the whole pan.

It is used to recognize and sort images. It also often acts as the eyes in models that handle images and text.

Related Concepts

• Transformer architecture
• Self-Attention
• Convolutional Neural Network
• Contrastive Language-Image Pre-training

Transformer
A Vision Transformer brings the Transformer from text tasks into image tasks.

Self-Attention
Self-Attention lets it compare patches across the whole image at once.

CNN
A Vision Transformer leans global, while a CNN leans local.

CLIP
CLIP often uses it as the vision encoder for pictures.