A fast network that links AI chips in one large computer cluster.
It is the hallway between busy classrooms. If it is slow, thousands of GPUs stand around waiting.
It moves data between chips during training and use. Fast links help the chips work as one team.
GPU
Fast network links let GPUs share data during training and use.
Distributed Computing
Distributed Computing uses the network to send jobs and collect results.
Model parallelism
Model parallelism needs the network to move data between GPUs often.