Parameter Server

Fact

A setup that keeps model parameters in one place during distributed training.

In Plain Words

Parameter Server is the lunch lady with the only real recipe card. The kitchen crew sends tweaks, and she hands back the new card.

In big AI training, many machines work at once. They send weight updates to the server, then get the latest weights back.

Related Concepts

• Parameter
• Data Parallelism
• Stochastic Gradient Descent
• Distributed Computing

Parameter
A Parameter Server keeps Parameters in one central place.

Data Parallelism
Data Parallelism lets many machines work at once, then send updates back.

SGD
A Parameter Server often coordinates SGD weight updates.

Distributed Computing
A Parameter Server is a classic setup in Distributed Computing.