A rule showing AI often improves as models, data, and compute grow.
Scaling law is AI’s gym-bro rule. Add more weights, snacks, and practice, and the bot usually gets stronger.
It is why labs buy huge computers and feed models more data. It works often, but a bigger bill does not always buy a wow moment.
Parameter
More Parameters often need larger Pretraining before they really help.
Pretraining
Scaling law pushes teams toward larger Pretraining runs and more data.
Compute-race
Scaling law fuels the chase for more compute, which becomes the Compute-race.
Emergence
As models keep growing, Emergence can bring new skills no one expected.