An optimization method that uses a probability model to pick the next trial.
Bayesian Optimization is like finding the best donut in a huge box. You only get a few bites, so each bite points to the next one.
It is used for HPO and costly lab tests. It finds good results with fewer tries.
HPO
Bayesian Optimization helps HPO find good settings with fewer training runs.
GP
A GP often acts as the stand-in model for what to try next.
Exploration-Exploitation Tradeoff
Bayesian Optimization uses uncertainty to balance trying new spots and using good spots.
Optimization
Bayesian Optimization is an Optimization method for problems where each test costs a lot.