Pick a starting point anywhere in the city. Four optimizers climb as high as they can, feeling only the slope under their feet. The loss is minus the elevation, so every hilltop is a local minimum and Mount Davidson (928 ft) is the global minimum. Most starting points never reach it.
Each patch of color is a basin of attraction: all the starting points from which that optimizer ends on the same hilltop. Darker violet means a higher summit, so a lower loss. Hover to compare; click to race from that spot.
Each one sees the same thing at every step: the slope of the ground where it stands, as a two-number gradient g. They differ only in what they do with it. η is the learning rate you can set in the panel.
The default learning rates are sensible picks, not tuned to make any optimizer win. Where each one ends up depends on η, so try the sliders before drawing conclusions.
SGD, Momentum and Muon only use the gradient as a whole arrow. Turn the map and the arrow turns with it, so their paths turn too and end on the same hill.
Adam, RMSprop, Adagrad and Lion treat east-west and north-south as separate numbers. Adam divides each one by its own recent size, so early on it moves about η along each axis. That makes diagonal steps √2 times longer than straight ones. Rotate the axes and the "diagonals" point somewhere else, so Adam takes a different route. From Hayes Valley, rotating 30° sends Adam to Alamo Square instead of Lafayette Park, and none of the other three move at all.
This matters in real training too. Adam's per-coordinate trick works best when the important directions in a problem line up with individual parameters. The demo below shows what happens when they don't.
Muon was built for the weight matrices inside neural networks. It takes the momentum matrix and replaces it with the nearest orthogonal matrix using a few Newton-Schulz iterations. That sets every singular value to about 1, so every direction in the update gets the same size.
On this map the position is just two numbers: a 1×2 matrix with one singular value. The whole procedure collapses to "follow the momentum and always step the same distance." That is the green climber. It never quite settles on a summit, because its step never shrinks unless you turn on the learning-rate decay.
To see what Muon is actually for, here is a small version of a job it does in a language model: storing facts. A 64×64 matrix has to map 64 keys to 64 values. Facts appear with very different frequencies, like words in text. The most common fact gets 42% of the weight, the rarest 0.08%. Each optimizer starts from zero and uses its own best learning rate.
A map has two dimensions. A language model has billions, and two things change at that scale.
True local minima get rare. At a real local minimum, the loss has to curve upward in every direction at once. With billions of directions that almost never happens. Training slows down mostly at saddle points and long, nearly flat valleys. The "stuck on the wrong hill" story on this page is real in two dimensions and much weaker in a big network.
The minima found tend to be about equally good. Different training runs of a large network usually reach very similar loss, a bit like how Twin Peaks and Mount Sutro are only a few feet lower than Mount Davidson. What differs more is how fast they get there and how the solutions behave on new data.
So read the basin maps as intuition for how each update rule moves, not as a prediction of where a real model ends up.