Notes/seed
Loss landscapes and terrain
Seed. The slope-reading intuition from backcountry skiing keeps wanting to be a statement about optimisation geometry, and I have not made it rigorous yet.
An honest seed: the analogy is suspicious and I want to either make it precise or discard it.
The skiing version of reading terrain is mostly about second-order information — not the slope angle in front of you but how it is changing, and where that puts you relative to a feature you cannot see. Convexity, basically, inferred from very sparse samples.
What I do not know is whether the resemblance to curvature in optimisation is load-bearing or merely a pun. Related: Attention is all you need, Terrain as a first filter.