The frontier map — territory 4 of 5
Optimization & Game Theory
The engineer runs optimizers; the scientist knows when and why they converge — and what breaks when two networks optimize against each other. This territory spans convex analysis, convergence rates, minimax dynamics and optimal transport: the mathematics behind Adam, WGAN and PPO.
The central picture: three landscapes, three destinies
What the loss surface allows determines what theory can promise
The roadmap of understanding — three levels deep
Level 1 — Core objects
the shapes of solvabilityThe ideas
- Convexity: every local minimum is global — the "easy" world
- Rates: how fast error shrinks (1/t vs 1/t² vs e^−t)
- Saddle points and plateaus — the honest hard parts
Where it lives
- Classic ML's guarantees; deep learning's lack of them
What it unlocks
- Knowing which promises theory can and cannot make
Level 2 — Working theory
the machinery behind every optimizerThe ideas
- Smoothness L and strong convexity μ; condition number κ = L/μ
- Momentum & Nesterov acceleration: provably faster, not a hack
- Adaptive methods: Adam as diagonal preconditioning; why AdamW decouples weight decay
- Proximal operators & constraints; large-batch scaling (LARS/LAMB)
Where it lives
- Every optimizer paper's convergence section
- LR-scaling rules when batch size changes
- Second-order-flavored optimizers (Shampoo, Muon) at LLM scale
What it unlocks
- Designing optimizers and schedules from principles
- Diagnosing "slow training" as conditioning, not bad luck
Level 3 — The frontier
games, transport, and nested problemsThe ideas
- Minimax optimization: Nash equilibria, rotational dynamics, extragradient fixes
- Optimal transport: Wasserstein distance as the geometry between distributions
- Trust regions: constrain each step's damage (TRPO → PPO's clipping)
- Bilevel problems: optimize through an optimization (meta-learning, RLHF)
Where it lives
- The WGAN story in full: vanishing gradients weren't a bug in the network — the JS divergence gave no useful slope between disjoint distributions; Wasserstein distance restored the geometry of descent
- PPO — the optimizer of RLHF
- Multi-agent learning, self-play (AlphaZero-style)
What it unlocks
- Stabilizing any adversarial or RL training scheme
- Choosing the distance a model minimizes — often the real design decision
- Reading alignment methods as constrained optimization
Cheat sheet: concept → landmark → what it's doing there
| Concept | Landmark use | What it's doing there |
|---|---|---|
| Acceleration (momentum) | Nesterov; every training recipe | Provably faster descent on ill-conditioned bowls |
| Diagonal preconditioning | Adam / AdamW | A cheap approximation of curvature, per weight |
| Minimax dynamics | GAN training theory | Why simultaneous descent orbits; how to damp it |
| Optimal transport / Wasserstein | WGAN | A distance with usable gradients between distributions |
| Trust regions | TRPO → PPO (RLHF) | Bounding each policy update's harm |
| Bilevel optimization | Meta-learning, hyperparameter search | Learning through an inner learner |
The territory's signature move: when training misbehaves structurally, change the
problem, not the hyperparameters — a better distance (WGAN), a constrained step (PPO), an
averaged iterate. The deepest fixes in this field are reformulations.