A field guide for those heading to the frontier
The Research Scientist's Math of AI
At research level, the math isn't under the hood — it is the product. New architectures, losses and training regimes are mathematical objects first and code second. This guide maps the five mathematical territories the 100% is made of: what each field is, which AI breakthroughs it produced, and what mastering it lets you do.
Where the line is
The difference between the 50% and the 100% is not more chapters — it's a change of role.
What 100% means
Math as the instrument of invention:
- Invent objectives and prove they behave (convergence, stability)
- Read a paper's theory section as the main course, not the garnish
- Move between fields fluently — a diffusion paper is stochastic calculus + score matching + numerics in one
- Recognize which mathematical tool a research problem is really asking for
Built on the other tracks
The scientist track presumes fluency in everything below it:
- The engineer's 50% — shapes, backprop, distributions, losses, numerics — is the working baseline here
- The architect's black-box view is still the map of what's worth inventing
Each territory page goes deeper in three levels — core objects, working theory, and the frontier where it powers current research.
The map: five territories, and the breakthroughs they produced
Every landmark result in AI is a flag planted in one of these fields.
Fields ↔ breakthroughs
The five territories — each with its own deep-dive page
1 · Analysis & Matrix Calculus
Derivatives grown up: Jacobians, Hessians, curvature and smoothness — the field in which backprop is a theorem and training dynamics become equations you can solve.
- Backprop as products of Jacobians
- Curvature, conditioning, sharp vs flat minima
- NTK: infinitely wide networks, solved
2 · Probability Theory & Stochastic Processes
Randomness with a time axis: Markov chains, Langevin dynamics and SDEs — the mathematics that turned "add noise, then learn to remove it" into the diffusion revolution.
- Forward / reverse SDEs — diffusion, derived
- MCMC & Langevin sampling
- SGD as a stochastic process
3 · Statistical Learning Theory
The deepest question in the field: why does minimizing training loss work on data the model has never seen? Classical bounds, and the modern puzzles that broke them.
- Capacity, VC dimension, uniform convergence
- Double descent & benign overfitting
- Neural scaling laws & compute-optimal training
4 · Optimization & Game Theory
What convergence even means: rates, conditioning, non-convex landscapes — and what happens when two networks optimize against each other.
- Convexity, rates, acceleration, Adam's theory
- Minimax dynamics: why GAN training orbits
- Optimal transport — the WGAN story in full
5 · Information Theory & the Geometry of Learning
Bits as the currency of learning: entropy, KL and mutual information — the field where compression, generation and alignment turn out to be the same subject.
- KL asymmetry: mode-covering vs mode-seeking
- ELBO: the variational workhorse (VAE, diffusion)
- InfoNCE, information bottleneck, DPO
Quick orientation: territory → landmarks → what mastery buys
| Territory | Landmark results | What mastering it lets you do |
|---|---|---|
| Analysis & matrix calculus | Backprop, NTK, SAM | Derive gradients for anything; analyze training dynamics |
| Stochastic processes | DDPM, score-based SDEs, DPM-Solver | Design generative processes and samplers |
| Statistical learning theory | Scaling laws, Chinchilla, double descent | Predict what more data/params/compute will buy |
| Optimization & game theory | Adam, Nesterov, WGAN, PPO | Invent optimizers; stabilize adversarial & RL training |
| Information theory | VAE/ELBO, InfoNCE, DPO | Derive objectives instead of guessing them |