A field guide for engineers moving into ML
The ML Engineer's Math of AI
The ML Engineer needs ~50% of the math — and it's a specific 50%: not proofs and theorems, but working understanding. Every part of it maps to a concrete place in the model and the training loop. This guide is that map: what each part is, where it lives, what it explains, and how deep each one goes.
Where the line is
The 50% is defined by what it must let you do — not by chapters of a textbook.
Inside the 50%
Working fluency — math as a reading and debugging skill:
- Read the equations in a paper and translate them into tensor code
- Explain why a training run diverges, plateaus or NaNs — and what to change
- Reason about shapes, distributions and dtypes as fluently as about types in software
- Adapt an existing loss or schedule to your data
Beyond it — the research 100%
Producing new math stays on the other side of the line:
- Proving an optimizer converges, deriving generalization bounds
- Inventing new objectives (the WGAN move) rather than applying them
- Measure theory, stochastic calculus, learning theory
That territory has its own guide.
The map: one training step, five kinds of math
Every part of the 50% is the machinery of one stage of the loop you'll run every day.
Anatomy of a training step
The five parts — each with its own deep-dive page
Ordered as a roadmap of understanding: each part makes the next one readable.
1 · Linear Algebra
The language every model is written in. Vectors as meaning, matrices as transformations, matmul as 90% of all AI compute — and shapes as your daily debugging reality.
- Dot product = similarity (attention, RAG)
- Shape algebra of a transformer layer
- Low-rank: why LoRA works
2 · Calculus & Optimization
How models learn: gradients as sensitivity, backprop as the chain rule on a graph, and the optimizer as a hiker descending the loss landscape.
- Backprop demystified
- Adam, learning rates, schedules
- Why residuals & norms exist at all
3 · Probability & Statistics
Models are uncertainty machines: every output is a distribution, training is "make the data probable", and evaluation is statistics whether you notice or not.
- Softmax, temperature, top-p — decoded
- MLE: where cross-entropy comes from
- Is that +0.5% eval win real?
4 · Losses & Information Theory
The loss is the product spec: whatever it rewards, the model becomes. One page connects cross-entropy, perplexity, KL and the exotic objectives behind each model family.
- Why "confidently wrong" costs infinity
- Perplexity — the LLM report card
- Contrastive, minimax, diffusion, DPO
5 · Numerics & Tensor Craft
Where math meets silicon: floating-point formats, stability tricks, and the memory arithmetic that decides what fits on a GPU — the part of the 50% you'll use in every debugging session.
- FP32 / BF16 / FP16 / INT4 — what the bits buy
- Why naive softmax overflows
- Training ≈ 16 bytes/param, inference ≈ 2
Quick orientation: which part explains what
| Part | Core fluency | It's the machinery of… | It explains… |
|---|---|---|---|
| 1 · Linear algebra | Shapes, matmul, dot products, low-rank | Every layer, attention, embeddings | Shape errors, attention cost, why LoRA works |
| 2 · Calculus & optimization | Chain rule, gradient descent, Adam, schedules | Backward pass, optimizer step | Divergence, plateaus, why architectures look as they do |
| 3 · Probability & statistics | Distributions, MLE, sampling, eval statistics | Output layers, decoding, evaluation | Temperature, hallucination-as-miscalibration, noisy evals |
| 4 · Losses & information | Cross-entropy, KL, perplexity, shaped objectives | The goal of every training run | Why each model family behaves the way it does |
| 5 · Numerics & tensors | Dtypes, stability idioms, memory arithmetic | The silicon everything runs on | NaNs, OOMs, quantization, mixed precision |