The frontier map — territory 2 of 5

Probability Theory & Stochastic Processes

Randomness with a time axis. Where the engineer works with single distributions, the scientist works with distributions over trajectories — Markov chains, Langevin dynamics, stochastic differential equations. This is the territory that turned "add noise, learn to remove it" into the diffusion revolution.

The central picture: generation as a reversed process

Diffusion models are a forward SDE, learned backwards

that is what the neural network learns samplers (DDIM, DPM-Solver) are just numerical integrators of this equation — fewer steps = faster generation

One equation explains the whole diffusion stack: training (learn the score), sampling (integrate the reverse SDE), and why fast samplers are possible (better integrators, same equation).

The roadmap of understanding — three levels deep

Level 1 — Core objects

distributions that evolve

stochastic process · Markov property · stationarity

The ideas

A process = a distribution over whole trajectories
Markov property: the future depends only on the present
Stationary distributions: where a process settles
Measure-theoretic grounding: why continuous probability needs care

Where it lives

Diffusion chains, RL environments, token-by-token generation

What it unlocks

Seeing generation, exploration and sampling as one family of objects

Level 2 — Working theory

chains, samplers, and Brownian motion

Markov chains & MCMC · Langevin dynamics · Itô calculus · Fokker–Planck

The ideas

MCMC: sample from a distribution you can only score (Metropolis–Hastings, HMC)
Langevin dynamics: gradient descent + noise = a sampler
Brownian motion & Itô's lemma: calculus when paths are jagged
Fokker–Planck: the bridge from one particle's SDE to the whole density's evolution

Where it lives

Bayesian inference at scale; energy-based models
The derivation sections of every diffusion paper

What it unlocks

Reading "dx = f dt + g dW" without flinching
The beautiful identity: sampling = noisy optimization — Langevin is SGD's twin

Level 3 — The frontier

where the generative revolution was derived

score-based SDEs · probability-flow ODEs · stochastic approximation · MDPs

The ideas

Score matching: learn ∇ log p without knowing p
The reverse-time SDE (Anderson) → diffusion generation
Probability-flow ODE: the deterministic twin → DDIM, few-step samplers, flow matching
Stochastic approximation: the theory of why SGD converges at all
MDPs & stochastic control: the substrate of RL and RLHF

Where it lives

DDPM / score-SDE / flow-matching papers
Sampler design (DPM-Solver, consistency models)
RL theory: exploration, policy evaluation

What it unlocks

Designing new generative processes, not just using diffusion
Inventing faster samplers (a pure "better integrator" problem)
Reasoning about SGD's noise as a feature, not a bug

Cheat sheet: concept → landmark → what it's doing there

Concept	Landmark use	What it's doing there
Markov chains	Diffusion (DDPM's chain), autoregressive generation	Complex distributions built from simple steps
Langevin dynamics	Score-based generation, SGLD	Turning gradients into samplers
Itô calculus / SDEs	Score-SDE framework	The continuous language of noising & denoising
Reverse-time SDE	Diffusion sampling	Generation as time-reversal of corruption
Probability-flow ODE	DDIM, flow matching, consistency models	Deterministic, few-step generation
Stochastic approximation	SGD convergence theory	Guarantees for learning from noisy gradients
MDPs	RL, RLHF	Sequential decisions under uncertainty

The territory's signature move: when a distribution is too hard to model directly, build a process that flows into it — then learn to run the process. Diffusion, flow matching, MCMC and even RL are all this one move.