The 50% map — part 5 of 5
Numerics & Tensor Craft — where math meets silicon
All the beautiful math of parts 1–4 runs on approximate numbers with finite bits. This part explains dtypes, stability tricks and memory arithmetic — the layer where most real-world debugging happens, and where the architect's VRAM rules of thumb come from.
Where it lives: the bits of a number
Floating-point formats — what the bits buy
sign
exponent — range
mantissa — precision
The roadmap of understanding — three levels deep
Level 1 — Intuition
numbers are approximate, and bytes are moneyThe ideas
- Floats are rounded approximations — error accumulates
- Range (biggest number) and precision (finest step) trade off
- Bytes per parameter × parameters = memory, bandwidth, money
Where it lives
- 7B params × 2 bytes (FP16) ≈ 14 GB — the architect's rule, derived
- Model files: .safetensors sizes are exactly this arithmetic
What it explains
- Why quantized models are ¼ the size
- Why "just use FP64" is never the answer in ML
Level 2 — Working fluency
the daily stability & shape toolkitThe ideas
- Broadcasting: how a (D,) tensor applies across (B, S, D)
- Views vs copies; contiguous memory; einsum as explicit shape algebra
- Stability idioms: subtract-the-max softmax, log-sum-exp, ε in denominators
- Seeds & determinism — reproducing a run before debugging it
Where it lives
- Every softmax / cross-entropy implementation ever shipped
- Normalization layers (that little
eps=1e-5) - Framework code you read daily
What it explains
- Why naive softmax of a logit of 100 overflows to inf — and why the max-subtraction trick fixes it for free
- Why log-space computation exists at all
- Why two "identical" runs can differ
Level 3 — The deep end
the memory arithmetic of training & servingThe ideas
- Mixed precision: compute in BF16/FP16, keep master weights in FP32; FP16 needs loss scaling
- Quantization: INT8/INT4 with per-group scales (PTQ vs QAT)
- Adam training ≈ 16 bytes/param (weights + grads + two optimizer states) vs inference ≈ 2
- KV cache grows with batch × context × layers × heads
- Gradient accumulation: simulate big batches on small GPUs
Where it lives
- Every training-OOM investigation
- Serving LLMs: KV cache, not weights, eats the headroom at long contexts
- The entire quantized-model ecosystem (GGUF, AWQ, GPTQ)
What it explains
- Why training a 7B model needs ~112 GB-class memory while running it needs 14
- Why LoRA exists: freeze the 16-bytes/param problem, train a sliver
- Why long-context serving gets expensive even with small models
Cheat sheet: concept → where → purpose
| Concept | Where you meet it | What it's doing there |
|---|---|---|
| BF16 | Default training dtype on modern GPUs | FP32's range at half the memory — no loss scaling needed |
| Loss scaling | FP16 mixed-precision training | Keeping tiny gradients from underflowing to zero |
| Subtract-max / log-sum-exp | Softmax & cross-entropy implementations | Stopping overflow without changing the math |
| Broadcasting | Norm layers, bias adds, masking | Small tensors applied across big ones, implicitly |
| Optimizer-state arithmetic | Training memory budgets | ~16 bytes/param with Adam — the number behind every OOM |
| KV cache math | LLM serving | Memory that grows with context × batch, not model size |
| Quantization (INT8/INT4) | Deployment, local inference | 4× smaller and faster for a small quality tax |
The load-bearing insight: inference costs ~2 bytes per parameter; Adam training costs
~16. The entire modern toolkit — LoRA, quantization, mixed precision, gradient accumulation — is
engineering built on that one gap. Know the arithmetic and you can size any job from a napkin.
Full circle: this is the same arithmetic the architect's guide uses from the outside in
its operational metrics — VRAM per parameter, context
costs, quantization trade-offs. The engineer knows why those rules of thumb are true.