Holding the LLM Stack in Your Head

The SeriesA dependency-ordered walk through the modern LLM stack, from the linear algebra under a single attention head, through training and inference, out to agent protocols shipping in 2026. Ten arcs, eighty-odd posts. The goal isn't rigor, it's intuition that survives contact with real systems.The ten arcs01Vectors, Matrices, and the Spaces They Live InVectors as lists of activations, matrix multiplication as a linear map, and why every neural network operation bottoms out in matmuls.02Norms, Dot Products, and SimilarityHow cosine similarity, L2 distance, and projections work, and why they show up everywhere from attention scores to embedding retrieval.03Distributions, Softmax, and the Chain Rule of WordsSoftmax, categorical distributions, Bayes' rule, and the chain rule of probability — the four tools that make language modeling a well-defined math problem.04Cross-Entropy, KL Divergence, and What Loss Functions MeasureWhy cross-entropy is the standard LM loss, what it actually measures about two distributions, and how it connects to perplexity.05Gradients and How Machines LearnWhat a gradient is, why it points uphill, how backpropagation computes one efficiently via the chain rule, and what SGD does with it.06Optimizers: Momentum, Adam, and Learning Rate SchedulesWhy vanilla SGD is too slow, how Adam adapts per-parameter, and how warmup and cosine decay shape training dynamics.07GPUs, Floating Point, and Why Precision MattersIEEE 754, the difference between fp32/fp16/bfloat16, why mixed-precision training works, and the basics of GPU parallelism.08A Short Prehistory of Statistical NLPThe arc from rule-based systems through statistical MT and log-linear models to neural approaches, giving you historical context for everything that follows.Don't know where to start?The series is a full first draft. I'll be grinding through polish, corrections, and the odd rewrite. If you spot something wrong, contact info is here.