Stanford CS336: Language Modeling from Scratch
Notes following Stanford CS336, Spring 2026, taught by Percy Liang and Tatsunori Hashimoto. The course connects model construction with systems, scaling, data, evaluation, and post-training.
Official course and materials · Official lecture recordings
Notes
A Tokenizer Changes the Work a Language Model Has to Do Lecture 1: course overview, lossless text representation, and BPE as a sequence-length and vocabulary tradeoff.
Course path
- Lectures 1–4: tokenization, resource accounting, architectures, and attention alternatives.
- Lectures 5–8: accelerators, kernels, and parallelism.
- Lectures 9–11: scaling laws and inference.
- Lectures 12–14: evaluation and data.
- Lectures 15–17: post-training, verifiable rewards, alignment, and multimodality.
The lecture notes above follow this path in order. Earlier material on LLM agents and deep learning systems provides complementary perspectives.