Chao Ma
April 19, 2026
Notes on entropy, information, coding, and probabilistic structure, starting from the Oxford Math information theory lectures.
Oxford Information Theory Lecture 1: Defining Entropy and Information Surprise as negative log-probability, entropy in bits, KL divergence as mismatch cost, mutual information as dependence, and conditional entropy through the chain rule.
---
title: "Information Theory"
author: "Chao Ma"
date: "2026-04-19"
---
Notes on entropy, information, coding, and probabilistic structure, starting from the Oxford Math information theory lectures.
---
::: {.content-grid}
::: {.content-card}
**[Oxford Information Theory Lecture 1: Defining Entropy and Information](oxford-lecture-1-defining-entropy-information.qmd)**
Surprise as negative log-probability, entropy in bits, KL divergence as mismatch cost, mutual information as dependence, and conditional entropy through the chain rule.
:::
:::