Papers in Deep Learning

Author

Chao Ma

Published

February 8, 2026

Research paper reading notes focused on key ideas, math intuition, and practical takeaways.

Generative Adversarial Nets GAN training as a minimax game between discriminator and generator, with the optimal discriminator derivation and the global optimum condition \(p_g=p_{data}\).

Why TPU Is Fast for Dot Product (First-Gen TPU) Focus on the first-generation TPU inference design: 8-bit arithmetic, latency-first execution, and the systolic-array MMU that makes ASIC specialization much faster than general-purpose processors for matrix MAC workloads.

Transformer: Attention Is All You Need The Transformer removes recurrence and centers the model on scaled dot-product attention, enabling parallel training and strong long-range dependency modeling.

Attention: The Origin of Transformer Introduce learnable alignment scores and a dynamic context vector, replacing the fixed encoder bottleneck in seq2seq models.

Batch Normalization: Accelerating Deep Network Training Normalize mini-batch activations with learnable scale and shift to stabilize training, improve conditioning, and speed convergence.

LoRA: Low-Rank Adaptation of Large Language Models Freeze the base model and learn a low-rank update \(\Delta W=BA\) for selected layers, achieving strong performance with far fewer trainable parameters and easy deployment.

---
title: "Papers in Deep Learning"
author: "Chao Ma"
date: "2026-02-08"
---

Research paper reading notes focused on key ideas, math intuition, and practical takeaways.

---

::: {.content-grid}

::: {.content-card}
**[Generative Adversarial Nets](generative-adversarial-nets.qmd)**
GAN training as a minimax game between discriminator and generator, with the optimal discriminator derivation and the global optimum condition $p_g=p_{data}$.
:::

::: {.content-card}
**[Why TPU Is Fast for Dot Product (First-Gen TPU)](why-tpu-is-fast-for-dot-product.qmd)**
Focus on the first-generation TPU inference design: 8-bit arithmetic, latency-first execution, and the systolic-array MMU that makes ASIC specialization much faster than general-purpose processors for matrix MAC workloads.
:::

::: {.content-card}
**[Transformer: Attention Is All You Need](transformer-attention-is-all-you-need.qmd)**
The Transformer removes recurrence and centers the model on scaled dot-product attention, enabling parallel training and strong long-range dependency modeling.
:::

::: {.content-card}
**[Attention: The Origin of Transformer](attention-origin-transformer.qmd)**
Introduce learnable alignment scores and a dynamic context vector, replacing the fixed encoder bottleneck in seq2seq models.
:::

::: {.content-card}
**[Batch Normalization: Accelerating Deep Network Training](batch-normalization.qmd)**
Normalize mini-batch activations with learnable scale and shift to stabilize training, improve conditioning, and speed convergence.
:::

::: {.content-card}
**[LoRA: Low-Rank Adaptation of Large Language Models](lora.qmd)**
Freeze the base model and learn a low-rank update $\Delta W=BA$ for selected layers, achieving strong performance with far fewer trainable parameters and easy deployment.
:::

:::