MIT 18.06SC Lecture 11: Matrix Spaces, Rank-1, and Graphs

linear algebra

MIT 18.06

matrix spaces

rank

Exploring matrix spaces as vector spaces, rank-1 matrices, dimension formulas for subspaces, and applications to differential equations

Author

Chao Ma

Published

October 22, 2025

Overview

This lecture explores new vector spaces beyond $\mathbb{R}^n$:

Matrix spaces as vector spaces with their own subspaces
Rank-1 matrices and their fundamental structure
Dimension formulas for intersections and sums of subspaces
Applications to differential equations and constraints

Matrix Subspaces

Background: Vector Spaces and Subspaces

For a review of vector space fundamentals (what makes something a subspace, closure under addition and scalar multiplication), see Lecture 5.3: Vector Spaces. Matrix subspaces follow the same rules—they just happen to be spaces of matrices rather than vectors in $\mathbb{R}^n$.

Space of All 3×3 Matrices

Notation: $M$ = all 3×3 matrices

Dimension: $\dim(M) = 9$

Each entry is a free variable, so a 3×3 matrix space has dimension $3 \times 3 = 9$.

Important Matrix Subspaces

Symmetric Matrices: $S$

Definition: A matrix $A$ is symmetric if $A = A^T$

Properties:

Sum of two symmetric matrices is symmetric (closed under addition)
Scalar multiple of a symmetric matrix is symmetric (closed under scalar multiplication)
Therefore $S$ is a subspace

Dimension: $\dim(S) = 6$

Free variables (independent entries):

$(1,1)$ - first diagonal entry
$(2,2)$ - second diagonal entry
$(3,3)$ - third diagonal entry
$(1,2) = (2,1)$ - upper/lower symmetric pair
$(1,3) = (3,1)$ - upper/lower symmetric pair
$(2,3) = (3,2)$ - upper/lower symmetric pair

Example: \[ \begin{bmatrix} a & b & c \\ b & d & e \\ c & e & f \end{bmatrix} \] has 6 free parameters: $a, b, c, d, e, f$.

Upper Triangular Matrices: $U$

Definition: A matrix $A$ is upper triangular if all entries below the diagonal are zero

Dimension: $\dim(U) = 6$

Free variables:

3 diagonal entries: $(1,1), (2,2), (3,3)$
3 above-diagonal entries: $(1,2), (1,3), (2,3)$

Example: \[ \begin{bmatrix} a & b & c \\ 0 & d & e \\ 0 & 0 & f \end{bmatrix} \]

Diagonal Matrices: $D = S \cap U$

Definition: The intersection of symmetric and upper triangular matrices

Dimension: $\dim(D) = 3$

Free variables:

$(1,1)$
$(2,2)$
$(3,3)$

Example: \[ \begin{bmatrix} a & 0 & 0 \\ 0 & b & 0 \\ 0 & 0 & c \end{bmatrix} \]

Key Insight

A matrix that is both symmetric and upper triangular must be diagonal.

Dimension Formula for Subspaces

Sum of Subspaces: $S + U$

Definition: $S + U$ contains all matrices that can be written as the sum of a symmetric matrix and an upper triangular matrix.

For 3×3 matrices: \[ S + U = M \]

Every 3×3 matrix can be decomposed into symmetric and upper triangular parts.

Dimension Formula

General formula: \[ \dim(S) + \dim(U) = \dim(S \cap U) + \dim(S + U) \]

For our example: \[ 6 + 6 = 3 + 9 \]

Inclusion-Exclusion Analogy

This is analogous to the inclusion-exclusion principle:

The sum of dimensions counts the intersection twice
So we must subtract it once to get the dimension of the union/sum

Example: Differential Equations as Vector Spaces

Second-Order Homogeneous Differential Equation

Equation: \[ \frac{d^2y}{dx^2} + y = 0 \]

or equivalently: \[ y'' + y = 0 \]

Solutions: The solution space is spanned by: \[ y = c_1 \cos(x) + c_2 \sin(x) \]

Dimension: 2 (two free parameters: $c_1$ and $c_2$)

Key Principle

The set of all solutions to a linear homogeneous differential equation forms a vector space. The dimension equals the order of the differential equation.

Rank-1 Matrices

Structure of Rank-1 Matrices

Definition: A matrix has rank 1 if it can be written as the outer product of a column vector and a row vector.

General form: \[ A = u v^T \]

where:

$u$ is a column vector (in $\mathbb{R}^m$)
$v^T$ is a row vector (in $\mathbb{R}^n$)

Example

\[ A = \begin{bmatrix} 1 & 4 & 5 \\ 2 & 8 & 10 \end{bmatrix} = \begin{bmatrix} 1 \\ 2 \end{bmatrix} \begin{bmatrix} 1 & 4 & 5 \end{bmatrix} \]

Verification: \[ \begin{bmatrix} 1 \\ 2 \end{bmatrix} \begin{bmatrix} 1 & 4 & 5 \end{bmatrix} = \begin{bmatrix} 1 \cdot 1 & 1 \cdot 4 & 1 \cdot 5 \\ 2 \cdot 1 & 2 \cdot 4 & 2 \cdot 5 \end{bmatrix} = \begin{bmatrix} 1 & 4 & 5 \\ 2 & 8 & 10 \end{bmatrix} \]

Key observation: All rows are multiples of each other (row 2 = 2 × row 1).

Rank-1 Matrices Do Not Form a Subspace

Claim: The set of all rank-1 matrices is not a subspace.

Reason: Rank-1 matrices are not closed under addition.

Counterexample: \[ A_1 = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}, \quad A_2 = \begin{bmatrix} 0 & 0 \\ 0 & 1 \end{bmatrix} \]

Both have rank 1, but: \[ A_1 + A_2 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \]

has rank 2.

General Principle

If $A$ has rank $n$ and $B$ has rank $m$, then $A + B$ can have rank anywhere from $|n - m|$ to $n + m$.

Null Space Example

Problem Setup

Given: $S$ is the set of all vectors $v$ in $\mathbb{R}^4$ such that: \[ v_1 + v_2 + v_3 + v_4 = 0 \]

Question: What is the dimension of $S$?

Solution Using Null Space

Define the matrix: \[ A = \begin{bmatrix} 1 & 1 & 1 & 1 \end{bmatrix} \]

Then $S$ is the null space of $A$: \[ S = N(A) = \left\{ v = \begin{bmatrix} v_1 \\ v_2 \\ v_3 \\ v_4 \end{bmatrix} : Av = 0 \right\} \]

Dimension calculation:

$\text{rank}(A) = 1$ (one independent row)
$n = 4$ (number of columns)
By the rank-nullity theorem: \[ \dim(N(A)) = n - r = 4 - 1 = 3 \]

Interpretation: Three degrees of freedom. If we choose any values for $v_1, v_2, v_3$, then $v_4$ is determined by $v_4 = -(v_1 + v_2 + v_3)$.

Summary

Key concepts:

Matrix spaces are vector spaces with dimension equal to the number of independent entries
Symmetric matrices (3×3): dimension 6
Upper triangular matrices (3×3): dimension 6
Diagonal matrices (3×3): dimension 3
Dimension formula: $\dim(S) + \dim(U) = \dim(S \cap U) + \dim(S + U)$
Rank-1 matrices: $A = uv^T$ (outer product structure)
Rank-1 matrices are not a subspace: sum of two rank-1 matrices can have rank 2

Source: MIT 18.06SC Linear Algebra, Lecture 11

--- title: "MIT 18.06SC Lecture 11: Matrix Spaces, Rank-1, and Graphs" author: "Chao Ma" date: "2025-10-22" categories: [linear algebra, MIT 18.06, matrix spaces, rank] description: "Exploring matrix spaces as vector spaces, rank-1 matrices, dimension formulas for subspaces, and applications to differential equations" --- {{< video https://www.youtube.com/watch?v=2IdtqGM6KWU >}} ## Overview This lecture explores new vector spaces beyond $\mathbb{R}^n$: 1. **Matrix spaces** as vector spaces with their own subspaces 2. **Rank-1 matrices** and their fundamental structure 3. **Dimension formulas** for intersections and sums of subspaces 4. **Applications** to differential equations and constraints ## Matrix Subspaces ::: {.callout-note} ## Background: Vector Spaces and Subspaces For a review of vector space fundamentals (what makes something a subspace, closure under addition and scalar multiplication), see [Lecture 5.3: Vector Spaces](mit1806-lecture5-3-spaces.qmd). Matrix subspaces follow the same rules—they just happen to be spaces of matrices rather than vectors in $\mathbb{R}^n$. ::: ### Space of All 3×3 Matrices **Notation**: $M$ = all 3×3 matrices **Dimension**: $\dim(M) = 9$ Each entry is a free variable, so a 3×3 matrix space has dimension $3 \times 3 = 9$. ## Important Matrix Subspaces ### Symmetric Matrices: $S$ **Definition**: A matrix $A$ is symmetric if $A = A^T$ **Properties**: - Sum of two symmetric matrices is symmetric (closed under addition) - Scalar multiple of a symmetric matrix is symmetric (closed under scalar multiplication) - Therefore $S$ is a **subspace** **Dimension**: $\dim(S) = 6$ **Free variables** (independent entries): 1. $(1,1)$ - first diagonal entry 2. $(2,2)$ - second diagonal entry 3. $(3,3)$ - third diagonal entry 4. $(1,2) = (2,1)$ - upper/lower symmetric pair 5. $(1,3) = (3,1)$ - upper/lower symmetric pair 6. $(2,3) = (3,2)$ - upper/lower symmetric pair **Example**: $$ \begin{bmatrix} a & b & c \\ b & d & e \\ c & e & f \end{bmatrix} $$ has 6 free parameters: $a, b, c, d, e, f$. ### Upper Triangular Matrices: $U$ **Definition**: A matrix $A$ is upper triangular if all entries below the diagonal are zero **Dimension**: $\dim(U) = 6$ **Free variables**: - 3 diagonal entries: $(1,1), (2,2), (3,3)$ - 3 above-diagonal entries: $(1,2), (1,3), (2,3)$ **Example**: $$ \begin{bmatrix} a & b & c \\ 0 & d & e \\ 0 & 0 & f \end{bmatrix} $$ ### Diagonal Matrices: $D = S \cap U$ **Definition**: The intersection of symmetric and upper triangular matrices **Dimension**: $\dim(D) = 3$ **Free variables**: 1. $(1,1)$ 2. $(2,2)$ 3. $(3,3)$ **Example**: $$ \begin{bmatrix} a & 0 & 0 \\ 0 & b & 0 \\ 0 & 0 & c \end{bmatrix} $$ ::: {.callout-tip} ## Key Insight A matrix that is both symmetric and upper triangular must be diagonal. ::: ## Dimension Formula for Subspaces ### Sum of Subspaces: $S + U$ **Definition**: $S + U$ contains all matrices that can be written as the sum of a symmetric matrix and an upper triangular matrix. For 3×3 matrices: $$ S + U = M $$ Every 3×3 matrix can be decomposed into symmetric and upper triangular parts. ### Dimension Formula **General formula**: $$ \dim(S) + \dim(U) = \dim(S \cap U) + \dim(S + U) $$ **For our example**: $$ 6 + 6 = 3 + 9 $$ ::: {.callout-note} ## Inclusion-Exclusion Analogy This is analogous to the inclusion-exclusion principle: - The sum of dimensions counts the intersection twice - So we must subtract it once to get the dimension of the union/sum ::: ## Example: Differential Equations as Vector Spaces ### Second-Order Homogeneous Differential Equation **Equation**: $$ \frac{d^2y}{dx^2} + y = 0 $$ or equivalently: $$ y'' + y = 0 $$ **Solutions**: The solution space is spanned by: $$ y = c_1 \cos(x) + c_2 \sin(x) $$ **Dimension**: 2 (two free parameters: $c_1$ and $c_2$) ::: {.callout-important} ## Key Principle The set of all solutions to a linear homogeneous differential equation forms a **vector space**. The dimension equals the order of the differential equation. ::: ## Rank-1 Matrices ### Structure of Rank-1 Matrices **Definition**: A matrix has rank 1 if it can be written as the outer product of a column vector and a row vector. **General form**: $$ A = u v^T $$ where: - $u$ is a column vector (in $\mathbb{R}^m$) - $v^T$ is a row vector (in $\mathbb{R}^n$) ### Example $$ A = \begin{bmatrix} 1 & 4 & 5 \\ 2 & 8 & 10 \end{bmatrix} = \begin{bmatrix} 1 \\ 2 \end{bmatrix} \begin{bmatrix} 1 & 4 & 5 \end{bmatrix} $$ **Verification**: $$ \begin{bmatrix} 1 \\ 2 \end{bmatrix} \begin{bmatrix} 1 & 4 & 5 \end{bmatrix} = \begin{bmatrix} 1 \cdot 1 & 1 \cdot 4 & 1 \cdot 5 \\ 2 \cdot 1 & 2 \cdot 4 & 2 \cdot 5 \end{bmatrix} = \begin{bmatrix} 1 & 4 & 5 \\ 2 & 8 & 10 \end{bmatrix} $$ **Key observation**: All rows are multiples of each other (row 2 = 2 × row 1). ### Rank-1 Matrices Do Not Form a Subspace **Claim**: The set of all rank-1 matrices is **not a subspace**. **Reason**: Rank-1 matrices are not closed under addition. **Counterexample**: $$ A_1 = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}, \quad A_2 = \begin{bmatrix} 0 & 0 \\ 0 & 1 \end{bmatrix} $$ Both have rank 1, but: $$ A_1 + A_2 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} $$ has rank 2. ::: {.callout-warning} ## General Principle If $A$ has rank $n$ and $B$ has rank $m$, then $A + B$ can have rank anywhere from $|n - m|$ to $n + m$. ::: ## Null Space Example ### Problem Setup **Given**: $S$ is the set of all vectors $v$ in $\mathbb{R}^4$ such that: $$ v_1 + v_2 + v_3 + v_4 = 0 $$ **Question**: What is the dimension of $S$? ### Solution Using Null Space Define the matrix: $$ A = \begin{bmatrix} 1 & 1 & 1 & 1 \end{bmatrix} $$ Then $S$ is the null space of $A$: $$ S = N(A) = \left\{ v = \begin{bmatrix} v_1 \\ v_2 \\ v_3 \\ v_4 \end{bmatrix} : Av = 0 \right\} $$ **Dimension calculation**: - $\text{rank}(A) = 1$ (one independent row) - $n = 4$ (number of columns) - By the rank-nullity theorem: $$ \dim(N(A)) = n - r = 4 - 1 = 3 $$ **Interpretation**: Three degrees of freedom. If we choose any values for $v_1, v_2, v_3$, then $v_4$ is determined by $v_4 = -(v_1 + v_2 + v_3)$. ## Summary **Key concepts**: 1. **Matrix spaces** are vector spaces with dimension equal to the number of independent entries 2. **Symmetric matrices** (3×3): dimension 6 3. **Upper triangular matrices** (3×3): dimension 6 4. **Diagonal matrices** (3×3): dimension 3 5. **Dimension formula**: $\dim(S) + \dim(U) = \dim(S \cap U) + \dim(S + U)$ 6. **Rank-1 matrices**: $A = uv^T$ (outer product structure) 7. **Rank-1 matrices are not a subspace**: sum of two rank-1 matrices can have rank 2 --- *Source: MIT 18.06SC Linear Algebra, Lecture 11*