Deep Connections: Invertibility, Null Space, Independence, Rank, and Pivots
Introduction
In linear algebra, concepts like invertibility, null space, linear independence, rank, and pivots often seem like separate topics. However, they are deeply interconnected - different views of the same mathematical reality. This post synthesizes these fundamental concepts and reveals how they all tell the same story about whether a linear transformation preserves information.
Invertibility: The Foundation
Reference: Lecture 3: Matrix Multiplication and Inverse
Definition
A square matrix \(A\) is invertible if there exists a matrix \(A^{-1}\) such that:
\[ AA^{-1} = A^{-1}A = I_n \]
Properties of Invertible Matrices
An invertible matrix \(A\) must satisfy:
- Square: \(m = n\) (same number of rows and columns)
- Full rank: \(\text{rank}(A) = n\)
- All pivot variables: \(r = n\) pivot positions
- No free variables: \(n - r = 0\) free variables
- Independent rows: All rows are linearly independent
Solving Systems with Invertible Matrices
When \(A\) is invertible, solving \(Ax = b\) becomes straightforward:
\[ \begin{aligned} Ax &= b \\ A^{-1}Ax &= A^{-1}b \\ x &= A^{-1}b \end{aligned} \]
The solution is unique and exists for every \(b\).
Null Space: A Direct Test for Invertibility
Reference: Lecture 7: Solving Ax=0
The Connection
The null space \(N(A)\) provides a direct test for invertibility:
- If \(N(A) = \{\mathbf{0}\}\): Matrix is invertible
- If \(N(A) \neq \{\mathbf{0}\}\): Matrix is NOT invertible
Why Non-Trivial Null Space Prevents Invertibility
Proof by contradiction:
Suppose \(N(A) \neq \{\mathbf{0}\}\) and \(A^{-1}\) exists. Let \(v_1 \in N(A)\) with \(v_1 \neq \mathbf{0}\).
Then:
\[ \begin{aligned} Av_1 &= \mathbf{0} \quad \text{(by definition of null space)} \\ A^{-1}(Av_1) &= A^{-1}\mathbf{0} \quad \text{(multiply both sides by } A^{-1}\text{)} \\ (A^{-1}A)v_1 &= \mathbf{0} \\ I_n v_1 &= \mathbf{0} \\ v_1 &= \mathbf{0} \end{aligned} \]
Contradiction! We assumed \(v_1 \neq \mathbf{0}\), but our logic forces \(v_1 = \mathbf{0}\).
Therefore, \(A^{-1}\) cannot exist when \(N(A)\) contains non-zero vectors.
A non-trivial null space means information is lost in the transformation \(A\), making it impossible to uniquely reverse.
Linear Independence: The Geometric View
Reference: Lecture 8: Solving Ax=b
Connection to Invertibility
If the rows (or columns) of a matrix are not all linearly independent, the matrix cannot be invertible.
For a square matrix:
- Independent rows/columns ⟺ \(\text{rank}(A) = n\)
- Dependent rows/columns ⟺ \(\text{rank}(A) < n\)
Why Dependence Prevents Invertibility
When rows are dependent:
- Rank: \(r < n\) (not full rank)
- Pivots: Only \(r < n\) pivot positions
- Free variables: \(n - r > 0\) free variables exist
Two Perspectives on Dependence
1. Algebraic Perspective
When free variables exist:
\[ R_{\text{rref}} = [I_r \mid F] \]
where \(F\) is the \((n-r)\)-dimensional free variable matrix.
The null space \(N(A)\) has dimension \(n - r > 0\), containing infinitely many vectors. By our earlier proof, this means \(A\) is not invertible.
2. Geometric Perspective
If rows are dependent, the transformation \(A\) collapses the \(n\)-dimensional space down to an \(r\)-dimensional subspace (where \(r < n\)).
- Information loss: The transformation maps multiple distinct inputs to the same output
- Cannot invert: We cannot uniquely recover the original \(n\)-dimensional vector from its \(r\)-dimensional image
- Missing dimensions: The \((n-r)\) dimensions of information are permanently lost
A \(3 \times 3\) matrix with rank 2 maps all of \(\mathbb{R}^3\) onto a 2-dimensional plane. Infinitely many points in 3D space map to each point on the plane. There’s no way to uniquely invert this mapping.
The Big Picture: Everything is Connected
All these concepts are different views of the same mathematical reality:
| Perspective | Invertible (\(A^{-1}\) exists) | Not Invertible (\(A^{-1}\) doesn’t exist) |
|---|---|---|
| Null Space | \(N(A) = \{\mathbf{0}\}\) | \(N(A) \neq \{\mathbf{0}\}\) |
| Rank | \(\text{rank}(A) = n\) (full rank) | \(\text{rank}(A) < n\) (rank deficient) |
| Pivots | \(n\) pivots (all columns) | \(r < n\) pivots |
| Free Variables | 0 free variables | \(n - r > 0\) free variables |
| Independence | Rows/columns independent | Rows/columns dependent |
| Dimension | \(\dim(N(A)) = 0\) | \(\dim(N(A)) = n - r > 0\) |
| Solutions to \(Ax = 0\) | Only \(x = \mathbf{0}\) | Infinitely many solutions |
| Determinant | \(\det(A) \neq 0\) | \(\det(A) = 0\) |
Fundamental Theorem of Invertible Matrices
For a square \(n \times n\) matrix \(A\), the following are equivalent (all true or all false):
- \(A\) is invertible
- \(A^{-1}\) exists
- \(\text{rank}(A) = n\)
- \(N(A) = \{\mathbf{0}\}\)
- Columns of \(A\) are linearly independent
- Rows of \(A\) are linearly independent
- \(\det(A) \neq 0\)
- \(Ax = 0\) has only the trivial solution
- \(Ax = b\) has a unique solution for every \(b\)
- \(A\) has \(n\) pivot positions
All these conditions are testing whether the linear transformation preserves information. If any information is lost (through dimensional collapse, non-trivial null space, or linear dependence), the transformation cannot be inverted.