The four fundamental subspaces

讲座摘要（lecture summary）​

In this lecture we discuss the four fundamental spaces associated with a matrix and the relations between them.

Four subspaces​

Any $m$ by $n$ matrix $A$ determines four subspaces (possibly containing only the zero vector):

Column space, $C(A)$​

$C(A)$ consists of all combinations of the columns of $A$ and is a vector space in $\mathbb{R}^{m}$ .

Nullspace, $N(A)$​

This consists of all solutions $\mathbf{x}$ of the equation $A\mathbf{x}=\mathbf{0}$ and lies in $\mathbb{R}^{n}$ .

Row space, $C(A^{T})$​

The combinations of the row vectors of $A$ form a subspace of $R^{n}$ . We equate this with $C(A^{T})$ , the column space of the transpose of $A$ .

Left nullspace, $N(A^{T})$​

We call the nullspace of $A^{T}$ the left nullspace of $A$ . This is a subspace of $\mathbb{R}^{m}$ .

Basis and Dimension​

Column space​

The $r$ pivot columns form a basis for $C(A)$

Nullspace​

The special solutions to $A\mathbf{x}=\mathbf{0}$ correspond to free variables and form a basis for $N(A)$ . An $m$ by $n$ matrix has $n-r$ free variables:

Row space​

We could perform row reduction on $A^{T}$ , but instead we make use of $R,$ the row reduced echelon form of $A$ .

Although the column spaces of $A$ and $R$ are different, the row space of $R$ is the same as the row space of $A$ . The rows of $R$ are combinations of the rows of $A$ , and because reduction is reversible the rows of $A$ are combinations of the rows of $R$ .

The first $r$ rows of $R$ are the "echelon" basis for the row space of $A$ :

Left nullspace​

The matrix $A^{T}$ has $m$ columns. We just saw that $r$ is the rank of $A^{T}.$ , so the number of free columns of $A^{T}$ must be $m-r$ :

The left nullspace is the collection of vectors $y$ for which $A^{T}y=0$ . Equivalently, $y^{T}A=0.$ ; here $y$ and 0 are row vectors. We say "left nullspace" because $y^{T}$ is on the left of $A$ in this equation.

To find a basis for the left nullspace we reduce an augmented version of $A$ :

From this we get the matrix $E$ for which $E A=R$ . (If $A$ is a square, invertible matrix then $E\stackrel{\smile}{=}A^{-1}$ .) In our example,

The bottom $m-r$ rows of $E$ describe linear dependencies of rows of $A$ , because the bottom $m-r$ rows of $R$ are zero. Here $m-r=1$ (one zero row in $R$ ).

The bottom $m-r$ rows of $E$ satisfy the equation $\mathbf{y}^{T}A=\mathbf{0}$ and form a basis for the left nullspace of $A$ .

New vector space​

The collection of all $3\times3$ matrices forms a vector space; call it $M$ . We can add matrices and multiply them by scalars and there’s a zero matrix (additive identity). If we ignore the fact that we can multiply matrices by each other, they behave just like vectors.

Some subspaces of $M$ include:

• all upper triangular matrices
• all symmetric matrices
• $D$ , all diagonal matrices

$D$ is the intersection of the first two spaces. Its dimension is $3$; one basis for $D$ is:

Problems and Solutions（习题及答案）​

Problem 10.1: (3.6 #11. Introduction to Linear Algebra: Strang) $A$ is an m by $n$ matrix of rank $r$ . Suppose there are right sides b for which $A\mathbf{x}=\mathbf{b}$ has no solution.

• a) What are all the inequalities ( $<$ or $\leq,$ ) that must be true between $m,n,$ and $r?$
• b) How do you know that $A^{T}\mathbf{y}=\mathbf{0}$ has solutions other than $\mathbf{y}=\mathbf{0}?$

Problem 10.2: (3.6 #24.) $A^{T}\mathbf{y}=\mathbf{d}$ is solvable when ${\mathbf d}$ is in which of the four subspaces? The solution $\mathbf{y}$ is unique when the ____________ contains only the zero vector.