Column space and nullspace

讲座摘要（lecture summary）

In this lecture we continue to study subspaces, particularly the column space and nullspace of a matrix.

Review of subspaces

A vector space is a collection of vectors which is closed under linear combinations. In other words, for any two vectors $\mathbf{v}$ and $\mathbf{w}$ in the space and any two real numbers $c$ and $d$, the vector $c\mathbf{v} + d\mathbf{w}$ is also in the vector space. A subspace is a vector space contained inside a vector space.

A plane $P$ containing $\begin{bmatrix} 0 \\ 0 \\ 0\end{bmatrix}$ and a line $L$ containing $\begin{bmatrix} 0 \\ 0 \\ 0\end{bmatrix}$ are both subspaces of $\R^3$. The union $P ∪ L$ of those two subspaces is generally not a subspace, because the sum of a vector in $P$ and a vector in $L$ is probably not contained in $P ∪ L$. The intersection $S ∩ T$ of two subspaces $S$ and $T$ is a subspace. To prove this, use the fact that both $S$ and $T$ are closed under linear combinations to show that their intersection is closed under linear combinations.

Column space of $A$

The column space of a matrix $A$ is the vector space made up of all linear combinations of the columns of $A$.

Solving $A\mathbf{x} = \mathbf{b}$

Given a matrix $A$, for what vectors $mathbf{b}$ does $Amathbf{x} = mathbf{b}$ have a solution $mathbf{x}$?

$$\text{Let}\enspace A = \begin{bmatrix*}[c] 1 & 2 & 2 \\ 2 & 1 & 3 \\ 3 & 1 & 4 \\ 4 & 1 & 5 \end{bmatrix*}.$$

Then $A\mathbf{x} = \mathbf{b}$ does not have a solution for every choice of $\mathbf{b}$ because solving $A\mathbf{x} = \mathbf{b}$ is equivalent to solving four linear equations in three unknowns. If there is a solution $\mathbf{x}$ to $A\mathbf{x} = \mathbf{b}$, then $\mathbf{b}$ must be a linear combination of the columns of $A$. Only three columns cannot fill the entire four dimensional vector space – some vectors $\mathbf{b}$ cannot be expressed as linear combinations of columns of $A$.

Big question: what $\mathbf{b}$’s allow $A\mathbf{x} = \mathbf{b}$ to be solved?

A useful approach is to choose $\mathbf{x}$ and find the vector $\mathbf{b} = A\mathbf{x}$ corresponding to that solution. The components of $\mathbf{x}$ are just the coefficients in a linear combination of columns of $A$.

The system of linear equations $A\mathbf{x} = \mathbf{b}$ is solvable exactly when $\mathbf{b}$ is a vector in the column space of $A$.

For our example matrix $A$, what can we say about the column space of $A$? Are the columns of $A$ independent? In other words, does each column contribute something new to the subspace?

The third column of $A$ is the sum of the first two columns, so does not add anything to the subspace. The column space of our matrix $A$ is a two dimensional subspace of $\R^4$.

Nullspace of $A$

The nullspace of a matrix $A$ is the collection of all solutions $\mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ x_3\end{bmatrix}$ to the equation $A\mathbf{x} = 0$.

The column space of the matrix in our example was a subspace of $\R^4$. The nullspace of $A$ is a subspace of $\R^3$. To see that it’s a vector space, check that any sum or multiple of solutions to $A\mathbf{x} = 0$ is also a solution: $A(\mathbf{x}_1 + \mathbf{x}_2) = A\mathbf{x}_1+ A\mathbf{x}_2 = 0 + 0$ and $A(c\mathbf{x}) = cA\mathbf{x} = c(0)$.

In the example:

$$\begin{bmatrix*}[c] 1 & 2 & 2 \\ 2 & 1 & 3 \\ 3 & 1 & 4 \\ 4 & 1 & 5 \end{bmatrix*} \begin{bmatrix} x_1 \\ x_2 \\ x_3\end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \end{bmatrix}$$

the nullspace $N(A)$ consists of all multiples of $\begin{bmatrix} 1 \\ 1 \\ -1\end{bmatrix}$ ; column $1$ plus column $2$ minus column $3$ equals the zero vector. This nullspace is a line in $\R^3$.

Other values of $\mathbf{b}$

The solutions to the equation:

$$\begin{bmatrix*}[c] 1 & 2 & 2 \\ 2 & 1 & 3 \\ 3 & 1 & 4 \\ 4 & 1 & 5 \end{bmatrix*} \begin{bmatrix} x_1 \\ x_2 \\ x_3\end{bmatrix} = \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4 \end{bmatrix}$$

do not form a subspace. The zero vector is not a solution to this equation. The set of solutions forms a line in $\R^3$ that passes through the points $\begin{bmatrix} 1 \\ 0 \\ 0\end{bmatrix}$ and $\begin{bmatrix*}[r] 0 \\ -1 \\ 1\end{bmatrix*}$ but not $\begin{bmatrix} 0 \\ 0 \\ 0\end{bmatrix}$

Problems and Solutions（习题及答案）

Exercises on column space and nullspace

Problem 6.1: (3.1 #30. Introduction to Linear Algebra: Strang) Suppose $\mathbf{S}$ and $\mathbf{T}$ are two subspaces of a vector space $\mathbf{V}$.

• a) Definition: The sum $\mathbf{S} + \mathbf{T}$ contains all sums $\mathbf{s} + \mathbf{t}$ of a vector $\mathbf{s}$ in $\mathbf{S}$ and a vector $\mathbf{t}$ in $\mathbf{T}$. Show that $\mathbf{S} + \mathbf{T}$ satisfies the requirements (addition and scalar multiplication) for a vector space.
• b) If $\mathbf{S}$ and $\mathbf{T}$ are lines in $\mathbf{R}^m$, what is the difference between $\mathbf{S} + \mathbf{T}$ and $\mathbf{S} ∪ \mathbf{T}$? That union contains all vectors from $\mathbf{S}$ and $\mathbf{T}$ or both. Explain this statement: The span of $\mathbf{S} ∪ \mathbf{T}$ is $\mathbf{S} + \mathbf{T}$.

Solution

a) Let $\mathbf{s}$, $\mathbf{s}'$ be vectors in $\mathbf{S}$, let $\mathbf{t}$, $\mathbf{t}'$ be vectors in $\mathbf{T}$, and let $c$ be a scalar. Then

$$(\mathbf{s} + \mathbf{t}) + (\mathbf{s}' + \mathbf{t}') = (\mathbf{s} + \mathbf{s}') + (\mathbf{t} + \mathbf{t}') \enspace \text{and} \enspace c(\mathbf{s} + \mathbf{t}) = c\mathbf{s} + c\mathbf{t}.$$

Thus $\mathbf{S} + \mathbf{T}$ is closed under addition and scalar multiplication; in other words, it satisfies the two requirements for a vector space.

b) If $\mathbf{S}$ and $\mathbf{T}$ are distinct lines, then $\mathbf{S} + \mathbf{T}$ is a plane, whereas $\mathbf{S} ∪ \mathbf{T}$ is only the two lines. The span of $\mathbf{S} ∪ \mathbf{T}$ is the set of all combinations of vectors in this union of two lines. In particular, it contains all sums $\mathbf{s} + \mathbf{t}$ of a vector $\mathbf{s}$ in $\mathbf{S}$ and a vector $\mathbf{t}$ in $\mathbf{T}$, and these sums form $\mathbf{S} + \mathbf{T}$.

Since $\mathbf{S} + \mathbf{T}$ contains both $\mathbf{S}$ and $\mathbf{T}$, it contains $\mathbf{S} ∪ \mathbf{T}$. Further, $\mathbf{S} + \mathbf{T}$ is a vector space. So it contains all combinations of vectors in itself; in particular, it contains the span of $\mathbf{S} ∪ \mathbf{T}$. Thus the span of $\mathbf{S} ∪ \mathbf{T}$ is $\mathbf{S} + \mathbf{T}$.

Problem 6.2: (3.2 #18.) The plane $x − 3y − z = 12$ is parallel to the plane $x − 3y − x = 0$. One particular point on this plane is $(12, 0, 0)$. All points on the plane have the form (fill in the first components)

$$\begin{bmatrix} x \\ y \\ z\end{bmatrix} = \begin{bmatrix} \\ 0 \\ 0\end{bmatrix} + y\begin{bmatrix} \\ 1 \\ 0\end{bmatrix} + z\begin{bmatrix} \\ 0 \\ 1\end{bmatrix}.$$

Solution

The equation $x = 12 + 3y + z$ says it all:

$$\begin{bmatrix} x \\ y \\ z\end{bmatrix} \left( =\begin{bmatrix*}[r] 12 + 3y + z \\ y \\ z\end{bmatrix*} \right) = \begin{bmatrix*}[c] \boxed{12} \\ 0 \\ 0\end{bmatrix*} + y\begin{bmatrix*}[c] \boxed{3} \\ 1 \\ 0\end{bmatrix*} + z\begin{bmatrix*}[c] \boxed{1} \\ 0 \\ 1\end{bmatrix*}$$

Problem 6.3: (3.2 #36.) How is the nullspace $\mathbf{N}(C)$ related to the spaces $\mathbf{N}(A)$ and $\mathbf{N}(B)$, if $C$ = $\begin{bmatrix} A \\ B \end{bmatrix}$ ?

Solution

$\mathbf{N}(C) = \mathbf{N}(A) ∩ \mathbf{N}(B)$ contains all vectors that are in both nullspaces:

$$C\mathbf{x} = \begin{bmatrix} A\mathbf{x} \\ B\mathbf{x} \end{bmatrix} = 0$$

if and only if $A\mathbf{x} = 0$ and $B\mathbf{x} = 0$.