Lecture 3: Multiplication and Inverse Matrices

讲座摘要（lecture summary）

Matrix Multiplication

We discuss four different ways of thinking about the product $AB = C$ of two matrices. If $A$ is an $m \times n$ matrix and $B$ is an $n × p$ matrix, then $C$ is an $m × p$ matrix. We use $c_{ij}$ to denote the entry in row $i$ and column $j$ of matrix $C$.

Standard (row times column)

The standard way of describing a matrix product is to say that $c_{ij}$ equals the dot product of row $i$ of matrix $A$ and column $j$ of matrix $B$. In other words, $c_{ij}=\displaystyle\sum_{k=1}^{n}a_{ik}b_{kj}$.

Columns

The product of matrix $A$ and column $j$ of matrix $B$ equals column $j$ of matrix $C$. This tells us that the columns of $C$ are combinations of columns of $A$.

Rows

The product of row $i$ of matrix $A$ and matrix $B$ equals row $i$ of matrix $C$. So the rows of $C$ are combinations of rows of $B$.

Column times row

A column of $A$ is an $m × 1$ vector and a row of $B$ is a $1 × p$ vector. Their product is a matrix:

$$\begin{bmatrix*}[c] 2 \\ 3 \\ 4 \end{bmatrix*} \begin{bmatrix*}[c] 1 & 6\\ \end{bmatrix*} = \begin{bmatrix*}[c] 2 & 12 \\ 3 & 18 \\ 4 & 24 \end{bmatrix*}$$

The columns of this matrix are multiples of the column of $A$ and the rows are multiples of the row of $B$. If we think of the entries in these rows as the coordinates $(2, 12)$ or $(3, 18)$ or $(4, 24)$, all these points lie on the same line; similarly for the two column vectors. Later we’ll see that this is equivalent to saying that the row space of this matrix is a single line, as is the column space.

The product of $A$ and $B$ is the sum of these ”column times row” matrices:

$$AB = \sum_{k=1}^{n} \begin{bmatrix*}[r] a_{1k} \\ \vdots \\ a_{mk} \end{bmatrix*} \begin{bmatrix*}[c] b_{k1} & \dots & b_{kn}\\ \end{bmatrix*}$$

Blocks

If we subdivide $A$ and $B$ into blocks that match properly, we can write the product $AB = C$ in terms of products of the blocks:

$$\begin{bmatrix*}[c] \textcolor{blue}{A_1} & \textcolor{blue}{A_2} \\ A_3 & A_4 \end{bmatrix*} \begin{bmatrix*}[c] \textcolor{#228B22}{B_1} & B_2 \\ \textcolor{#228B22}{B_3} & B_4 \end{bmatrix*} = \begin{bmatrix*}[c] C_1 & C_2 \\ C_3 & C_4 \end{bmatrix*}.$$

Here $C_1 = A_1 B_1 + A_2 B_3$

Inverses

Square matrices

If $A$ is a square matrix, the most important question you can ask about it is whether it has an inverse $A^{−1}$. If it does, then $A^{−1}A = I = AA^{−1}$ and we say that $A$ is invertible or nonsingular.

If A is singular – i.e. A does not have an inverse – its determinant is zero and we can find some non-zero vector $\bm{x}$ for which $A\bm{x} = 0$. For example:

$$\begin{bmatrix*}[c] 1 & 2 \\ 2 & 6 \end{bmatrix*} \begin{bmatrix*}[r] 3 \\ -1 \end{bmatrix*} = \begin{bmatrix*}[c] 0 \\ 0 \end{bmatrix*}.$$

In this example, three times the first column minus one times the second column equals the zero vector; the two column vectors lie on the same line.

Finding the inverse of a matrix is closely related to solving systems of linear equations:

$$\begin{array}{cccc} \begin{bmatrix*}[c] 1 & 3 \\ 2 & 7 \end{bmatrix*} & \begin{bmatrix*}[c] a & c \\ b & d \end{bmatrix*} & = & \begin{bmatrix*}[c] 1 & 0 \\ 0 & 1 \end{bmatrix*} \\ A & A^{-1} & &I \end{array}$$

can be read as saying ”A times column $j$ of $A^{−1}$ equals column $j$ of the identity matrix”. This is just a special form of the equation $A\bm{x} = \bm{b}$.

Gauss-Jordan Elimination

We can use the method of elimination to solve two or more linear equations at the same time. Just augment the matrix with the whole identity matrix $I$:

$$\bigg[\begin{array}{cc|cc} 1 & 3 & 1 & 0 \\ 2 & 7 & 0 & 1 \end{array}\bigg] \to \bigg[\begin{array}{cc|cc} 1 & 3 & 1 & 0 \\ 0 & 1 & -2 & 1 \end{array}\bigg] \to \bigg[\begin{array}{cc|cc} 1 & 0 & 7 & -3 \\ 0 & 1 & -2 & 1 \end{array}\bigg]$$

(Once we have used Gauss’ elimination method to convert the original matrix to upper triangular form, we go on to use Jordan’s idea of eliminating entries in the upper right portion of the matrix.)

$$A^{-1} = \begin{bmatrix*}[r] 7 & -3 \\ -2 & 1 \end{bmatrix*}.$$

As in the last lecture, we can write the results of the elimination method as the product of a number of elimination matrices $E_{ij}$ with the matrix $A$. Letting $E$ be the product of all the $E_{ij}$, we write the result of this Gauss-Jordan elimination using block matrices: $E[\begin{array}{c|c} A & I \end{array}] = [\begin{array}{c|c} I & E \end{array}]$. But if $EA = I$, then $E = A^{−1}$.

Problems and Solutions（习题及答案）

Problem 3.1: Add $AB$ to $AC$ and compare with $A(B + C)$ :

$$A = \begin{bmatrix*}[r] 1 & 2 \\ 3 & 4 \end{bmatrix*} \quad B = \begin{bmatrix*}[r] 1 & 0 \\ 0 & 0 \end{bmatrix*} \quad C = \begin{bmatrix*}[r] 0 & 0 \\ 5 & 6 \end{bmatrix*}$$

Solution

We first add $AB$ to $AC$ :

$$\begin{array}{c} AB=\begin{bmatrix*}[r] 1 & 2 \\ 3 & 4 \end{bmatrix*}\begin{bmatrix*}[r] 1 & 0 \\ 0 & 0 \end{bmatrix*} = \begin{bmatrix*}[r] 1 & 0 \\ 3 & 0 \end{bmatrix*}, AB=\begin{bmatrix*}[r] 1 & 2 \\ 3 & 4 \end{bmatrix*}\begin{bmatrix*}[r] 0 & 0 \\ 5 & 6 \end{bmatrix*} = \begin{bmatrix*}[r] 10 & 12 \\ 20 & 24 \end{bmatrix*} \\\\ \to AB + AC = \begin{bmatrix*}[r] 1 & 0 \\ 3 & 0 \end{bmatrix*} + \begin{bmatrix*}[r] 10 & 12 \\ 20 & 24 \end{bmatrix*} = \begin{bmatrix*}[r] 11 & 12 \\ 23 & 24 \end{bmatrix*}. \end{array}$$

We then compute $A(B + C)$ :

$$B + C = \begin{bmatrix*}[r] 1 & 0 \\ 0 & 0 \end{bmatrix*} + \begin{bmatrix*}[r] 0 & 0 \\ 5 & 6 \end{bmatrix*} = \begin{bmatrix*}[r] 1 & 0 \\ 5 & 6 \end{bmatrix*}$$$$\to A (B+C) = \begin{bmatrix*}[r] 1 & 2 \\ 3 & 4 \end{bmatrix*} \begin{bmatrix*}[r] 1 & 0 \\ 5 & 6 \end{bmatrix*} = \begin{bmatrix*}[r] 11 & 12 \\ 23 & 24 \end{bmatrix*} = AB + AC.$$

Therefore, $AB + AC = A(B + C)$.

Problem 3.2: (2.5 #24. Introduction to Linear Algebra: Strang) Use GaussJordan elimination on $[U \: I]$ to find the upper triangular $U^{−1}$ :

$$UU^{−1} = I \begin{bmatrix*}[r] 1 & a & b \\ 0 & 1 & c \\ 0 & 0 & 1 \end{bmatrix*} \begin{bmatrix*}[r] & & \\ x_1 & x_2 & x_3 \\ & & \end{bmatrix*} = \begin{bmatrix*}[r] 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix*}.$$

Solution

Row reduce $[U \: I]$ to get $[I \: U^{−1}]$ as follows (here, $R_i =$ row $i$)

$$\begin{bmatrix*}[r] 1 & a & b && 1 & 0 & 0 \\ 0 & 1 & c && 0 & 1 & 0 \\ 0 & 0 & 1 && 0 & 0 & 1 \end{bmatrix*} \to \begin{array}{r} (R_1 = R_1 -aR_2) \\ (R_2 = R_2 - cR_2) \\ \quad \end{array} \begin{bmatrix*}[r] 1 & 0 & b-ac && 1 & -a & 0 \\ 0 & 1 & 0 && 0 & 1 & -c \\ 0 & 0 & 1 && 0 & 0 & 1 \end{bmatrix*}$$$$\to \begin{array}{r} (R_1 = R_1 -(b-ac)R_3) \\ \quad \\ \quad \end{array} \begin{bmatrix*}[r] 1 & 0 & 0 && 1 & -a & ac-b \\ 0 & 1 & 0 && 0 & 1 & -c \\ 0 & 0 & 1 && 0 & 0 & 1 \end{bmatrix*} = [I \:\: L^{-1}]$$