Multivariable calculus and vector analysis

Multiplying matrices and vectors

Vectors and matrices in arbitrary dimensions

We’ve dealt with two- and three-dimensional vectors (i.e., vectors in ${\mathbf{R}}^2$ and ${\mathbf{R}}^3$). We can generalize this concept to an arbitrary number of dimensions, say $n$ dimensions. We refer to an $n$-dimensional vector as a vector in ${\mathbf{R}}^n$ and write it as an n-tuple of numbers:

$$\begin{array}{cc}\mathbf{x}=\left(x_1,x_2,x_3,\dots ,x_n\right).& \text{ (1)}\end{array}$$

For example, $\mathbf{a}=\left(1,6,-23,0.23,0,400\right)$ is a vector in ${\mathbf{R}}^6$. (Don’t think there’s any practical use to dealing with dimensions higher than three? See this page for examples of why we often need to study ${\mathbf{R}}^n$ for $n$ larger than 3.)

You can view a matrix simply as a generalization of a vector, where we arrange numbers in both rows and columns. Let’s keep the number of rows and columns arbitrary, letting $m$ be the number of rows and $n$ the number of columns. We refer to such a matrix as an $m\times n$ matrix and write it as

$$\begin{array}{lll}\hfill A=\left[\begin{array}{cccc}\hfill a_{11}\hfill & \hfill a_{12}\hfill & \hfill \dots \hfill & \hfill a_{1n}\hfill \\ \hfill a_{21}\hfill & \hfill a_{22}\hfill & \hfill \dots \hfill & \hfill a_{2n}\hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill \\ \hfill a_{m1}\hfill & \hfill a_{m2}\hfill & \hfill \dots \hfill & \hfill a_{mn}\hfill \end{array}\right].& & \hfill \end{array}$$

You have seen example matrices earlier, such as a $2\times 3$ matrix.

Vectors as matrices

In many cases, we make our lives simpler by viewing a vectors as a special class of matrices. From looking at the above vector and matrix, the only apparent difference between vectors and matrices is that vectors have only one row while the matrices have multiple rows. However, there is one important twist (literally) that isn’t apparent above. When we view vectors as matrices, we actually view them as a rotated version of the standard form (equation (1)), writing an $n$-dimensional vector as a $n\times 1$ column matrix

$$\begin{array}{lll}\hfill \mathbf{x}=\left[\begin{array}{c}\hfill x_1\hfill \\ \hfill x_2\hfill \\ \hfill x_3\hfill \\ \hfill ⋮\hfill \\ \hfill x_n\hfill \end{array}\right].& & \hfill \end{array}$$

We often call $\mathbf{x}$ an $n\times 1$ column vector, so use the terms “column vector” and “column matrix” synonymously. (Note that a column vector has many rows but only one column.)

Matrix-vector product

We define multiplication between a matrix $A$ and a vector $\mathbf{x}$ (i.e., the matrix-vector product) only for the case when the number of columns in $A$ equals the number of rows in $\mathbf{x}$. So, if $A$ is an $m\times n$ matrix (i.e., with $n$ column), then the product $A\mathbf{x}$ is defined for $n\times 1$ column vectors $\mathbf{x}$. If we let $A\mathbf{x}=\mathbf{b}$, then $\mathbf{b}$ is an $m\times 1$ column vector. In other words, the number of rows in $A$ (which can be anything) determines the number of rows in the product $\mathbf{b}$.

The general formula for a matrix-vector product is

$$\begin{array}{lll}\hfill A\mathbf{x}=\left[\begin{array}{cccc}\hfill a_{11}\hfill & \hfill a_{12}\hfill & \hfill \dots \hfill & \hfill a_{1n}\hfill \\ \hfill a_{21}\hfill & \hfill a_{22}\hfill & \hfill \dots \hfill & \hfill a_{2n}\hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill \\ \hfill a_{m1}\hfill & \hfill a_{m2}\hfill & \hfill \dots \hfill & \hfill a_{mn}\hfill \end{array}\right]\left[\begin{array}{c}\hfill x_1\hfill \\ \hfill x_2\hfill \\ \hfill ⋮\hfill \\ \hfill x_n\hfill \end{array}\right]=\left[\begin{array}{c}\hfill a_{11}{x}_1+a_{12}{x}_2+\cdots +a_{1n}{x}_n\hfill \\ \hfill a_{21}{x}_1+a_{22}{x}_2+\cdots +a_{2n}{x}_n\hfill \\ \hfill ⋮\hfill \\ \hfill a_{m1}{x}_1+a_{m2}{x}_2+\cdots +a_{mn}{x}_n\hfill \\ \hfill \hfill \end{array}\right].& & \hfill \end{array}$$

Although it may look confusing at first, the process of matrix-vector multiplication is actually quite simple. One takes the dot product of $\mathbf{x}$ with each of the rows of $A$. (This is why the number of columns in $A$ has to equal the number of components in $\mathbf{x}$.) The first component of the matrix-vector product is the dot product of $\mathbf{x}$ with the first row of $A$, etc. In fact, if $A$ has only one row, the matrix-vector product is really a dot product in disguise, as described here.

For example, if

$$\begin{array}{lll}\hfill A=\left[\begin{array}{ccc}\hfill 1& \hfill -1& \hfill 2\\ \hfill 0& \hfill -3& \hfill 1\end{array}\right]& & \hfill \end{array}$$

and $\mathbf{x}=\left(2,1,0\right)$, then

$$\begin{array}{llll}\hfill A\mathbf{x}& =\left[\begin{array}{ccc}\hfill 1& \hfill -1& \hfill 2\\ \hfill 0& \hfill -3& \hfill 1\end{array}\right]\left[\begin{array}{c}2\hfill \\ 1\hfill \\ 0\hfill \end{array}\right]& \hfill & \\ \hfill & =\left[\begin{array}{c}\hfill 2\cdot 1-1\cdot 1+0\cdot 2\\ \hfill 2\cdot 0-1\cdot 3+0\cdot 1\end{array}\right]& \hfill & \\ \hfill & =\left[\begin{array}{c}\hfill 1\\ \hfill -3\end{array}\right].& \hfill & \end{array}$$

Matrix-matrix product

Since we view vectors as column matrices, the matrix-vector product is simply a special case of the matrix-matrix product (i.e., a product between two matrices). Just like for the matrix-vector product, the product $AB$ between matrices $A$ and $B$ is defined only if the number of columns in $A$ equals the number of rows in $B$. In math terms, we say we can multiply an $m\times n$ matrix $A$ by an $n\times p$ matrix $B$. (If $p$ happened to be 1, then $B$ would be an $n\times 1$ column vector and we’d be back to the matrix-vector product.)

The product $AB$ is an $m\times p$ matrix which we’ll call $C$, i.e., $AB=C$. To calculate the product $B$, we view $B$ as a bunch of $n\times 1$ column vectors lined up next to each other:

$$\begin{array}{lll}\hfill \left[\begin{array}{cccc}\hfill b_{11}\hfill & \hfill b_{12}\hfill & \hfill \dots \hfill & \hfill b_{1p}\hfill \\ \hfill b_{21}\hfill & \hfill b_{22}\hfill & \hfill \dots \hfill & \hfill b_{2p}\hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill \\ \hfill b_{n1}\hfill & \hfill b_{n2}\hfill & \hfill \dots \hfill & \hfill b_{np}\hfill \end{array}\right]=\left[\left[\begin{array}{c}\hfill b_{11}\hfill \\ \hfill b_{21}\hfill \\ \hfill ⋮\hfill \\ \hfill b_{n1}\hfill \\ \hfill \hfill \end{array}\right]\left[\begin{array}{c}\hfill b_{12}\hfill \\ \hfill b_{22}\hfill \\ \hfill ⋮\hfill \\ \hfill b_{n2}\hfill \\ \hfill \hfill \end{array}\right]\cdots \left[\begin{array}{c}\hfill b_{1p}\hfill \\ \hfill b_{2p}\hfill \\ \hfill ⋮\hfill \\ \hfill b_{np}\hfill \\ \hfill \hfill \end{array}\right]\right]& & \hfill \end{array}$$

Then each column of $C$ is the matrix-vector product of $A$ with the respective column of $B$. In other words, the component in the $i$th row and $j$th column of $C$ is the dot product between the $i$th row of $A$ and the $j$th column of $B$. In math, we write this component of $C$ as $c_{ij}=a_{i1}{b}_{1j}+a_{i2}{b}_{2j}+\cdots +a_{in}{b}_{nj}$.

I think an example makes the process clear. Let $A$ be the $2\times 3$ matrix

$$\begin{array}{lll}\hfill A=\left[\begin{array}{ccc}\hfill 0& \hfill 4& \hfill -2\\ \hfill -4& \hfill -3& \hfill 0\end{array}\right]& & \hfill \end{array}$$

and $B$ be the $3\times 2$ matrix

$$\begin{array}{lll}\hfill B=\left[\begin{array}{cc}\hfill 0& \hfill 1\\ \hfill 1& \hfill -1\\ \hfill 2& \hfill 3\end{array}\right].& & \hfill \end{array}$$

Then,

$$\begin{array}{llll}\hfill AB& =\left[\begin{array}{ccc}\hfill 0& \hfill 4& \hfill -2\\ \hfill -4& \hfill -3& \hfill 0\end{array}\right]\left[\begin{array}{cc}\hfill 0& \hfill 1\\ \hfill 1& \hfill -1\\ \hfill 2& \hfill 3\end{array}\right]& \hfill & \\ \hfill & =\left[\begin{array}{ccc}\hfill 0\cdot 0+4\cdot 1-2\cdot 2& \hfill & \hfill 0\cdot 1+4\cdot \left(-1\right)-2\cdot 3\\ \hfill -4\cdot 0-3\cdot 1+0\cdot 2& \hfill & \hfill -4\cdot 1-3\cdot \left(-1\right)+0\cdot 3\end{array}\right]& \hfill & \\ \hfill & =\left[\begin{array}{ccc}\hfill 0+4-4& \hfill & \hfill 0-4-6\\ \hfill 0-3+0& \hfill & \hfill -4+3+0\end{array}\right]& \hfill & \\ \hfill & =\left[\begin{array}{cc}\hfill 0& \hfill -10\\ \hfill -3& \hfill -1\end{array}\right].& \hfill & \end{array}$$

Want more examples?

Contents by topic

1. Geometry in higher dimensions

1a. Preliminaries

1b. Lines and planes

1c. Matrices

• Matrices and determinants
• Multiplying matrices and vectors
• Dot product in matrix notation
• Matrices and linear functions
• The relationship between determinants and area or volume
• Matrix and vector multiplication examples

1d. Visualizing functions in higher dimensions

2. Differential Calculus

3. Integral calculus

4. Vector calculus

5. The fundamental theorems of vector calculus

6. Review material