Multivariable calculus and vector analysis

Local extrema

Review of functions of one variable

You remember how to find local extrema (maxima or minima) of a single variable function $f\left(x\right)$. Let’s assume $f\left(x\right)$ is differentiable. Then the first step is to find the critical points $x=a$, where $f{}^{\text{'}}\left(a\right)=0$. Just because $f{}^{\text{'}}\left(a\right)=0$, it does not mean that $f\left(x\right)$ has a local maximum or minimum at $x=a$. But, at all extrema, the derivative will be zero, so we know that the extrema must occur at critical points.

For example, in the graph below, $f\left(x\right)$ is plotted by a green line. The three critical points are marked by colored circles. The red circle marks a local maximum and the blue circle marks a local minimum. The yellow circle marks a critical point that is neither a maximum or a minimum. Even though $f{}^{\text{'}}\left(x\right)=0$ at the yellow circle, the yellow circle does not mark a local extremum.

At each of these critical points, the linear approximation (i.e., tangent line) to $f\left(x\right)$ is a horizontal line since $f^{\text{'}}\left(a\right)=0$. We can determine if $f$ has a local extremum at $x=a$ by looking at the secord-order Taylor polynomial, which for a function of one variable is

$$\begin{array}{lll}\hfill f\left(x\right)\approx f\left(a\right)+\frac{1}{2}f{}^{\text{'}\prime }\left(a\right){\left(x-a\right)}^2,& & \hfill \end{array}$$

since $f{}^{\text{'}}\left(a\right)=0$. As long as $f{}^{\text{'}\prime }\left(a\right)\ne 0$, the Taylor polynomial says that $f\left(x\right)$ looks like the top or bottom of a parabola for $x$ near $a$. If $f{}^{\text{'}\prime }\left(a\right)>0$, then $f\left(x\right)$ is approximately a parabola pointing upward and $f$ has a local minimum at $x=a$, as illustrated by the blue circle, above. If $f{}^{\text{'}\prime }\left(a\right)<0$, then $f\left(x\right)$ is approximately a parabola pointing downward and $f$ has a local maximum at $x=a$, as illustrated by the red circle, above. On the other hand, if $f{}^{\text{'}\prime }\left(a\right)=0$, then the second-order Taylor polynomial doesn’t gives us any more information. At the point $x=a$, $f$ could have a local maximum, or it could have a local minimum, or it might not even have a local extremum, as illustrated by the yellow point, above.

Functions of multiple variables

If $f\left(\mathbf{x}\right)$ is a function of multiple variables, categorizing local extrema proceeds in an analogous way. So that we can visualize $f\left(\mathbf{x}\right)$, we look only at the case of two variables, $\mathbf{x}=\left(x,y\right)$, where we can graph $f\left(x,y\right)$ as a surface. Assuming $f\left(x,y\right)$ is differentiable, local extrema can occur only at critical points $\left(x,y\right)=\left(a,b\right)$, where the derivative of $f\left(x,y\right)$ is zero, i.e., those points $\left(a,b\right)$ where $Df\left(a,b\right)=\left[00\right]$.

If $Df\left(a,b\right)=\left[00\right]$, then the linear approximation (i.e, tangent plane) of $f\left(x,y\right)$ at $\left(a,b\right)$ is a horizontal plane. As in the one-variable case, we can determine if $f$ has a local extremum at $\left(a,b\right)$ by looking at the secord-order Taylor polynomial. If we let $\left(a,b\right)=\mathbf{a}$ (remember that $\left(x,y\right)=\mathbf{x}$), then the second-order Taylor polynomial is

$$\begin{array}{lll}\hfill f\left(\mathbf{x}\right)\approx f\left(\mathbf{a}\right)+\frac{1}{2}{\left(\mathbf{x}-\mathbf{a}\right)}^{T}Hf\left(\mathbf{a}\right)\left(\mathbf{x}-\mathbf{a}\right).& & \hfill \end{array}$$

All this equation says is that, around $\mathbf{x}=\mathbf{a}$, the graph of $z=f\left(x,y\right)$ looks like a quadric surface (unless $Hf\left(a,b\right)$ is zero). In fact, $f\left(x,y\right)$ will look like a paraboloid.

Depending on the second derivative matrix $Hf\left(a,b\right)$, the graph of $f\left(x,y\right)$ might look like an elliptic paraboloid pointing upward, centered at the point $\left(a,b\right)$ (shown by the blue dot, below). In this case, we say that $Hf\left(a,b\right)$ is positive definite, and $f$ has a local minimum at $\left(a,b\right)$.

Alternatively, the graph of $f\left(x,y\right)$ might look like an elliptic paraboloid pointing downward, centered at the point $\left(a,b\right)$ (shown by the red dot, below). In this case, we say that $Hf\left(a,b\right)$ is negative definite, and $f$ has a local maximum at $\left(a,b\right)$.

There is a third possibility that couldn’t happen in the one-variable case. The graph of $f\left(x,y\right)$ might look like a hyperbolic paraboloid centered at the point $\left(a,b\right)$ (shown by the green dot, below). In this case, the graph looks like a local maximum if you move in one direction (the direction where one’s legs would go if one sat on the saddle) and the graph looks like a local minimum if you move in another direction (the direction corresponding to the front and back if one sat on the saddle). In this case, we say that $Hf\left(a,b\right)$ is indefinite, and $f$ has neither a local maximum nor a local minimum at the critical point. Such a critical point is called a saddle point.

There are other cases, which correspond to the yellow point in the one-variable case, above. These are cases where one cannot tell from the second-order Taylor polynomial if $f$ has a local maximum, a local minimum, or neither at the critical point. One would have to look at higher-order terms of the Taylor polynomial to determine the local behavior of the function.

Here are some examples (which assume knowledge of determining the definiteness of $Hf\left(a,b\right)$ that is not discussed in this reading.)

Contents by topic

1. Geometry in higher dimensions

2. Differential Calculus

2a. The partial derivative

2b. The derivative and differentiability

2c. The chain rule, directional derivative and gradient

2d. Applications of differential calculus

• Taylor's theorem
• Taylor polynomial example
• Local extrema
• Extrema examples

3. Integral calculus

4. Vector calculus

5. The fundamental theorems of vector calculus

6. Review material