The linear approximation

The linear approximation in one-variable calculus

You just reviewed the fact that in one-variable calculus, a function is differentiable if and only if it has a tangent line. Using the neuron firing example of the that reading, the tangent line at one point looked liked the following figure.

PIC

The equation of the tangent line at i = a is

f(i) = r(a) + r'(a)(i - a), (1)
where r'(a) is the derivative of r(i) at the point where i = a. The tangent line f(i) is called a linear approximation to r(i). The fact that r(i) is differentiable means that it is nearly linear around i = a.

Why do we care if r(i) is differentiable? Well, unfortunately, when studying a neuron, the function r(i) may not be a pretty function. In fact, we might not even have a nice equation for r(i). Although it is bad news for mathematicians, neurons don’t come with the function r(i) written on them.

In some applications, though, we may know that the input i isn’t going to vary a whole lot. We may know that i is going to be close to some value a. In that case, we may approximate r(i) by its linear approximation f(i) around i = a. If r(i) is differentiable at i = a, we make only a small error with this approximation. Moreover, since f(i) is linear, it is easy to work with, much easier than the original r(i). So if we had to do some calculation involving the response of the neuron, we’d make our lives easier by assuming its output rate is f(i) rather than r(i).

The linear approximation in two dimensions

The two dimensions, a function is differentiable if and only if it has a tangent plane. Using the two-dimensional version of the neuron firing example, the tangent plane at one point looked liked the following figure.

The equation for the tangent plane at (i,s) = (a1,a2) was the expression

f(i,s) = r(a1,a2) + [ ] ∂r(a1,a2) ∂i (i - a1) + [ ] ∂r-(a1,a2 ) ∂s (s - a2). (2)

Just like in the one-variable case, the tangent plane f(i,s) is called a linear approximation to r(i,s). The fact that r(i,s) is differentiable means that it is close to its linear approximation around (i,s) = a.

We can use the fact that r(i,s) is differentiable to simplify calculations that involve the neural output rate in response to input i and nicotine s. For example, looking at the above graph, suppose we wanted to analyze how small changes in nicotine effect the neural response. We might be interested in the response just for input i and nicotine s near the green point. In that case, we could use the equation of the tangent plane. Although it may look complicated, it is actually has a very simply dependence on both i and s. Using this linear approximation will be a lot easier than using whatever complicated formula we have for the actual r(i,s). Since r(i,s) isn’t differentiable around the red point, we are unable to use a linear approximation there.

Examples of linear approximations can be found in the examples on differentiability.