7.1: Derivatives

Suppose we have discretized a function of one variable, obtaining a set of $\psi_n \equiv \psi(x_n)$ as described above. For simplicity, we assume that the discretization points are evenly-spaced and arranged in increasing order (this is the simplest and most common discretization scheme). The spacing between points is defined as

$$h \equiv x_{n+1} - x_n.$$

Let us discuss how the first and higher-order derivatives of $\psi(x)$ can be represented under discretization.

7.1.1 First Derivative

The most straightforward representation of the first derivative is the forward-difference formula:

$$\psi'(x_n) \approx \frac{\psi_{n+1} - \psi_n}{h}$$

This is inspired by the usual definition of the derivative of a function, and approaches the true derivative as $h \rightarrow 0$. However, it is not a very good approximation. To see why, let's analyze the error in the formula, which is defined as the absolute value of the difference between the formula and the exact value of the derivative:

$$\mathcal{E} = \left|\psi'(x_n) - \frac{\psi_{n+1} - \psi_n}{h}\right|$$

We can expand $\psi_{n+1}$ in a Taylor series around $x_{n}$:

$$\psi_{n+1} = \psi_n + h\, \psi'(x_n) + \frac{h^2}{2}\psi''(x_n) + \frac{h^3}{6}\psi'''(x_n) + O(h^4)$$

Plugging this into the error formula, we find that the error decreases linearly with the spacing:

$$\mathcal{E} = \left| \frac{h}{2}\psi''(x_n) + O(h^2)\right| \sim O(h).$$

There is a better alternative, called the mid-point formula. This approximates the first derivative by sampling the points to the left and right of the desired position:

$$\psi'(x_n) \approx \frac{\psi_{n+1} - \psi_{n-1}}{2h}.$$

To see why this is better, let us write down the Taylor series for $\psi_{n\pm1}$:

$$\psi_{n+1} = \psi_n \,+\, h\, \psi'(x_n) \,+\, \frac{h^2}{2}\psi''(x_n) \,+\, \frac{h^3}{6}\psi'''(x_n) \,+\, \frac{h^4}{24}\psi''''(x_n) + O(h^5)$$

$$\psi_{n-1} = \psi_n \,-\, h\, \psi'(x_n) \,+\, \frac{h^2}{2}\psi''(x_n) \,-\, \frac{h^3}{6}\psi'''(x_n) \,+\, \frac{h^4}{24}\psi''''(x_n) \,+\, O(h^5)$$

Note that the two series have the same terms involving even powers of $h$, whereas the terms involving odd powers of $h$ have opposite signs. Hence, if we subtract the second series from the first, the result is

$$\psi_{n+1} - \psi_{n-1} = 2 h\, \psi'(x_n) + 2 \frac{h^3}{6}\psi'''(x_n) + O(h^5)$$

Because the $O(h^{2})$ terms are equal in the two series, they cancel out under subtraction, and only the $O(h^{3})$ and higher terms survive. After re-arranging the above equation, we get

$$\psi'(x_n) = \frac{\psi_{n+1} - \psi_{n-1}}{2 h} + O(h^2).$$

Hence, the error of the mid-point formula scales as $O(h^2)$, which is a good improvement over the $O(h)$ error of the forward-difference formula. What's especially nice is that the mid-point formula requires the same number of arithmetic operations to calculate as the forward-difference formula, so this is a free lunch!

It is possible to come up with better approximation formulas for the first derivative by including terms involving $\psi_{n\pm 2}$ etc., with the goal of canceling the $O(h^{3})$ or higher terms in the Taylor series. For most practical purposes, however, the mid-point rule is sufficient.

7.1.2 Second Derivative

The discretization of the second derivative is easy to figure out too. We again write down the Taylor series for $\psi_{n\pm1}$:

$$\psi_{n+1} = \psi_n \,+\, h\, \psi'(x_n) \,+\, \frac{h^2}{2}\psi''(x_n) \,+\, \frac{h^3}{6}\psi'''(x_n) \,+\, \frac{h^4}{24}\psi''''(x_n) \,+\, O(h^5)$$

$$\psi_{n-1} = \psi_n \,-\, h\, \psi'(x_n) \,+\, \frac{h^2}{2}\psi''(x_n) \,-\, \frac{h^3}{6}\psi'''(x_n) \,+\, \frac{h^4}{24}\psi''''(x_n) \,+\, O(h^5)$$

When we add the two series together, the terms involving odd powers of $h$ cancel, and the result is

$$\psi_{n+1} + \psi_{n-1} = 2\psi_n + h^2 \psi''(x_n) + \frac{h^4}{12}\psi''''(x_n) + O(h^5).$$

A minor rearrangement of the equation then gives

$$\psi''(x_n) \approx \frac{\psi_{n+1} - 2\psi_n + \psi_{n-1}}{h^2} + O(h^2).$$

This is called the three-point rule for the second derivative, because it involves the value of the function at the three points $x_{n+1}$, $x_{n}$, and $x_{n-1}$.