Discussion on Errors¶

Truncation errors for Taylor series¶

When we are doing numerical analysis, there are usually two sources of error, round-off and truncation error. The round-off errors are due to the inexactness in the representation of real numbers on a computer and the arithmetic operations done with them. While the truncation errors are due to the approximate nature of the method used, they are usually from using an approximation in place of an exact mathematical procedure, such as that we use the Taylor series to approximate a function. For example, we can use Taylor series to approximate the function $e^x$:

\[e^x = 1+x+\frac{x^2}{2!} + \frac{x^3}{3!} + \frac{x^4}{4!} + \dots\]

Since it takes the infinite sequence to approximate the function, if we only take a few items, we will have a truncation error. For example, if we only use the first 4 items to approximate $e^2$, which will be:

\[ \begin{align}\begin{aligned}e^2 \approx 1+2+\frac{2^2}{2!} + \frac{2^3}{3!} = 6.3333$$, \\We see there is an error associated with it, since we truncate the rest of the terms in the Taylor series. Therefore the function $f(x)$ can be written as the Taylor series approximation plus a truncation error term:\\$$f(x) = f_n(x) + E_n(x)\end{aligned}\end{align} \]

with more terms we use, the approximation will be more close to the exact value. Let’s use Python to see the above example.

TRY IT! Approximate $e^2$ using different order of Taylor series, and print out the results.

import numpy as np

exp = 0
x = 2
for i in range(10):
    exp = exp + \
       ((x**i)/np.math.factorial(i))
    print(f'Using {i}-term, {exp}')
    
print(f'The true e^2 is: \n{np.exp(2)}')

Using 0-term, 1.0
Using 1-term, 3.0
Using 2-term, 5.0
Using 3-term, 6.333333333333333
Using 4-term, 7.0
Using 5-term, 7.266666666666667
Using 6-term, 7.355555555555555
Using 7-term, 7.3809523809523805
Using 8-term, 7.387301587301587
Using 9-term, 7.3887125220458545
The true e^2 is: 
7.38905609893065

Estimate truncation errors¶

We can see the higher order we use to approximate the function at the value, the closer we are to the true value. For each order we choose, there is an error associated with it, and the approximation is only useful if we have an idea of how accurate the approximation is. This is the motivation that we need to understand more about the errors.

From the Taylor series, if we use only the first $n$ terms, we can see:

\[ f(x) = f_n(x) + E_n(x) = \sum_{k = 0}^{n} \frac{f^{(k)}(a)(x-a)^k}{k!} + E_n(x) \]

The $E_n(x)$ is the remainder of the Taylor series, or the truncation error that measures how far off the approximation $f_n(x)$ is from $f(x)$. We can estimate the error using the Taylor Remainder Estimation Theorem, which states:

If the function $f(x)$ has $n+1$ derivatives for all $x$ in an interval $I$ containing $a$, then, for each x in $I$, there exists $z$ between $x$ and $a$ such that

\[ E_n(x) = \frac{f^{(n+1)}(z)(x-a)^{(n+1)}}{(n+1)!} \]

In many times, if we know $M$ is the maximum value of $|f^{(n+1)}|$ in the interval, we will have:

\[ |E_n(x)| \le \frac{M|x-a|^{(n+1)}}{(n+1)!} \]

Therefore, we can get a bound for the truncation error using this theorem. Let’s see the example.

TRY IT! Estimate the remainder bound for the approximation using taylor series for $e^2$ using n = 9.

Let’s work on the error when we use n = 9. We know that $(e^x)' = e^x$, and, a = 0. Therefore, the error related to x = 2 is:

\[ E_n(x) = \frac{f^{(9+1)}(z)(x)^{(9+1)}}{(9+1)!} = \frac{e^{z}2^{10}}{10!} \]

Recall that $ 0 \le z \le 2$, and $e < 3$, we will have

\[ |E_n(x)| \le \frac{3^{2}2^{10}}{10!} = 0.00254 \]

Therefore, if we use Taylor series with n = 9 to approximate $e^2$, our absolute error should be less than 0.00254. Let’s also verifty it below.

abs(7.3887125220458545-np.exp(2))

0.0003435768847959153

Round-off errors for Taylor series¶

Numerically, to add many terms in a sum, we should be mindful of numerical accumulation of errors that is due to floating point round-off errors. Let us see the following example.

EXAMPLE: Approximate $e^{-30}$ using different order of Taylor series, and print out the results.

exp = 0
x = -30
for i in range(200):
    exp = exp + \
       ((x**i)/np.math.factorial(i))
    
print(f'Using {i}-term, our result is {exp}')    
print(f'The true e^2 is: {np.exp(x)}')

Using 199-term, our result is -8.553016433669241e-05
The true e^2 is: 9.357622968840175e-14

From the above example, it is clear that our estimation using Taylor series are not close to the true value anymore, no matter how many terms you include into the calculation. This is due to the round-off errors we discussed before. When using negative large arguments, in order to get a small result, the Taylor series need alternating large numbers to cancel to achieve that. We need many digits of precision in the series to capture both the large and the small numbers with enough remaining digits to get the result in the desired output precision. Thus we have this error in the above example.

< 18.2 Approximations with Taylor Series | Contents | 18.4 Summary and Problems >

Python Numerical Methods

Discussion on Errors¶

Truncation errors for Taylor series¶

Estimate truncation errors¶

Round-off errors for Taylor series¶