October 2

Course Readings, 14,15, 199-218

Last time we discussed catastrophic cancellation error when subtracting two relatively close numbers. This error can come up in a couple of ways. Suppose you decide to evaluate $\exp(-2)$ using the series:

$$\exp(-10) = 1 - 10 + 10^2/2! - 10^3/3! + 10^4/4! - \cdots$$

Notice that the later terms in this series get very small, and so does the entire series. Once you get past the first dozen terms you are repeatedly subtracting very small values (odd terms like $-10^{23}/23!$) from the partial sum, which is itself small. This gives repeated cancellation error.

Similarly, whenever you compare two floating point numbers for equality, you are vulnerable to cancellation error:

$$1 - \frac{1}{3} = \frac{2}{3}?$$

The line above is certainly an equation, but the floating-point representations of $1-1/3$ and $2/3$ may be close but different.