Question

Using R code please show:

5. The sample variance of a set of observations aı,...,n is given by S is the sample mean. Show that the second formula is more efficient (requires fewer operations) but can suffer from catastrophic cancellation. cancellation with an example sample of size n = 2. ate catastrophic

Answer 1

Catastrophic Cancellation occurs when small numbers are computed from large numbers, which themselves are subject to roundoff error.

Let's consider we have a sample of size =2

X₁ = 3.246587455684 & X₂ = 4.578854847658

X̄ = 3.912721151671

Using 1st formula, (Use a high precision calculator)

S² = $\frac{(3.246587455684-3.912721151671)^2+(4.578854847658-3.912721151671)^2}{1}$

S² = 0.8874682018586018798083

Using 2nd Formula,

S² = $\frac{(3.246587455684^2+4.578854847658^2)-(2*3.912721151671)^2}{2-1}$

S² = 0.8874682018586018798080

On subtracting S² from 1st formula with S² from second formula we do not get zero instead we get 3*10^-22. That is a Catastrophy. Although this number is very small it does not change the fact that answer should be zero but it is not. Hence we can say that although 2nd formula requires less operations compared to 1st it can in some cases lead to catastrophic cancellation.