Arithmetic Associativity – Not So Fast

Arithmetic is associative, right? Well, in the world of paper and pencil, where you can often do calculations exactly, that can be true. However, in the computing world, where real numbers can't always be represented exactly because of working with finite precision datatypes, it turns out that you can't depend on the arithmetic to behave the way you were taught in grade school.

Contents

Let's Do Some Math

Suppose I want to check the following:

$$ \sqrt {2} = 2/\sqrt {2} $$

I can do this analytically using the Symbolic Math Toolbox.

symsqrt2 = sqrt(sym(2));
shouldBeZero = symsqrt2 - 2/symsqrt2

shouldBeZero =
0

Now let's perform the same calculation numerically.

mightBeZero = sqrt(2) - 2/sqrt(2)

mightBeZero =
   2.2204e-16

What is happening here is that we are seeing the influence of the accuracy of floating point numbers and calculations with them. I discussed this in an earlier post as well.

Let's Try Another Example

Now let's try something a little different. First, let's find out what the value of eps is for %sqrt {2}%. This should be the smallest (in magnitude) floating point number which, when added to %sqrt {2}%, produces a number different than %sqrt {2}%.

sqrt2eps = eps(sqrt(2))

sqrt2eps =
   2.2204e-16

Next, we want a number smaller in magnitude than this to play with. I'll use half its value.

halfsqrt2eps = sqrt2eps/2

halfsqrt2eps =
   1.1102e-16

And now let's calculate the following expressions, symbolically and numerically.

$$expr1 = \sqrt{2} - \sqrt{2} + halfsqrt2eps$$

$$expr2 = (\sqrt{2} - \sqrt{2}) + halfsqrt2eps$$

$$expr3 = \sqrt{2} + (-\sqrt{2} + halfsqrt2eps)$$

First we do them all symbolically.

expr1 = symsqrt2 - symsqrt2 + sym(sqrt2eps)/2
expr2 = (symsqrt2 - symsqrt2) + sym(sqrt2eps)/2
expr3 = symsqrt2 + (-symsqrt2 + sym(sqrt2eps)/2)
double(expr1)

expr1 =
1/9007199254740992
expr2 =
1/9007199254740992
expr3 =
1/9007199254740992
ans =
   1.1102e-16

Symbolic results are all the same and return half the value of eps.

Now we'll calculate the same expressions numerically.

expr1 = sqrt(2) - sqrt(2) + halfsqrt2eps
expr2 = (sqrt(2) - sqrt(2)) + halfsqrt2eps
expr3 = sqrt(2) + (-sqrt(2) + halfsqrt2eps)

expr1 =
   1.1102e-16
expr2 =
   1.1102e-16
expr3 =
   2.2204e-16

So what's going on here? As I stated earlier, this example illustrates that floating point arithmetic is not associative the way symbolic arithmetic is. There's on reason to get upset about this. But it is worth understanding. And it might well be worth rewriting a computation occasionally, especially if you are trying to compute a very small difference between two large numbers.

Have You Rewritten Expressions to Get Better Accuracy?

Have you found yourself in a situation where you needed to rewrite how to calculate a numeric result (like here, by different groupings) to ensure you got a more accurate solution. Let me know about it here.

Get the MATLAB code

Published with MATLAB® R2013b

Published with MATLAB® R2013b