# Video tutorial: Tolerances in comparisons4

Posted by Doug Hull,

Sometimes you will do a calculation where the answer looks like ‘0’ or ‘1’ or some other number like that. The natural thing to do is to do a test like this:

if (answer == 1)
Do something
end

However, if the calculation was done in double precision, answer might actually be 1.000000001 or something like that. This video discusses a strategy for overcoming this:

if abs(answer - 1) < 1e-14
Do something
end


4 views (last 30 days)  | |