Stuart’s MATLAB Videos

Watch and Learn

Video tutorial: Tolerances in comparisons

Posted by Doug Hull,

1 views (last 30 days)  |   Likes   |   4 comments
Sometimes you will do a calculation where the answer looks like ‘0’ or ‘1’ or some other number like that. The natural thing to do is to do a test like this: 

if (answer == 1)
     Do something
end
However, if the calculation was done in double precision, answer might actually be 1.000000001 or something like that. This video discusses a strategy for overcoming this: 

if abs(answer - 1) < 1e-14
     Do something
end
 
|
Category: 
Format: Video,
Level: Advanced
  • print

See Also

Comments

To leave a comment, please click here to sign in to your MathWorks Account or create a new one.