{"id":76,"date":"2009-12-14T03:08:28","date_gmt":"2009-12-14T03:08:28","guid":{"rendered":"https:\/\/blogs.mathworks.com\/seth\/2009\/12\/14\/round-off-error\/"},"modified":"2009-12-14T03:15:29","modified_gmt":"2009-12-14T03:15:29","slug":"round-off-error","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/simulink\/2009\/12\/14\/round-off-error\/","title":{"rendered":"Round-off Error"},"content":{"rendered":"<p><a href=\"https:\/\/blogs.mathworks.com\/loren\/\">Loren<\/a>\r\nrecently posted a simple example of how <a\r\nhref=\"https:\/\/blogs.mathworks.com\/loren\/2009\/12\/04\/comparing-single-threaded-vs-multithreaded-floating-point-calculations\/\">single-threaded\r\ncomputations might produce different results from multi-threaded computations<\/a>.\u00a0\r\nThe cause is floating-point round off, and most people are surprised to learn\r\nthat the &quot;different&quot; results actually agree!\u00a0 In this post, I want to\r\nexplore this same example in Simulink.<\/p>\r\n\r\n<p><strong>Order of Operations is Important<\/strong><\/p>\r\n\r\n<p>Loren\u2019s example shows that the answer you get from the\r\nfollowing equation depends on the order that you perform the operations:<\/p>\r\n\r\n<p><span style='font-family:\"Courier New\"'>1e-16 + 1e-16 + 1e30 - 1e30<\/span><\/p>\r\n\r\n<p>Here is the same equation expressed in Simulink blocks.\u00a0 The\r\nmodel doesn\u2019t explicitly specify the order of operations, so they are done from\r\nfirst to last input (top to bottom, ie <span style='font-family:\"Courier New\"'>((1e-16\r\n+ 1e-16) + 1e30) - 1e30<\/span>).\u00a0 <\/p>\r\n\r\n<p><img decoding=\"async\" src=\"https:\/\/blogs.mathworks.com\/images\/seth\/2009Q4\/lorenOriginal.png\" alt=\"Simulink model adding 4 values with a single rectangular sum block\"><\/p>\r\n\r\n<p>Here is the same equation with a different parenthetical\r\ngrouping, where the inputs to each operator are signals of the same magnitude.<\/p>\r\n\r\n<p><span style='font-family:\"Courier New\"'>(1e-16 + 1e-16) + (1e30 - 1e30)<\/span><\/p>\r\n\r\n<p><img decoding=\"async\" src=\"https:\/\/blogs.mathworks.com\/images\/seth\/2009Q4\/lorenSumScaled.png\" alt=\"Simulink model adding values of similar magnitude with 3 sum blocks\"><\/p>\r\n\r\n<p>In another grouping, the inputs to the operators have very\r\ndifferent magnitudes.<\/p>\r\n\r\n<p><span style='font-family:\"Courier New\"'>(1e-16 + 1e30) + (1e-16 - 1e30)<\/span><\/p>\r\n\r\n<p><img decoding=\"async\" src=\"https:\/\/blogs.mathworks.com\/images\/seth\/2009Q4\/loenSumNotScaled.png\" alt=\"Simulink model adding values of very different magnitudes\"><\/p>\r\n\r\n<p>I like these examples because they show how floating point\r\nround off affects the result.\u00a0 An important aspect of these calculations to consider\r\nis the spacing between floating-point numbers of a certain magnitude.\u00a0 The EPS\r\ncommand gives you the difference between a number and the next floating-point\r\nprecision number.\u00a0 What is EPS for 1e30?<\/p>\r\n\r\n<p><code style=\"font-size: 11pt;\">&gt;&gt; eps(1e30)<br>\r\nans =<br>\r\n\u00a0 1.4074e+014<\/code><\/p>\r\n\r\n<p>Considering that the spacing of floating-point numbers around <span\r\nstyle='font-family:\"Courier New\"'>1e30 <\/span>is MUCH larger than <span\r\nstyle='font-family:\"Courier New\"'>1e-16<\/span>, you can why there is sensitivity\r\nto the order of these operations.\u00a0 For any normal floating-point number X, you\r\ncan expect no difference in the result if you add less than half EPS(X).<\/p>\r\n\r\n<p><code style=\"font-size: 11pt;\">&gt;&gt; 1e30 + 7e13 == 1e30<br>\r\nans =<br>\r\n\u00a0\u00a0\u00a0\u00a0 1<br>\r\n&gt;&gt; 1e30 + 8e13 == 1e30<br>\r\nans =<br>\r\n\u00a0\u00a0\u00a0\u00a0 0<\/code><\/p>\r\n\r\n<p><strong>How could this really affect me?<\/strong><\/p>\r\n\r\n<p>Imagine a model where precision in the results is a critical\r\nconsideration.\u00a0 Choices made about the units of parameters, and the order of\r\noperations could have an impact on precision of the calculation, especially if\r\nworking with a more compact number representation like single precision\r\nfloating-point.<\/p>\r\n\r\n<p><strong>Now it\u2019s your turn<\/strong><\/p>\r\n\r\n<p>Have you ever run into a strange result only to find out it was within\r\nthe bound of floating-point error?\u00a0 Share your experience with us in a <a\r\nhref=\"https:\/\/blogs.mathworks.com\/seth\/?p=76&amp;#comment\">comment here<\/a>.<\/p>\r\n","protected":false},"excerpt":{"rendered":"<p>Loren\r\nrecently posted a simple example of how single-threaded\r\ncomputations might produce different results from multi-threaded computations.\u00a0\r\nThe cause is floating-point round off, and most people... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/simulink\/2009\/12\/14\/round-off-error\/\">read more >><\/a><\/p>","protected":false},"author":40,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[67,76],"tags":[114,458],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/posts\/76"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/users\/40"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/comments?post=76"}],"version-history":[{"count":0,"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/posts\/76\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/media?parent=76"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/categories?post=76"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/tags?post=76"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}