Results for: Precision

Ulps Plots Reveal Math Function Accuracy 2

Posted by Cleve Moler,

"ULP" stands for "unit in the last place." An ulps plot samples a fundamental math function such as $\sin{x}$, or a more esoteric function like a Bessel function. The samples are compared with more accurate values obtained from a higher precision computation. A plot of the accuracy, measured in ulps, reveals valuable information about the underlying algorithms.... read more >>

The Pentium Papers — My First MATLAB Central Contribution

Posted by Cleve Moler,

MATLAB Central is celebrating its 15th birthday this fall. In honor of the occasion, MathWorks bloggers are reminiscing about their first involvement with the Web site. My first contribution to the File Exchange was not MATLAB software, but rather a collection of documents that I called the Pentium Papers. I saved this material in November and December of 1994 when I was deeply involved in the Intel Pentium Floating Point Division Affair…. read more >>

Floating Point Denormals, Insignificant But Controversial

Posted by Cleve Moler,

Denormal floating point numbers and gradual underflow are an underappreciated feature of the IEEE floating point standard. Double precision denormals are so tiny that they are rarely numerically significant, but single precision denormals can be in the range where they affect some otherwise unremarkable computations. Historically, gradual underflow proved to be very controversial during the committee deliberations that developed the standard. ... read more >>