intermediate float bug

Description This page describes the intermediate float bug in Davinci, and how to work around it. Any decimal number entered into Davinci is first read as a single-precision float. In most cases, single-precision floating point format (typically seven digits of accuracy) is sufficient. However, great care must be taken when double precision (typically 15 digits of accuracy) is required. dv> format(2.5) "float" This means that entering a number with the double function only actually converts a single-point precision number to double, and the double is stored with double precision but is only really accurate to single precision. dv> SCALE=16 16 dv> a = 4.2 4.199999809265137 dv> format(a) "float" dv> b = double(a) 4.199999809265137 dv> format(b) "double" In general, this is very hard to get around. Low-magnitude integers convert exactly to floats, so mathematical tricks help with a few values. dv> e = exp(double(1)) 2.718281828459045 dv> pi = 4*atan(double(1)) 3.141592653589793 For tricks like that to work, the integer must be converted to a double before it is used in computations. Note the difference in the following apparently similar commands. The first value is correct to all 15 places past the decimal; the second value is only correct to 7 places past the decimal, despite also being a double. dv> sqrt(double(2)) 1.414213562373095 dv> double(sqrt(2)) 1.414213538169861 Using double on an integer, as above, is only really useful when dealing numbers that can be expressed as some simple function of integers. Fortunately there is a trick that works on all numbers. The function atod converts a string directly to a double format, without an intermediate float. To ensure correct double-precision values, the atod function should be used on a string specifying the number's value. dv> 4.2 4.199999809265137 dv> double(4.2) 4.199999809265137 dv> atod("4.2") 4.200000000000000 Examples of functions consciously designed with intermediate float bug workarounds (using atod or an integer trick): erf fact gamma inverf invnorm

Wiki Navigation Bar Contents Tutorial User's Guide Davinci Cheat Sheet Functions Function Categories Syntax Common Procedures Glossary Change Log Feature Requests Q & A Tips and Tricks Download Davinci Installation FAQ Discussion Page Editing Guide Mailing List News Contact Developers davinci-dev [AT] mars.asu.edu All other topics See navigation on the left  Quick News Last Updated: Feb-2022 Use new library_update() function Mac, Windows, Ubuntu, and RedHat/CentOS versions released Updates to the Davinci Library More News See the News page