intermediate float bug
DavinciWiki Navigation Bar
Contents
Contact Developers
- davinci-dev [AT] mars.asu.edu
All other topics
- See navigation on the left
This problem only affects old versions of davinci. If you're using davinci 2.18 (in which real numbers default to double) you can ignore this page, and if you're not you should certainly update.
This page describes the intermediate float bug in Davinci, and how to work around it. See Syntax for a description of the ranges of float and double formats.
Contents |
The problem
In most cases, single-precision floating point format (float in Davinci) is sufficient. A number stored as a float is typically accurate to seven significant figures. However, great care must be taken when double precision (double in Davinci) is required. A number stored as a double is typically accurate to fifteen significant figures.
Any decimal number entered into Davinci is first read as a single-precision float.
dv> format(2.5) "float"
This means that entering a number with the double function only actually converts a single-precision float number to double, and the double is stored with double precision but is only really accurate to single precision.
dv> SCALE=16 16 dv> a = 4.2 4.199999809265137 dv> format(a) "float" dv> b = double(4.2) 4.199999809265137 dv> format(b) "double"
The solution
Tricks with integers
In general, this bug is very hard to avoid. Low-magnitude integers convert exactly to floats (single or double precision), so mathematical tricks help with a few values.
dv> e = exp(double(1)) 2.718281828459045 dv> pi = 4*atan(double(1)) 3.141592653589793
For tricks like that to work, the integer must be converted to a double before it is used in computations. Note the difference in the following apparently similar commands. The first value is correct to all 15 places past the decimal; the second value is only correct to 7 places past the decimal, despite also being a double.
dv> sqrt(double(2)) 1.414213562373095 dv> double(sqrt(2)) 1.414213538169861
Complete solution: strings and the atod function
Using double on an integer, as above, is only really useful when dealing numbers that can be expressed as some simple function of integers. Fortunately there is a trick that works on all numbers. The function atod converts a string directly to a double format, without an intermediate float. To ensure correct double-precision values, the atod function should be used on a string specifying the number's value.
dv> 4.2
4.199999809265137
dv> double(4.2)
4.199999809265137
dv> atod("4.2")
4.200000000000000
Workaround examples
The following functions were consciously designed to avoid the intermediate float bug. Take a look at their code to see examples of the workarounds.
Functions using atod
In addition, the values in the physical constants structure (con) and the solar system structures (Sun, Earth, Moon, Mars, Phobos, Deimos) are defined using atod. See conhelp and solsys.
