Why computer floating points are inaccurate
Juan P. Lima
Posted on April 5, 2022
Some days ago I went through a curious situation, I had a simple function that would turn money into cents, It was working properly until this happened:
18.17 was the float being sent to the function, the function was basically floatNumber x 100 and I had no idea why every number I tested was working just fine but this one was not, I tested it on my browser and the same thing happened, I had to find a way how to make it work…
It happens in almost every programming language
So, after some research, I discovered the 0.1 + 0.2 problem and why in most programming languages 0.1 + 0.2 != 0.3(yes, you read it right), take a look at some of these examples:
JS:
Java:
Dart:
As you can see, in these languages 0.1 + 0.2 != 0.3.
Why does this happen?
Double-precision floating-point format, also known as binary64(https://en.wikipedia.org/wiki/Double-precision_floating-point_format), is a computer number format most of our processors follow, it stores our number in bits this way:
Having 64 bits, 1bit is for the sign so we know if the number is positive or negative, 11bits are the exponent(https://en.wikipedia.org/wiki/Exponentiation) that will tell us if our mantissa represents an integer or a fraction, and 52bits for the mantissa(https://en.wikipedia.org/wiki/Significand) that gives us from 15 to 17 decimal digits precision, the problem is: some numbers just can’t be fully represented exactly in this way, only approximately.
The solution
My solution: The solution for my problem, converting money into cents was the following:
All I needed was to move the decimal points 2x to the right, and I did so by using the e notation(https://en.wikipedia.org/wiki/Scientific_notation#E_notation) which represents 10 to the power of the number that comes next, in my case, 2.
Final thoughts
I brought only one solution, but there are more solutions out there that will fit better what you want to do, so go ahead, find out more solutions, and don’t forget to come back here and comment the solution you’ve found so more people will benefit from this article, wish you all the best.
Read more
https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
https://en.wikipedia.org/wiki/Double-precision_floating-point_format
Posted on April 5, 2022
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.