1

Why is this not a duplicate of these great SO articles?

While the two posts linked in the comments below are excellent I am specifically looking for information that helps me to address this issue in native JS. I know that JS shouldn't be the first choice for complex math, but given the limitation that this calculator is meant to run in the browser it is the tool that I have decided to work with.

Background

I'm trying to make a calculator with TypeScript without any libraries (like Big.js) and without using string concatenation in the inner logic of the calculator.

Examples

When a user wants to type the number 8.5:

  • The 8 key is pressed
  • The decimal key is pressed
  • The 5 key is pressed

Mathematically I create this number in the display with the following snippet:

8 + 5 * 0.1

This works but if I continue down the decimal places I encounter something unexpected:

8.5 + 5 * 0.01 // 8.55
8.55 + 5 * 0.001 // 8.555000000000001

Question

What is the best way to handle this without converting the number to a string? Is there an intelligent way to impose a limit on the precision of the calculator so that it only supports accuracy to so many decimal places?

Thanks for your help!

Improve this question
6
  • 2
    Possible duplicate of Is floating point math broken? Commented Sep 23, 2019 at 18:54
  • 1
    Possible duplicate of How to deal with floating point number precision in JavaScript? Commented Sep 23, 2019 at 19:03
  • JavaScript has some known issues with math. Commented Sep 23, 2019 at 19:08
  • That first potential duplicate has a fascinating answer. I think they both give me a better high level understanding of the problem but neither of them give much guidance for me in JavaScript specifically. I'll edit my question to reflect that. Commented Sep 23, 2019 at 19:09
  • 1
    You should know that no calculation or algorithm can produce 8.55 or 8.555 in a JavaScript Number because those values are not representable in the format used for Number. The closest possible values are 8.550000000000000710542735760100185871124267578125 and 8.55499999999999971578290569595992565155029296875. Commented Sep 23, 2019 at 21:27

2 Answers 2

Reset to default
1

Use .toFixed(): https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/toFixed

or .toPrecision(): https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/toPrecision depending on your needs.

Note that you don't need to convert numbers all the time. The only place to convert - is for the final output to the view. In this case we can even leave it in string format.

That is an answer to the question "how to get manage with" (like in description). About "why do we get such result" - first comment provides great answer.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

These conversions for display would successful conceal many errors, but they do not solve the problem of accurately converting the numbers from the input decimal to the internal Number format. The errors in that conversion may be important for various algorithms; merely concealing small errors during output may be inadequate.
It is the way to deal with precision in result layout. If we need some advanced calculation logic which takes into the account some extremely high precision, we should use Tolerance Value for comparison and other similar staff. But it is not that case.
Comparing with a tolerance makes this worse by introducing false positives for the comparison.
1

The easiest way to get the Number value that is closest to what the user enters is to build a numeral in a string from the user’s keypress and then convert them with String.toNumber().

Numbers such as 8.55 or 8.555 are not exactly representable in the Number format. The closest values are 8.550000000000000710542735760100185871124267578125 and 8.55499999999999971578290569595992565155029296875. Converting the strings “8.55” and “8.555” with .toNumber() should produce these values.

Because these are the closest representable values, no calculation or algorithm can produce any closer values in the Number format.

For simple additions, subtractions, and limited multiplications, you can mimic decimal arithmetic by rounding to a desired number of decimal digits after each Number operation. However, this is generally not a feasible approach because other operations and various sequences of operations will exceed the ability to mimic decimal arithmetic reasonably.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.