Working with decimals in javascript

Question

I have a javascript function that converts feet into a usable object that will take the total feet and break it down into feet / inch / fraction.

In the example below the decimal fraction portion should be zero and my inch should be 4

Lets say I've got the following.

var inches = 40;

Now when I call the function below like so unPackDecimalFeet(40 /12);

  unPackDecimalFeet: function (feetInchFraction) {

        var inchFraction = (feetInchFraction - ~~feetInchFraction) * 12;
        return {
            feet: ~~feetInchFraction,
            inch: ~~inchFraction,
            fraction: inchFraction - ~~inchFraction
        };
    }

My return value is below.

feet: 3
fraction: 0.9999999999999964
inch: 3

**The above return value should read.

feet: 3
fraction: 0
inch: 4

How would I need to process the feet so that I can get the correct return value?

Some Guy · Accepted Answer · 2012-09-06 18:51:09Z

1

I'd say multiply it by 1000 in the beginning, then do your calculations, and then divide by 1000. IEEE Floating Point arithmetic is the problem here. If you need more calculations, look into BigInteger libraries. (BigInt, sometimes)

answered Sep 6, 2012 at 18:51

Some Guy

16.2k10 gold badges61 silver badges68 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Vinyl Windows Over a year ago

Can you please give me a break down.

d4v3y0rk · Accepted Answer · 2012-09-06 19:28:56Z

0

var inchesIn;
inchesIn = 40;
document.writeln('Feet: ' + Math.floor(inchesIn/12));
document.writeln('Inches: ' + inchesIn%12);

answered Sep 6, 2012 at 19:28

d4v3y0rk

3451 silver badge9 bronze badges

Comments

Pete · Accepted Answer · 2012-09-06 19:42:59Z

You're doing a lot to try and save yourself from tiny little inconsequential fraction issues throughout your code. I think maybe simpler is better:

unPackDecimalFeet : (function (feet) {
    var flooredFeet = 0 | feet,
        flooredInches = 0 | ((feet - flooredFeet) * 12),
        inchFraction = (feet * 12) - (0 | (feet * 12));

    return {
        feet: flooredFeet,
        inch: flooredInches,
        fraction: +(inchFraction.toFixed(2)) //Rounded to two decimals
    };
})

The only "trickiness" there is with 0 | as an alternative to Math.floor(). Feel free to use Math.floor() if it makes you feel better about maintainability. Also, I rounded off the "fraction" part of the response to the nearest 100th of an inch. You can round more precisely if you want, but some rounding is encouraged to prevent things like 0.9999999999999994 or whatever due to IEEE-754 gone wild.

Collectives™ on Stack Overflow

Working with decimals in javascript

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related