Powershell adding unwanted decimals (addition)

Question

I am doing some addition in one of my scripts, here is some simplified code:

foreach($entry in $arr){
...
switch($entry.AccessRights)
{

"GenericRead" {$score = 1}
"GenericWrite" {$score = 2}
"GenericAll" {$score = 3}
default {$score = 0.1}
}
$users | where {$_.username -eq $entry.username} | % {$_.aclscore+=$score}
}

You would expect the output to be something like 123.5 or something. But at some point in between (when the score is 0.1) it deviates by 0.0000000000001 plus or minus, so i might get results like 66,1000000000001 or even 84,8999999999999.

Question 1: Why?

Question 2: What can i do to work around that apart from rounding afterwards?

G42 · Accepted Answer · 2017-08-15 07:04:41Z

5

On testing, PowerShell is implicitly casting to the datatype of your variable to [double]. Instead, explicitly cast as [decimal]

Double datatype can hold a wide range of values, but sacrifices exact precision.
Decimal uses more memory (12 bytes as oppose to 8 in double) and has a shorter range of values but retains precision.

This is by no means an in-depth comparison; I recommend you read this table of datatypes and look online for a more complete explanation as this is very fundamental and is language-agnostic.

Code

foreach($entry in $arr){
    ...
    switch($entry.AccessRights)
    {

        "GenericRead" {[decimal]$score = 1}
        "GenericWrite" {[decimal]$score = 2}
        "GenericAll" {[decimal]$score = 3}
        default {[decimal]$score = 0.1}

    }

    $users | where {$_.username -eq $entry.username} | % {$_.aclscore+=$score}

}

Edit - Further explanation

Lack of precision in the double data type arises because numbers are stored in binary, and some numbers cannot be expressed exactly in binary.

Walter Mitty's comment provides a good example of a 1/3, a number that cannot be expressed exactly using a finite number of digits in decimal, or in binary:

1/3 = 0.333333333..... [decimal]
1/3 = 0.010101010..... [binary]

Similarly, the fraction 1/10 cannot be expressed exactly in binary. Whereas it can in decimal.

1/10 = 0.1             [decimal]
1/10 = 0.000110011.... [binary]

edited Aug 15, 2017 at 7:04

answered Aug 14, 2017 at 12:36

G42

10.1k2 gold badges22 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Walter Mitty Over a year ago

Retains precision for decmal fractins, yes. For anumber like one third, both double and decimal will represent it as an approximation.

G42 Over a year ago

@WalterMitty True. OP is using 0.1 do I didn't get into the general cases. Added explanation on the decimal/binary fractions

Collectives™ on Stack Overflow

Powershell adding unwanted decimals (addition)

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related