Ruby Float Round Error Bug?

Question

I am getting the following rounding error when I try to Unit test the class below:

class TypeTotal
    attr_reader  :cr_amount,  :dr_amount,
                 :cr_count,  :dr_count

    def initialize()
        @cr_amount=Float(0);    @dr_amount=Float(0)
        @cr_count=0;            @dr_count= 0
    end

    def increment(is_a_credit, amount, count=1)
        case is_a_credit        
         when true
            @cr_amount = Float(amount)+ Float(@cr_amount)
            @cr_count += count
         when false
            @dr_amount = Float(amount)+ Float(@dr_amount)
            @dr_count += count
        end
    end
end

Unit Test:

require_relative 'total_type'
require 'test/unit'

class TestTotalType < Test::Unit::TestCase 
  #rounding error
  def test_increment_count()
    t = TypeTotal.new()
       t.increment(false, 22.22, 2)       
       t.increment(false, 7.31, 3) 
     assert_equal(t.dr_amount, 29.53)    
  end
end

Output:

  1) Failure:
test_increment_count(TestTotalType) [total_type_test.rb:10]:
<29.529999999999998> expected but was
<29.53>.

1 tests, 1 assertions, 1 failures, 0 errors, 0 skips

I am using Floats because it was recommended in the Pick Ax book for dollar values because they shouldn't be effective by round errors.

I am running on ruby 1.9.2p290 (2011-07-09) [i386-mingw32] on Windows 7 64-bit Home and Windows XP 32-bit Pro.

I've tried

casting my variables to floats
removing += and spelling out increment

The behavior appears random:

12.22 + 7.31 works
11.11 + 7.31 doesn't work
11.111 + 7.31 works

Any ideas whats going wrong?

Whatever you do, don't file it as a bug!

Andrew Grimm
– Andrew Grimm

2011-10-13 22:15:16 +00:00
Commented Oct 13, 2011 at 22:15 — Andrew Grimm
– Andrew Grimm, Commented Oct 13, 2011 at 22:15

Jon Skeet · Accepted Answer · 2011-10-13 19:38:03Z

5

Are you sure that was the advice given? I'd expect the advice to be not to use Floats, precisely because they use binary floating point arithmetic, and so are prone to rounding errors. From the Float documentation:

Float objects represent inexact real numbers using the native architecture's double-precision floating point representation.

If you could quote the exact advice you're referring to, that would help.

I would suggest you use BigDecimal instead, or use an integer with implicit units of "cents" or "hundreds of a cent" or something similar.

edited Oct 13, 2011 at 19:38

answered Oct 13, 2011 at 11:20

Jon Skeet

1.5m893 gold badges9.3k silver badges9.3k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Nathan Over a year ago

Can't believed I missed the footnote explaining why the example was wrong... I guess it shows you should code on well rested brain :)

knut · Accepted Answer · 2011-10-13 18:40:15Z

2

The problems with float are already mentioned.

When you test with floats, you should not use assert_equal but assert_in_delta.

Example:

require 'test/unit'

class TestTotalType < Test::Unit::TestCase 
  TOLERANCE = 1E-10 #or another (small) value

  #rounding error
  def test_increment_count()
    t = TypeTotal.new()
       t.increment(false, 22.22, 2)       
       t.increment(false, 7.31, 3) 
     #~ assert_equal(t.dr_amount, 29.53)  #may detect float problems
     assert_in_delta(t.dr_amount, 29.53, TOLERANCE)    
  end
end

answered Oct 13, 2011 at 18:40

knut

28k6 gold badges88 silver badges113 bronze badges

Comments

Nathan · Accepted Answer · 2011-10-13 19:20:09Z

The solution was to use Big Decimal:

require 'bigdecimal'

class TypeTotal
    attr_reader  :cr_amount,  :dr_amount,
                 :cr_count,  :dr_count

    def initialize()
        @cr_amount=BigDecimal.new("0");  @cr_count=0,
        @dr_amount=BigDecimal.new("0");  @dr_count=0

    end

    def increment(is_a_credit, amount, count=1)
        bd_amount = BigDecimal.new(amount)
        case is_a_credit        
         when true
            @cr_amount= bd_amount.add(@cr_amount, 14)
            @cr_count += count
         when false
            @dr_amount= bd_amount.add(@dr_amount, 14)
            @dr_count = count
        end
    end

The Pick Ax (p53) book used float for currency as an example but had a foot note explaining that you need to either add .5 cent to when you display the value or use Big Decimal.

Thanks for are your help!

Collectives™ on Stack Overflow

Ruby Float Round Error Bug?

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related