C# Floating Point Accuracy

Question

Trying to create a Mandelbrot set, I have been trying to using 8 and 15 digit floating point variables in my program, and have run into an issue: the double approximates to 0. I tried unit testing and wrote this code to isolate the issue. Using the variable viewer, the h and w values both were on 0.0, as opposed to 0.00185185185185 and 0.0015625, yet when I just write double h = 0.0015625, it works.

Many thanks for assistance.

int apparentwidth = 3;
int apparentheight = 2;
int height = 1080;
int width = 1920;

double w = (apparentwidth / width);
double h = (apparentheight / height);

Console.WriteLine(w);
Console.WriteLine(h);

the math is being performed on integers and then you are converting to floating point. You probably want to convert before doing the math. — kenny
– kenny, Commented Jan 31, 2014 at 1:41

Grant Winney · Accepted Answer · 2014-01-31 01:55:16Z

3

You're dividing two int variables, and the result is an int. You're storing the result in a double, but the scale (portion after the decimal) has already been lost.

Use double throughout:

 double apparentwidth = 3;
 double apparentheight = 2;
 double height = 1080;
 double width = 1920;

 double w = (apparentwidth / width);
 double h = (apparentheight / height);

Or cast one of the variables to a double when dividing:

 double w = ((double)apparentwidth / width);
 double h = ((double)apparentheight / height);

edited Jan 31, 2014 at 1:55

answered Jan 31, 2014 at 1:41

Grant Winney

66.6k13 gold badges120 silver badges167 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Timeout Over a year ago

I do not agree with using double throughout. There does not appear to be a need to use a 64 bit number here except preserving precision on divide by casting. Heck, based on what's in the OP a short would likely suffice.

TheBuzzSaw · Accepted Answer · 2014-01-31 01:41:34Z

1

You are doing integer division on accident.

double w = (double)apparentwidth / (double)width;
double h = (double)apparentheight / (double)height;

answered Jan 31, 2014 at 1:41

TheBuzzSaw

8,8765 gold badges41 silver badges59 bronze badges

Comments

Sam · Accepted Answer · 2014-01-31 02:06:11Z

1

Just to provide further explanation to the other answers:

When you do mathematical operations on ints, the result is an int, and this is achieved by effectively truncating the non-integer portion:

3 / 2 == 1

When an operation is performed involving at least one double, the int input is first converted to a double, so the result is a double:

1.0 / 4 == 0.25
4 * 0.25 == 1

And of course, if both inputs are double the result is double and no implicit conversion occurs:

1.0 / 4.0 == 0.25

answered Jan 31, 2014 at 2:06

Sam

42.8k39 gold badges184 silver badges232 bronze badges

Collectives™ on Stack Overflow

C# Floating Point Accuracy

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related