http://www.learncpp.com/cpp-tutorial/25-floating-point-numbers/ I have been about this lately to review C++.
In general computing class professors tend not to cover these small things, although we knew what rounding errors meant.
Can someone please help me with how to avoid rounding error?
The tutorial shows a sample code
#include <iomanip>
int main()
{
using namespace std;
cout << setprecision(17);
double dValue = 0.1;
cout << dValue << endl;
}
This outputs
0.10000000000000001
By default float is kept 6-digits of precisions. Therefore, when we override the default, and asks for more (n this case, 17!!), we may encounter truncation (as explained by the tutorial as well). For double, the highest is 16.
In general, how do good C++ programmers avoid rounding error? Do you guys always look at the binary representation of the number?
Thank you.