What you have observed is due to the imprecision of floating-point numbers. By default Lua uses 64-bit floats which have about 16 decimal digits of precision.
The expression ((12 / 10) - 1) * 10 doesn't have an exact binary floating-point representation, because 12/10 doesn't have an exact binary floating-point representation, or more fundamentally, 1/5 doesn't have an exact binary floating-point representation. Here is a classic paper on the topic.
> ((12 / 10) - 1) * 10
2.0
Here it only looks like the result is 2.0 because that is what the printed representation of the result indicates. But looking more closely:
> string.format("%.16f", ((12 / 10) - 1) * 10)
1.9999999999999996
Now it can be seen that the actual value of the expression ((12 / 10) - 1) * 10, when computed using floating-point arithmetic, is in fact less than 2.0. This is the reason that the floor of this expression is 1.
How would I get 2 from this almost-2 number?
In general this requires more detailed information about the nature of the calculations and the origins of the numbers used. This is the topic of the field called Numerical Analysis.
A coarse solution would be to add an epsilon value to the result before flooring it to correct for floating-point error. This epsilon should be as small as possible to avoid raising values which are slightly less than an integral value above that value.
An epsilon of 1.0e-15 is small enough in this case that it should not affect many results, but note that this epsilon would vary for other calculations. I am assuming here that the input numbers a, b, c, and d are integers, but if any of these are floating-point values the epsilon may need to be adjusted to account for additional floating-point error.
function f (a, b, c, d, epsilon)
local epsilon = epsilon or 1e-15
local result = ((a / b) - c) * d
return math.floor(result + epsilon)
end
For the posted expression of ((12 / 10) - 1) * 10:
> f(12, 10, 1, 10)
2
One comment below suggests adding 0.5 to the result before flooring: do not do this as it will cause errors in many cases.
Consider the expression ((37 / 25) - 1) * 10 which is very near (but not exactly) 4.8.
With a small epsilon (1e-15) the correct result is obtained:
> f(37, 25, 1, 10)
4
But with an epsilon of 0.5 an incorrect result is obtained:
> f(37, 25, 1, 10, 0.5)
5
