I’m having a little problem with numeric precision and being a NOOB to
Ruby, I’m not quite sure where I am going wrong.
When I perform the following calculation:
n = ((9703.0 * 16346204482307500.0) + 1).to_f
d = (65537.0).to_f
res = (n / d).to_f
Ruby is returning a value of 249,419,480,327.0
However, the more precise number I am expecting is 249,419,480,328.784
How do I do this to get the more precise value?