I’m experimenting with arbitrary precision numbers but I ran into a
precision problem with #power method.
The base:
b =
BigDecimal(“1.034482758620689655172413793103448275862068965517241379310344827586206896551724”)
This is nice:
[65] pry(main)> b.power(4.0001,5).to_s(“F”)
=> “1.1452”
But this one yields gajilion digits, despite the low precision
parameter:
[64] pry(main)> b.power(4.0,5).to_s(“F”)
=>
“1.145230820564952260841164968378904565512151464552278373093579496692262339862090976570839595577995167
408710258016262277652022322103944542550980444830272550796764510852122423760853182822668783694175299491
998229840756361333048675137604431608257538375836478005205851705333523733848357300705094580513261348742578974976”
Using #round solves the problem, but there is clearly a problem. Any
idea how to properly parameter the method so I don’t need to use #round?
Checking the C source (I’m not a C ninja) it looks that a FLOAT value
gets converted if it equals with its rounded self which is certainly
true (although I don’t know the precision) if the value is X.0
Using #round solves the problem, but there is clearly a problem. Any
idea how to properly parameter the method so I don’t need to use #round?
Checking the C source (I’m not a C ninja) it looks that a FLOAT value
gets converted if it equals with its rounded self which is certainly
true (although I don’t know the precision) if the value is X.0
I would not worry about the number of digits in the BigDecimal instance.
Internally you should use most precise figures. If you want to do
output
then you can apply proper formatting, e.g. with printf.
Using #round solves the problem, but there is clearly a problem. Any
idea how to properly parameter the method so I don’t need to use #round?
Checking the C source (I’m not a C ninja) it looks that a FLOAT value
gets converted if it equals with its rounded self which is certainly
true (although I don’t know the precision) if the value is X.0
I would not worry about the number of digits in the BigDecimal instance.
Internally you should use most precise figures. If you want to do
output
then you can apply proper formatting, e.g. with printf.
Cheers
robert
My problem is that if calculation continues with so many digits, it will
be horribly slow. If I repeat the #power method with the long result,
after ten iteration it yields 5.000 digits, I assume a few ten thousand
iteration will result in an OutOfMemory error.
My problem is that if calculation continues with so many digits, it will
be horribly slow. If I repeat the #power method with the long result,
after ten iteration it yields 5.000 digits, I assume a few ten thousand
iteration will result in an OutOfMemory error.
Then there’s probably no way around invoking #round when appropriate.
Btw., what kind of calculations do you do?
In the documentation to ‘BigDecimal#power’, the optional `prec’ argument
is not even mentioned.
You can reduce the number of significant digits by class method limit:
b=BigDecimal.new(“1.034482758620689655172413793103448275862068965517241379310344827586206896551724”)
BigDecimal::limit(5)
b.power(4, 123456789).to_s(‘F’) # “1.1453”, prec argument is ignored
It seems there is a bug in this method implementation. Too busy to
investigate myself.
In the documentation to ‘BigDecimal#power’, the optional `prec’ argument
is not even mentioned.
You can reduce the number of significant digits by class method limit:
It even states that “Note that n must be an Integer.” which is not true,
it can be Float, Rational, Bigdecimal. The C source might tell you more,
there are a bunch of type checks in there.
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.