I’m trying to understand the argument to BigDecimal#sqrt. I figured
it was for precision and it does seem to affect it, but I can’t
understand the exactly relationship. Can anyone tell me what the
argument controls and how?
Thanks.
James Edward G. II
yeah, it looks right.
interesting format variation:
minfieldwidth = 22
precision = 16
sprintf “%*.*f”, minfieldwidth, precision,
BigDecimal.new(“66543”).sqrt(precision)
=> " 257.9592991151898786"
On 12/22/05, James Edward G. II [email protected] wrote:
I’m trying to understand the argument to BigDecimal#sqrt. I figured
it was for precision and it does seem to affect it, but I can’t
understand the exactly relationship. Can anyone tell me what the
argument controls and how?
This is the C source, if you haven’t looked at it yet.
BigDecimal_sqrt(VALUE self, VALUE nFig)
{
ENTER(5);
Real *c, *a;
S_INT mx, n;
GUARD_OBJ(a,GetVpValue(self,1));
mx = a->Prec *(VpBaseFig() + 1);
n = GetPositiveInt(nFig) + VpDblFig() + 1;
if(mx <= n) mx = n;
GUARD_OBJ(c,VpCreateRbObject(mx, "0"));
VpSqrt(c, a);
return ToValue(c);
}
I don’t fully understand what’s going on with it, because my C is
pretty weak, but hopefully this will provide some insight from
someone.
On Dec 27, 2005, at 10:27 PM, mathew wrote:
James Edward G. II wrote:
Let me try asking me question a different way… Is this the
correct way to get a printable sqrt() with 16 digits precision?
sprintf “%.16f”, BigDecimal.new(“66543”).sqrt(16)
=> “257.9592991151890260”
Yes.
I’ve written some documentation for BigDecimal which will hopefully
be checked in to Ruby some time.
Is it available online anywhere before then or could I trouble you to
send it to me privately?
James Edward G. II
On Dec 22, 2005, at 4:04 PM, James Edward G. II wrote:
I’m trying to understand the argument to BigDecimal#sqrt. I
figured it was for precision and it does seem to affect it, but I
can’t understand the exactly relationship. Can anyone tell me what
the argument controls and how?
Let me try asking me question a different way… Is this the correct
way to get a printable sqrt() with 16 digits precision?
sprintf “%.16f”, BigDecimal.new(“66543”).sqrt(16)
=> “257.9592991151890260”
James Edward G. II
On Dec 22, 2005, at 4:04 PM, James Edward G. II wrote:
I’m trying to understand the argument to BigDecimal#sqrt. I
figured it was for precision and it does seem to affect it, but I
can’t understand the exactly relationship. Can anyone tell me what
the argument controls and how?
Let me try asking my question a different way… Is this the correct
way to get a printable square root with N digits precision (N = 16 in
this example)?
sprintf “%.16f”, BigDecimal.new(“66543”).sqrt(16)
=> “257.9592991151890260”
James Edward G. II
James Edward G. II wrote:
Let me try asking me question a different way… Is this the correct
way to get a printable sqrt() with 16 digits precision?
sprintf “%.16f”, BigDecimal.new(“66543”).sqrt(16)
=> “257.9592991151890260”
Yes.
I’ve written some documentation for BigDecimal which will hopefully be
checked in to Ruby some time.
mathew