Pack("l"), 64 bit question

danielb · February 15, 2006, 5:52pm

Hi all,

Ruby 1.8.4
Solaris 10

Is this correct?

32 bit

irb(main):002:0> [-1].pack(“l”)
=> “\377\377\377\377”
irb(main):003:0> “\377\377\377\377”.unpack(“l”)
=> [-1]

64 bit

irb(main):002:0> [-1].pack(“l”)
=> “\377\377\377\377”
irb(main):003:0> “\377\377\377\377”.unpack(“l”)
=> [4294967295]

Regards,

Dan

danielb · February 15, 2006, 5:55pm

On Thu, 16 Feb 2006, Daniel B. wrote:

irb(main):003:0> “\377\377\377\377”.unpack(“l”)
Dan
yes. ‘l’ is unsigned. i think you may want ‘i’.

regards.

-a

danielb · February 15, 2006, 6:04pm

On Thu, 16 Feb 2006, Daniel B. wrote:

irb(main):003:0> “\377\377\377\377”.unpack(“l”)
Dan
btw. this is useful for me

irb(main):008:0> to_bin = lambda{|s| ‘[ ’ << s.unpack(‘c*’).map{|b|
‘%8.8b’ % b}.join(’ ‘) << ’ ]’}
=> #Proc:0xb7569f90@:8(irb)

irb(main):009:0> to_bin[ [-1].pack(“l”) ]
=> “[ 11111111 11111111 11111111 11111111 ]”

irb(main):010:0> to_bin[ [-1].pack(“i”) ]
=> “[ 11111111 11111111 11111111 11111111 ]”

irb(main):011:0> to_bin[ [1].pack(“l”) ]
=> “[ 00000001 00000000 00000000 00000000 ]”

irb(main):012:0> to_bin[ [1].pack(“i”) ]
=> “[ 00000001 00000000 00000000 00000000 ]”

-a

danielb · February 15, 2006, 6:38pm

Hi,

In message “Re: pack(“l”), 64 bit question”
on Thu, 16 Feb 2006 01:50:34 +0900, Daniel B.
[email protected] writes:

|Ruby 1.8.4
|Solaris 10
|
|Is this correct?
|
|# 32 bit
|irb(main):002:0> [-1].pack(“l”)
|=> “\377\377\377\377”
|irb(main):003:0> “\377\377\377\377”.unpack(“l”)
|=> [-1]
|
|# 64 bit
|irb(main):002:0> [-1].pack(“l”)
|=> “\377\377\377\377”
|irb(main):003:0> “\377\377\377\377”.unpack(“l”)
|=> [4294967295]

Hmm, unpack(“l”) should have returned negative value unless “_” suffix
is supplied.

						matz.

danielb · February 15, 2006, 6:50pm

Hi,

In message “Re: pack(“l”), 64 bit question”
on Thu, 16 Feb 2006 02:35:53 +0900, Yukihiro M.
[email protected] writes:

||irb(main):003:0> “\377\377\377\377”.unpack(“l”)
||=> [4294967295]
|
|Hmm, unpack(“l”) should have returned negative value unless “_” suffix
|is supplied.

This means EXTEND32() macro in pack.c is not working on 64bit Solaris
(and perhaps on other 64bit systems neither). I have no 64bit machine
at hand. Could somebody confirm?

						matz.

danielb · February 15, 2006, 6:50pm

Yukihiro M. wrote:

|Is this correct?
|irb(main):003:0> “\377\377\377\377”.unpack(“l”)
|=> [4294967295]

Hmm, unpack(“l”) should have returned negative value unless “_” suffix
is supplied.
  					matz.

Here’s some more info that may or may not be useful:

64 bit

irb(main):004:0> “\377\377\377\377”.unpack(“l_”)
=> [nil]
irb(main):005:0> [-1].pack(“l_”)
=> “\377\377\377\377\377\377\377\377”
irb(main):006:0> “\377\377\377\377\377\377\377\377”.unpack(“l_”)
=> [-1]

Regards,

Dan

danielb · February 16, 2006, 9:52am

Hi,

In message “Re: pack(“l”), 64 bit question”
on Thu, 16 Feb 2006 17:45:50 +0900, H.Yamamoto
[email protected] writes:
|
|>This means EXTEND32() macro in pack.c is not working on 64bit Solaris
|>(and perhaps on other 64bit systems neither). I have no 64bit machine
|>at hand. Could somebody confirm?
|
|Me neigher. But we can use HP TestDrive
|
|Probably this patch will solve the problem.

Daniel, could you try this patch on your 64bit box?

						matz.

|— pack.c 13 Oct 2005 14:30:49 -0000 1.62.2.12
|+++ pack.c 16 Feb 2006 06:01:07 -0000
|@@ -347,11 +347,11 @@ num2i32(x)
| return 0; /* not reached /
| }
|
|-#if SIZEOF_LONG == SIZE32 || SIZEOF_INT == SIZE32
|+#if SIZEOF_LONG == SIZE32
| # define EXTEND32(x)
| #else
| / invariant in modulo 1<<31 */
|-# define EXTEND32(x) do {if (!natint) {(x) = (I32)(((1<<31)-1-(x))^~(~0<<31));}} while(0)
|+# define EXTEND32(x) do { if (!natint) {(x) = (((1L<<31)-1-(x))^~(~0L<<31));}} while(0)
| #endif
| #if SIZEOF_SHORT == SIZE16
| # define EXTEND16(x)

danielb · February 16, 2006, 9:46am

Hello.

This means EXTEND32() macro in pack.c is not working on 64bit Solaris
(and perhaps on other 64bit systems neither). I have no 64bit machine
at hand. Could somebody confirm?

Me neigher. But we can use HP TestDrive

Probably this patch will solve the problem.

Index: pack.c

RCS file: /src/ruby/pack.c,v
retrieving revision 1.62.2.12
diff -u -w -b -p -r1.62.2.12 pack.c
— pack.c 13 Oct 2005 14:30:49 -0000 1.62.2.12
+++ pack.c 16 Feb 2006 06:01:07 -0000
@@ -347,11 +347,11 @@ num2i32(x)
return 0; /* not reached */
}

-#if SIZEOF_LONG == SIZE32 || SIZEOF_INT == SIZE32
+#if SIZEOF_LONG == SIZE32

define EXTEND32(x)

#else
/* invariant in modulo 1<<31 */
-# define EXTEND32(x) do {if (!natint) {(x) =
(I32)(((1<<31)-1-(x))^~(~0<<31));}} while(0)
+# define EXTEND32(x) do { if (!natint) {(x) =
(((1L<<31)-1-(x))^~(~0L<<31));}} while(0)
#endif
#if SIZEOF_SHORT == SIZE16

define EXTEND16(x)

danielb · February 16, 2006, 5:50pm

Yukihiro M. wrote:

|
| }
| # define EXTEND16(x)

Looks good. The only one I found odd was the [nil] returned by
“\377\377\377\377”.unpack(“l_”). Is that expected? The 32 bit version
returns
[-1].

64 bit Ruby

irb(main):001:0> [-1].pack(“l”)
=> “\377\377\377\377”
irb(main):002:0> “\377\377\377\377”.unpack(“l”)
=> [-1]
irb(main):003:0> “\377\377\377\377”.unpack(“l_”)
=> [nil]
irb(main):004:0> [-1].pack(“l_”)
=> “\377\377\377\377\377\377\377\377”
irb(main):005:0> “\377\377\377\377\377\377\377\377”.unpack(“l”)
=> [-1]
irb(main):006:0> “\377\377\377\377\377\377\377\377”.unpack(“l_”)
=> [-1]

Thanks,

Dan

danielb · February 16, 2006, 11:50pm

Hi,

In message “Re: pack(“l”), 64 bit question”
on Fri, 17 Feb 2006 01:47:36 +0900, Daniel B.
[email protected] writes:

|> Daniel, could you try this patch on your 64bit box?

|Looks good. The only one I found odd was the [nil] returned by
|“\377\377\377\377”.unpack(“l_”). Is that expected? The 32 bit version returns
|[-1].

No, it should return [4294967295]. How about a patch from Ville
Mattila in [ruby-talk:180126]?

						matz.

danielb · February 17, 2006, 1:48am

Hello.

|Looks good. The only one I found odd was the [nil] returned by
|"\377\377\377\377".unpack(“l_”). Is that expected? The 32 bit version returns
|[-1].

No, it should return [4294967295]. How about a patch from Ville
Mattila in [ruby-talk:180126]?

Are these also unexpected?

irb(main):009:0> [123].pack(“s”).unpack(“l”)
=> [nil]
irb(main):010:0> [123].pack(“s”).unpack(“q”)
=> [nil]
irb(main):011:0> [123].pack(“l”).unpack(“q”)
=> [nil]

danielb · February 16, 2006, 11:11am

Yukihiro M. [email protected] writes:

|
|Probably this patch will solve the problem.

Daniel, could you try this patch on your 64bit box?

The pach works for me on my opteron 64 box. Here is another.
The sun studio cc, warned
cc: Warning: -fsimple option is ignored.
“…/pack.c”, line 1954: warning: integer overflow detected: op “<<”
“…/pack.c”, line 1954: warning: initializer does not fit or is out of
range: -144115188075855872

Is the patch correct?

index: pack.c

RCS file: /src/ruby/pack.c,v
retrieving revision 1.62.2.12
@@ -1951,7 +1951,7 @@ pack_unpack(str, fmt)
case ‘w’:
{
unsigned long ul = 0;

          unsigned long ulmask = 0xfeL << ((sizeof(unsigned long)

1. - 8);

          unsigned long ulmask = 0xfeUL << ((sizeof(unsigned long)

1UL) * 8UL);

          while (len > 0 && s < send) {
              ul <<= 7;

danielb · February 17, 2006, 3:16am

Hi,

In message “Re: pack(“l”), 64 bit question”
on Fri, 17 Feb 2006 09:46:25 +0900, H.Yamamoto
[email protected] writes:

|Are these also unexpected?
|
|irb(main):009:0> [123].pack(“s”).unpack(“l”)
|=> [nil]
|irb(main):010:0> [123].pack(“s”).unpack(“q”)
|=> [nil]
|irb(main):011:0> [123].pack(“l”).unpack(“q”)
|=> [nil]

Wait, … I see. “\377\377\377\377”.unpack(“l_”) gives you nil
because it’s shorter than sizeof(long) on the platform. So they are
all OK. Thank you.

Ocean, could you commit your patch?

						matz.