Forum: Ruby-core [ruby-trunk - Feature #6083][Open] Hide a Bignum definition

C4e88907313843cf07f6d85ba8162120?d=identicon&s=25 Koichi Sasada (Guest)
on 2012-02-25 06:03
(Received via mailing list)
Issue #6083 has been reported by Koichi Sasada.

----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083

Author: Koichi Sasada
Status: Open
Priority: Normal
Assignee:
Category:
Target version: 2.0.0


Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.
9361878d459f1709feec780518946ee5?d=identicon&s=25 Yui NARUSE (Guest)
on 2012-02-25 09:18
(Received via mailing list)
Issue #6083 has been updated by Yui NARUSE.


Binary hackers can handle C structs even if it is a hidden private
struct.
So it should be enough simply to declare "struct RBignunm is not a
public API".
----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083

Author: Koichi Sasada
Status: Open
Priority: Normal
Assignee:
Category:
Target version: 2.0.0


Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.
02da662c083396641da96c1d32fc86ed?d=identicon&s=25 KOSAKI Motohiro (Guest)
on 2012-02-25 10:07
(Received via mailing list)
> Binary hackers can handle C structs even if it is a hidden private struct.
> So it should be enough simply to declare "struct RBignunm is not a public API".

I disagree. As far as we consider there is backdoor, every binary
compatibility effort are
not meaningful. But I believe it clearly has a worth.
F24ff61beb80aa5f13371aa22a35619c?d=identicon&s=25 mame (Yusuke Endoh) (Guest)
on 2012-03-29 18:17
(Received via mailing list)
Issue #6083 has been updated by mame (Yusuke Endoh).

Status changed from Open to Assigned
Assignee set to matz (Yukihiro Matsumoto)

ko1 wrote:
> I propose to move the struct RBignum definition from include/ruby/ruby.h to
bignum.c.  I believe no one use struct RBignum directly (except core).

Agreed.
I hope C API would be organized.  This can be preliminary case.


naruse wrote:
> Binary hackers can handle C structs even if it is a hidden private struct.
So it should be enough simply to declare "struct RBignunm is not a
public API".

If you think so, it could be simpler.
README.EXT has no description about RBignum.
So we don't *need* to do anything.
But we can *kindly* do something.

--
Yusuke Endoh <mame@tsg.ne.jp>
----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083#change-25392

Author: ko1 (Koichi Sasada)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.
C4e88907313843cf07f6d85ba8162120?d=identicon&s=25 ko1 (Koichi Sasada) (Guest)
on 2012-07-25 12:08
(Received via mailing list)
Issue #6083 has been updated by ko1 (Koichi Sasada).


Matz, what do you think about it?

----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083#change-28430

Author: ko1 (Koichi Sasada)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.
0ec4920185b657a03edf01fff96b4e9b?d=identicon&s=25 matz (Yukihiro Matsumoto) (Guest)
on 2012-10-30 18:07
(Received via mailing list)
Issue #6083 has been updated by matz (Yukihiro Matsumoto).

Target version changed from 2.0.0 to next minor

I am sorry but it's too late. I admit my fault.

Matz.


----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083#change-32030

Author: ko1 (Koichi Sasada)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: next minor


Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.
02da662c083396641da96c1d32fc86ed?d=identicon&s=25 KOSAKI Motohiro (Guest)
on 2012-10-31 07:31
(Received via mailing list)
> Target version changed from 2.0.0 to next minor
> I am sorry but it's too late. I admit my fault.

Matz, I don't like this decision. I believe we should choose next
_major_
or 2.0.0. minor release should not have binary incompatibility as far
as possible.
Is this typo? If no, I'd like to propose pull it back. declaration
moving is not big task.

 - kosaki
308cbef6e86dfc49cce3b2d4cf42aedc?d=identicon&s=25 SASADA Koichi (Guest)
on 2012-10-31 14:27
(Received via mailing list)
(2012/10/31 0:31), KOSAKI Motohiro wrote:
>> Target version changed from 2.0.0 to next minor
>> I am sorry but it's too late. I admit my fault.
>
> Matz, I don't like this decision. I believe we should choose next _major_
> or 2.0.0. minor release should not have binary incompatibility as far
> as possible.
> Is this typo? If no, I'd like to propose pull it back. declaration
> moving is not big task.

Maybe, we don't define what is major and what is minor.

On 2.0.x, we shouldn't change binary compatibility.
But 2.1.x or later, I think it is acceptable.
(and some people don't accept :))
02da662c083396641da96c1d32fc86ed?d=identicon&s=25 KOSAKI Motohiro (Guest)
on 2012-10-31 19:37
(Received via mailing list)
On Wed, Oct 31, 2012 at 9:26 AM, SASADA Koichi <ko1@atdot.net> wrote:
> Maybe, we don't define what is major and what is minor.
>
> On 2.0.x, we shouldn't change binary compatibility.
> But 2.1.x or later, I think it is acceptable.
> (and some people don't accept :))

Ok, please ignore my last mail then. :)
6738588a11b852833edf6aec90ef6fa3?d=identicon&s=25 Yukihiro Matsumoto (Guest)
on 2012-11-01 15:47
(Received via mailing list)
Hi,

In message "Re: [ruby-core:48640] Re: [ruby-trunk - Feature #6083] Hide
a Bignum definition"
    on Wed, 31 Oct 2012 15:31:08 +0900, KOSAKI Motohiro
<kosaki.motohiro@gmail.com> writes:

|> Target version changed from 2.0.0 to next minor
|> I am sorry but it's too late. I admit my fault.
|
|Matz, I don't like this decision. I believe we should choose next _major_
|or 2.0.0. minor release should not have binary incompatibility as far
|as possible.
|Is this typo? If no, I'd like to propose pull it back. declaration
|moving is not big task.

Hiding (or no hiding) Bignum definition would not change binary
compatibility, probably you mean source compatibility?

But keeping compatibility during the release cycle is important, I
admit.  So I changed my mind.  If there's no big opposition, I can
accept this proposal.

              matz.
0ec4920185b657a03edf01fff96b4e9b?d=identicon&s=25 matz (Yukihiro Matsumoto) (Guest)
on 2012-11-01 15:54
(Received via mailing list)
Issue #6083 has been updated by matz (Yukihiro Matsumoto).

Target version changed from next minor to 2.0.0


----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083#change-32169

Author: ko1 (Koichi Sasada)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.
F24ff61beb80aa5f13371aa22a35619c?d=identicon&s=25 mame (Yusuke Endoh) (Guest)
on 2012-11-01 17:33
(Received via mailing list)
Issue #6083 has been updated by mame (Yusuke Endoh).

Target version changed from 2.0.0 to Next Major

> If there's no big opposition, I can accept this proposal.

Sorry, but I must voice big opposition.

Unfortunately, hiding RBignum will cause significant incompatibility.
The macro RBIGNUM_BDIGITS uses RBignum internally.  Many extension
libraries actually uses RBIGNUM_BDIGITS to construct a bignum object.

So, to hide RBignum, we must carefully design a new alternative API
to construct a bignum.  It will take a time.
Worse, source compatibility will break anyway because it is almost
impossible to make it the same as the current RBIGNUM_BDIGITS.

(Consider replacing the internal representation with GMP.
RBIGNUM_BDIGITS must malloc a BDIGIT array and export GMP into the
array.  But no one will free the allocated memory.)

Thus, I'm moving this to "Next Major".  Again, I'm sorry.

--
Yusuke Endoh <mame@tsg.ne.jp>
----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083#change-32172

Author: ko1 (Koichi Sasada)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: Next Major


Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.
9d2f78236e45a335301ba1195026105d?d=identicon&s=25 shyouhei (Shyouhei Urabe) (Guest)
on 2012-11-01 18:05
(Received via mailing list)
Issue #6083 has been updated by shyouhei (Shyouhei Urabe).


+1 for next major.

No one is against this basic concept of hiding Bignums, meseems.  It's
just a bad timing for us.
----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083#change-32173

Author: ko1 (Koichi Sasada)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: Next Major


Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.
308cbef6e86dfc49cce3b2d4cf42aedc?d=identicon&s=25 SASADA Koichi (Guest)
on 2012-11-01 18:37
(Received via mailing list)
(2012/11/01 11:05), shyouhei (Shyouhei Urabe) wrote:
> +1 for next major.

+1 for next major or next minor.

We need discussion about how to break binary compatibility in another
thread :)
F24ff61beb80aa5f13371aa22a35619c?d=identicon&s=25 Yusuke Endoh (Guest)
on 2012-11-01 19:02
(Received via mailing list)
2012/11/2 SASADA Koichi <ko1@atdot.net>:
> (2012/11/01 11:05), shyouhei (Shyouhei Urabe) wrote:
>> +1 for next major.
>
> +1 for next major or next minor.
>
> We need discussion about how to break binary compatibility in another
> thread :)


Note that this issue (#6083) will break source compatibility.
B11f10c4cd9d53970e7be20caa43f940?d=identicon&s=25 Tanaka Akira (Guest)
on 2013-08-06 07:16
(Received via mailing list)
2012/2/25 Koichi Sasada <redmine@ruby-lang.org>:
> Feature #6083: Hide a Bignum definition
> https://bugs.ruby-lang.org/issues/6083
>
> Now, the struct RBignum which is a definition of Bignum in C is located in
include/ruby/ruby.h.  It means we can't change implementation of Bignum.  For
example, using GMP as Bignum representation.
>
> I propose to move the struct RBignum definition from include/ruby/ruby.h to
bignum.c.  I believe no one use struct RBignum directly (except core).
>
> It has possibility to break binary compatibility.

I like this proposal.

However I think GMP or other multiprecision arithmetic library can be
used
for slow operations such as multiplication and division
without hiding internal of struct RBignum.

The format conversion between RBignum and such library's implementation
should be O(n) (n is number of bits of the input numbers).
O(n) cost is negligible for operations slower than O(n), if input
numbers are big enough.

The conversion cost is not negligible for O(n) operations such as
bitwise and.
But I doubt how such library is faster than Ruby for such operations.
B11f10c4cd9d53970e7be20caa43f940?d=identicon&s=25 unknown (Guest)
on 2014-02-12 12:48
(Received via mailing list)
Issue #6083 has been updated by Akira Tanaka.

File hide-bignum-internal.patch added

I made a patch for this issue.

It just moves struct RBignum and related macros from include/ruby/ruby.h
to internal.h, not bignum.c.
It is because other files (such as gc.c, marshal.c, etc.) access
internal of struct RBignum.

----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083#change-45095

* Author: Koichi Sasada
* Status: Assigned
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category:
* Target version: Next Major
----------------------------------------
Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.



---Files--------------------------------
hide-bignum-internal.patch (4.66 KB)
0ec4920185b657a03edf01fff96b4e9b?d=identicon&s=25 unknown (Guest)
on 2014-02-14 08:59
(Received via mailing list)
Issue #6083 has been updated by Yukihiro Matsumoto.


Even though this breaks source compatibility, it is worth changing it
(to mark dangerous operation).
So go ahead.

Matz.


----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083#change-45142

* Author: Koichi Sasada
* Status: Assigned
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category:
* Target version: Next Major
----------------------------------------
Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.



---Files--------------------------------
hide-bignum-internal.patch (4.66 KB)
B11f10c4cd9d53970e7be20caa43f940?d=identicon&s=25 unknown (Guest)
on 2014-02-14 16:29
(Received via mailing list)
Issue #6083 has been updated by Akira Tanaka.

Status changed from Assigned to Closed
% Done changed from 0 to 100

Applied in changeset r44957.

----------
* include/ruby/ruby.h,
  internal.h,
  ext/-test-/bignum/bigzero.c: Hide a Bignum definition.
  [ruby-core:42891] [Feature #6083]

----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083#change-45164

* Author: Koichi Sasada
* Status: Closed
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category:
* Target version: Next Major
----------------------------------------
Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.



---Files--------------------------------
hide-bignum-internal.patch (4.66 KB)
38b74ccb8896076ccd0adbab2eba0809?d=identicon&s=25 Aaron Patterson (tenderlove)
on 2014-02-25 01:02
(Received via mailing list)
Issue #6083 has been updated by Aaron Patterson.


Hi,

r44957 breaks the sqlite3 gem.  I guess it uses the RBIGNUM_LEN macro:

  https://github.com/sparklemotion/sqlite3-ruby/blob...

What should we use as a replacement?  @Naruse, I think you added this
check to the sqlite3 gem:

  https://github.com/sparklemotion/sqlite3-ruby/comm...

Thanks!

----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083#change-45454

* Author: Koichi Sasada
* Status: Closed
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category:
* Target version: Next Major
----------------------------------------
Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.



---Files--------------------------------
hide-bignum-internal.patch (4.66 KB)
B11f10c4cd9d53970e7be20caa43f940?d=identicon&s=25 unknown (Guest)
on 2014-02-25 01:42
(Received via mailing list)
Issue #6083 has been updated by Akira Tanaka.


nobu wrote a patch:
https://github.com/nobu/sqlite3-ruby/compare/bignu...

I'm not sure the status to merge it to the upstream.
(I think the patch should be similified by removing code for nails !=
0.)

Note that the original condition, RBIGNUM_LEN(result) * SIZEOF_BDIGITS
<= 8, is wrong.
It tests -2**64 < x < 2**64 but it should test -2**63 <= x < 2**63 which
accepts NUM2LL.

----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083#change-45455

* Author: Koichi Sasada
* Status: Closed
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category:
* Target version: Next Major
----------------------------------------
Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.



---Files--------------------------------
hide-bignum-internal.patch (4.66 KB)
38b74ccb8896076ccd0adbab2eba0809?d=identicon&s=25 Aaron Patterson (tenderlove)
on 2014-02-25 02:40
(Received via mailing list)
Issue #6083 has been updated by Aaron Patterson.


Ah, thanks! I think I can get it merged upstream.

----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083#change-45456

* Author: Koichi Sasada
* Status: Closed
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category:
* Target version: Next Major
----------------------------------------
Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.



---Files--------------------------------
hide-bignum-internal.patch (4.66 KB)
F1d6cc2b735bfd82c8773172da2aeab9?d=identicon&s=25 Nobuyoshi Nakada (nobu)
on 2014-02-25 10:59
(Received via mailing list)
Issue #6083 has been updated by Nobuyoshi Nakada.


I've forgotten to push the latest commit, which removes unused code for
`nails != 0`.

----------------------------------------
Feature #6083: Hide a Bignum definition
https://bugs.ruby-lang.org/issues/6083#change-45464

* Author: Koichi Sasada
* Status: Closed
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category:
* Target version: Next Major
----------------------------------------
Now, the struct RBignum which is a definition of Bignum in C is located
in include/ruby/ruby.h.  It means we can't change implementation of
Bignum.  For example, using GMP as Bignum representation.

I propose to move the struct RBignum definition from include/ruby/ruby.h
to bignum.c.  I believe no one use struct RBignum directly (except
core).

It has possibility to break binary compatibility.



---Files--------------------------------
hide-bignum-internal.patch (4.66 KB)
This topic is locked and can not be replied to.