Additional requirements for a Ruby env

Hi!

I know there are Ruby libs outthere that are coming with C extensions.
To get them working you usually do a ruby setup.rb on your
environment. But, having C extensions, they will require that your ENV
has a few more things available (I assume a make, C/C++ compiler at
least).

My environment is a Win XP machine, but I am not doing anything
related to C/C++ (and I haven’t done anything for quite a long
time… so my knowledge became quite rusty about). I would like to
hear from you, more experienced rubiest, what would be the lightest
env that would allow me to use such Ruby libs (I would like to hear
more options with some pros/cons, so that I can decide which one would
better fit).

Many thanks in advance,

./alex

I maintain several libraries that require extensions, and it’s a
complete
pain in the neck to support. I can’t imagine that the requirement for a
locally-installed C compiler is anything but a barrier to adoption. I
know
some of you will come back and say “well, my extensions work just fine,”
but
that’s not my point. They’re certainly no problem for someone who knows
their way around a compiler, and I’m very grateful for the people who
have
sent me patches to get my stuff to work on obscure platforms. But how
can
you turn this into essentially a non-issue, so that it doesn’t turn away
the
many people who don’t have the patience to muck around with a compiled
extension?

This is obviously least problematical on Linux, but it can be a
significant
problem on production Linux servers and also OSX and Solaris machines,
which
often are built without compilers. Yes, I know they’re easy to add but
again, not my point. This should be a non-issue.

And let’s talk about Windows. You’d think Windows would be the easiest
since
there is one dominant compiler vendor (Microsoft) for the platform, but
no.
There are several different versions out there (VC6, VC7, VC2003 (no
longer
available but still out there), and VC2005), all with subtle
incompatibilities. You have to be very careful with debugging builds
because
if you get the wrong library-flavor at runtime, your heap allocations
don’t
work. Etc etc. Yes, you can download VC2005 express, but I really hate
asking my users to do that! For those who would respond “well, if they
want
your library badly enough, they’ll do it…,” I have to say that I’m not
happy to restrict my target audience to such a degree.

I’d like to see a framework for building and testing binary releases of
Ruby
extensions on a range of different platforms (including Unix flavors).
It
might be possible to ship binary gems with several different binaries
built
into the gem, and the gem installer detect which is the most compatible
one
on any given target machine. There already is nice stuff in extconf for
specifying required header and library dependencies. Can this be
extended in
a standard way so that a binary-gem builder can specify a range of
platforms
and have all the respective binaries included in the production gem?
Maybe
this would require some collaboration, with people making different
build
environments available to others, possibly through a secure mechanism
sitting behind a Rails app? I’d be willing to make some different Unix
and
Solaris flavors available in a build tree so that the gems hosted at
Rubyforge could be built on my hardware and the binaries loaded back to
Rubyforge for final integration by rake. And hopefully others could
contribute different platforms.

I know I have taken the OP in a completely different direction, but I
feel
that the question he asked is only an important question because of
issues
that we could address in better ways.

Guys any hints for this? Sorry for pushing it to the top, but I am
still wondering what shall I use.

once again thanks for any hints and ideas,

./alex

.w( the_mindstorm )p.

Alexandru P. wrote:

time… so my knowledge became quite rusty about). I would like to
hear from you, more experienced rubiest, what would be the lightest
env that would allow me to use such Ruby libs (I would like to hear
more options with some pros/cons, so that I can decide which one would
better fit).

the problem is that the ruby setup.rb method assumes ruby is running on
the system that built it, which if you DL’d the windows one click
installer, is 100%
false!

ruby doesnt look for just any old compiler, it looks for the one that
created it.
it is not an easy simple fix.

-stu

On Tue, Jul 18, 2006 at 12:54:22AM +0900, Alex Y. wrote:

Your options are as follows:

  • Rely on the kindness of others to provide Win32 builds of C extensions
    that you need.
  • Obtain MS VC6 (somehow) and compile them yourself.
  • Obtain MinGW GCC and friends and compile them yourself.

In the first and second case, you can use the One-Click Installer Ruby.
In the third case, you’ll either need to compile Ruby yourself, or use
the MinGW build from ruby-lang.org.

You can also use extensions compiled using mingw with mswin32 (VC6)
binaries,
like the one upon which the One Click Installer distro is built. All you
need
is an appropriate rbconfig.rb.

I’ve also had some success cross-compiling with mingw.

Mauricio F. wrote:

In the third case, you’ll either need to compile Ruby yourself, or use
the MinGW build from ruby-lang.org.

You can also use extensions compiled using mingw with mswin32 (VC6) binaries,
like the one upon which the One Click Installer distro is built. All you need
is an appropriate rbconfig.rb.
I know that’s the theory, but aren’t there a few libs that fail under
those conditions? RMagick and Ferret spring instantly to mind, but I
may be misunderstanding something here…

Alexandru P. wrote:

Guys any hints for this? Sorry for pushing it to the top, but I am
still wondering what shall I use.

once again thanks for any hints and ideas,
Your options are as follows:

  • Rely on the kindness of others to provide Win32 builds of C extensions
    that you need.
  • Obtain MS VC6 (somehow) and compile them yourself.
  • Obtain MinGW GCC and friends and compile them yourself.

In the first and second case, you can use the One-Click Installer Ruby.
In the third case, you’ll either need to compile Ruby yourself, or use
the MinGW build from ruby-lang.org.

In any of the three cases, you will be relying on the C extension in
question having been tested on Windows previously, unless you are
willing to be the test case yourself.

I don’t know about Borland’s offerings here - they may or may not be
appropriate.

My money’s on MinGW for the medium-long term. I believe (Curt? You
there?) that the OCI will be converted to a MinGW build in future.


Alex

By following the thread and considering the fact that my system relies
on the one click installer I assume my only option is having MS VC6.
Is this conclusion correct? (because I failed to understand the
reasons why other compilers are not gonna work :frowning: ).

./alex

.w( the_mindstorm )p.

On Tue, 18 Jul 2006, Austin Z. wrote:

On 7/17/06, Alexandru P. [email protected] wrote:

By following the thread and considering the fact that my system relies
on the one click installer I assume my only option is having MS VC6.
Is this conclusion correct? (because I failed to understand the
reasons why other compilers are not gonna work :frowning: ).

Binary incompatibilities from Microsoft.

(applause from audience!)

-a

On 7/17/06, Alexandru P. [email protected]
wrote:

By following the thread and considering the fact that my system relies
on the one click installer I assume my only option is having MS VC6.
Is this conclusion correct? (because I failed to understand the
reasons why other compilers are not gonna work :frowning: ).

Binary incompatibilities from Microsoft.

-austin

Sorry but I still don’t get it… why do I need to compile Ruby
myself if I intend to use a different compiler for extensions. That is
the puzzling piece.

./alex

.w( the_mindstorm )p.

Austin Z. wrote:

On 7/17/06, Alexandru P. [email protected] wrote:

By following the thread and considering the fact that my system relies
on the one click installer I assume my only option is having MS VC6.
Is this conclusion correct? (because I failed to understand the
reasons why other compilers are not gonna work :frowning: ).

Binary incompatibilities from Microsoft.

-austin

yes, but as far as i know the problem isn’t with the resulting dll’s
(not
the extension dll, nor the ruby.dll) but only with this little import
lib
that makes the ruby dll available in the extension. right so far?

If so, this problem can be solved:

http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/190272

we ‘just’ need to build win32 extensions this way (regardless of
compiler
version) and the problem goes away. (yes, this will take time to
convince
each and every maintainer of a ruby extension)

But right now this information isn’t known very well, i asked before

“please, can we have that in readme.ext ?” but got no answer.

If there is something wrong with this way to compile extensions on win32
i would like to know, if not we should really follow those instructions.

cheers

Simon

Alexandru P. wrote:

Sorry but I still don’t get it… why do I need to compile Ruby
myself if I intend to use a different compiler for extensions. That is
the puzzling piece.

./alex

Because your extension is (statically) linked against (if you use the
one-click) msvcrt-ruby18.lib (which will in turn load the dll) and the
library format does change between compilers.

  • at least that’s how i understand it.

cheers

Simon

On 7/17/06, Alex Y. [email protected] wrote:

  • Obtain MinGW GCC and friends and compile them yourself.
    appropriate.

My money’s on MinGW for the medium-long term. I believe (Curt? You
there?) that the OCI will be converted to a MinGW build in future.

No doubt, the compiler situation on Windows is a mess. There is some new
news, Austin Zeigler has been working with the Microsoft VC++ team (who
was
dismayed to learn the Ruby on Windows was compiled with VC++ 6) to
resolve
the issues with Ruby and VC2005.

I’ll probably take over working with MS on this at some point. If we can
get
this working, then
VC2005 express would definitely be the preferred solution. If that
fails,
then it will be MinGW.

Curt

Man I haven’t thought that my initial question (for which I was a
little ashamed) would bring so many things to light.

I would say that by reading this thread, I just figured out that I
will probably have, from time to time, to kindly ask experienced
users/initial developers (but this might be another problem, because
they may lack the resources too) to provide me with the Win
compatibile package compiled against a One-Click-Ruby (damn this
sounds like I will never get them).

I have some questions about last Austin’s message:

On 7/18/06, [email protected] [email protected] wrote:

On Tue, 18 Jul 2006, Curt H. wrote:
[…]
i think this is a greatly missed point. if it could be guaranteed that
any ruby could compile binary extensins for itself (because it required a
decent compiler toolchain to compile itself) then developers would be freed to
develop binary extensions that speed ruby up and know that all ruby’s could
compile them up themselves.

How this can be done? I mean, it looks like Ruby should come with a
toolset for each avail platform that would allow compilation/building
of extensions. If I get this correctly, this looks pretty impossible,
because as far as I saw, there are different persons building distros
for different platforms. Not to speak, that maybe this may raise
licensing problems. In case I got it wrongly, I apologize.

The benefits would be amazing. But, I really don’t think there is
really a solution for this “support-all-platforms” problem (and this
is a very old one).

BR,

./alex

.w( the_mindstorm )p.

Francis C. wrote:

a standard way so that a binary-gem builder can specify a range of
contribute different platforms.
This sounds like a suggestion for a Ruby extension compile farm. That’s
an idea I could seriously get behind.

On Tue, 18 Jul 2006, Curt H. wrote:

No doubt, the compiler situation on Windows is a mess. There is some new
news, Austin Zeigler has been working with the Microsoft VC++ team (who was
dismayed to learn the Ruby on Windows was compiled with VC++ 6) to resolve
the issues with Ruby and VC2005.

I’ll probably take over working with MS on this at some point. If we can get
this working, then VC2005 express would definitely be the preferred
solution. If that fails, then it will be MinGW.

why is that though? a VC2005 will still result in a broken ruby that
will be
unable to compile things like sqlite. what i mean by that it that it
will not
allow one to

  • download sqlite
  • compile it
  • download sqlite-ruby
  • compile that

which is to say that every single ruby extension that does

ruby extconf.rb && make && make install

will be unavialable to the windows ruby community.

if that is the case then people will immediately begin down the road
they’re
on now : some will compile with mingw, some with vc++ 6, etc, etc, etc
and,
whammo, we’ll be right back in the boat we’re in now - binary
imcompatibility madness.

people have to realize that, if ruby is compiled with a microsoft then
any
extension must also be compiled with vc++ and anything that compiles
against
as well! that’s an extremely steep hill to climb - for instance totally
ansi
packages like the gsl (note i said ansi, not posix!) do not compile
easily
with microsoft compilers (in fact companies charge 600$ to do it!). in
addition, 90% of the neat stuff out there like postgres, sqlite,
open-ssl -
all compile flawlessly on mingw and, therfore, allow people to compile
ruby
extensions against them. but here’s the rub: microsoft doesn’t provide
and
compiler toolchain which plays well with 90% of the popular
open-source
projects out there. it’s not the compiler that’s the toughest thing -
it’s
the lack of make, ld, ar, sh, etc that so many packages depend on that
makes a
microsoft based ruby so disappointing : it’s a ruby that cannot be
easily
extended – one of the fundemental aspects of any modern language.

i think this is a greatly missed point. if it could be guaranteed that
any ruby could compile binary extensins for itself (because it
required a
decent compiler toolchain to compile itself) then developers would be
freed to
develop binary extensions that speed ruby up and know that all ruby’s
could
compile them up themselves. think about what that might to for ruby’s
speed!
as it stand now making a binary installation that’s portable is simply
too
great a burden to expect many developers to put them selves through - we
do
this for free after all. why should tim have to figure out how to make
a
cross platform image magic installation when the build process of ruby
itself
has already done so? why should the next developer have to re-invent
the
wheel already again? what i’m saying is that the standards of
sh/configure/gcc, etc solve the bane of every binary ruby extension
developers
worst nightmare - portability - already. to not leverage this fact
is a
massive violation of dry to say the least.

in addition, having a decent environment guaranteed for every ruby opens
many,
many possibilities - imagine if this worked for any ruby

system ‘command >/dev/null 2>&1’

guess how many times that’s come up on the list :wink:

in summary, a move towards any vc product will be a move not away from
the abi
incompatibilty problem - but simply towards a different one.

hopefully i will not start any flames, but that’s my 2 cts.

-a

Timothy H. wrote:

Francis C. wrote:

a standard way so that a binary-gem builder can specify a range of
contribute different platforms.
This sounds like a suggestion for a Ruby extension compile farm. That’s
an idea I could seriously get behind.

That’s exactly what I had in mind. If the farm had at least one of every
important platform that’s out there, including all of the stupid Windows
flavors, that would make life so much easier. And I was really serious
about generating and distributing gems with multiple binaries, one of
which gets chosen and installed on each target machine. I’m willing to
contribute bandwidth and security services for an effort like this.

It’s pretty nasty for me as an extension developer to deal with all of
this garbage. But that’s nothing compared to what we are putting our
users through. This is something that really needs a solution.

And as far as production servers are concerned, even Linux ones, almost
every enterprise shop I have ever worked with requires that these
machines be built without compilers, and not only for the security risk.
You simply can’t manage application software on a few hundred machines
if each one of them has to compile its own binaries.

Ara, you make some very good points here. The O.-Click Installer is at
a
cross-roads here and I definitely want to choose the path carefully,
after
considering all valid pros and cons.

I’m going to re-post this to ruby-talk as a new thread with the subject
“One-Click Installer: MinGW? or VC2005?” and explicit ask people to give
their take on solid pros and cons for each path.

Thanks,
Curt

On 7/18/06, [email protected] [email protected] wrote:

why is that though? a VC2005 will still result in a broken ruby that will be
unable to compile things like sqlite. what i mean by that it that it will not
allow one to

No, it won’t allow. But that’s because Ruby needs to provide that
environment just as Perl has done. However, MINGW is a poor choice
because (1) it is slowly updated, (2) it conforms to a much older ABI,
(3) it doesn’t have support for newer APIs, etc., etc., etc. MINGW is
a small step above Cygwin, which is awful for Windows (30%+ slower,
IME).

in addition, having a decent environment guaranteed for every ruby opens many,
many possibilities - imagine if this worked for any ruby

system ‘command >/dev/null 2>&1’

I would oppose this. If you’re calling #system, you had best know your
platform. /dev/null is not valid – and never will be valid – on
Windows.

The #1reason to use VC++ is that it is the platform-standard compiler.

MINGW isn’t.

-austin