Forum: JRuby cext build failure

Decided to quickly checkout the cext branch on got a build 
error....likely a pilot error...suggestions?

Jon


[C:\Users\Jon\Documents\JavaDev\jruby]echo %PATH%
C:\ant\bin;C:\Program Files\CollabNet\Subversion 
Client;C:\Windows\system32;C:\W
indows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\P
rogram Files\ATI 
Technologies\ATI.ACE\Core-Static;C:\tools;C:\Python27\Scripts;C
:\Python27;C:\ruby187\bin;c:\scala\bin;C:\gnuwin32\curl\bin;C:\gnuwin32\diff\bin
;C:\gnuwin32\grep\bin;C:\gnuwin32\findutils\bin;C:\gnuwin32\sed\bin;C:\gnuwin32\
less\bin;C:\gnuwin32\upx\bin;C:\Program Files\Wix;C:\git\cmd


[C:\Users\Jon\Documents\JavaDev\jruby]echo %JAVA_HOME%
C:\Program Files\Java\jdk1.6.0_21


[C:\Users\Jon\Documents\JavaDev\jruby]git br -v
* cext   4b28b4b mark two potential segfaults
  master f740f78 Fix JRUBY-4777 - LOG_NODELAY should have been 
LOG_NDELAY.


[C:\Users\Jon\Documents\JavaDev\jruby]ant clean build-jruby-cext-native
...
jar-complete:
    [mkdir] Created dir: 
C:\Users\Jon\Documents\JavaDev\jruby\build\jar-complete\META-INF\jruby.home
     [copy] Copying 1343 files to 
C:\Users\Jon\Documents\JavaDev\jruby\build\jar-complete\META-INF\jruby.home

BUILD FAILED
C:\Users\Jon\Documents\JavaDev\jruby\build.xml:443: 
C:\Users\Jon\Documents\JavaDev\jruby\lib\native does not exist.

Total time: 33 seconds
C:\ant\bin\ant.bat [199]  Usage : COLOR [BRIght] fg ON [BRIght] bg

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Try "ant clean cext" on current cext head. I just added that as a
shorter target to run, and juggled dependencies a little bit.

The lib/native dir should be created during the normal JRuby build, by
copying the jffi-related libraries there. I deleted that dir, ran "ant
clean cext" and it appears to build ok.

- Charlie

On Tue, Aug 10, 2010 at 1:51 PM, Jon <jon.forums@gmail.com> wrote:
> ;C:\gnuwin32\grep\bin;C:\gnuwin32\findutils\bin;C:\gnuwin32\sed\bin;C:\gnuwin32\
>
> Total time: 33 seconds
> C:\ant\bin\ant.bat [199]  Usage : COLOR [BRIght] fg ON [BRIght] bg
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> Try "ant clean cext" on current cext head. I just added that as a
> shorter target to run, and juggled dependencies a little bit.
> 
> The lib/native dir should be created during the normal JRuby build, by
> copying the jffi-related libraries there. I deleted that dir, ran "ant
> clean cext" and it appears to build ok.
> 
> - Charlie

Almost...Windows doesn't like many of build.xml's <exec> and doesn't 
like the CC in Makefile.

The attached patch to build.xml got me a bit farther, but I still get 
the following build failure...barfs that it's missing sys/select.h. 
Will try to get some more time to look at it later.

...
[exec] gcc -DNDEBUG -O2 -fno-omit-frame-pointer -fno-strict-aliasing -W 
-Wall
-Wno-unused -Wno-parentheses -Werror -Wundef 
-I"/c/Users/Jon/Documents/JavaDe
v/jruby/cext/src/../..//build" 
-I"/c/Users/Jon/Documents/JavaDev/jruby/cext/src"
 -I"/c/Users/Jon/Documents/JavaDev/jruby/cext/src/../..//build"/jni 
-I"/c/Users/
Jon/Documents/JavaDev/jruby/cext/src"/include 
-I"/c/Users/Jon/Documents/JavaDev/
jruby/cext/src"/include/ruby  -I"C:\Program 
Files\Java\jdk1.6.0_21/include" -I"C
:\Program Files\Java\jdk1.6.0_21/include/win32"  -D_REENTRANT 
-D_LARGEFILE64_SOU
RCE -D_GNU_SOURCE -mwin32 -D_JNI_IMPLEMENTATION_ -c 
/c/Users/Jon/Documents/JavaD
ev/jruby/cext/src/st.c -o 
/c/Users/Jon/Documents/JavaDev/jruby/cext/src/../..//b
uild/st.o
     [exec] In file included from 
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/
include/ruby.h:2:0,
     [exec]                  from 
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/
st.c:5:
     [exec] 
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/include/ruby/ruby.h:27
:24: fatal error: sys/select.h: No such file or directory
     [exec] compilation terminated.
     [exec] make: *** 
[/c/Users/Jon/Documents/JavaDev/jruby/cext/src/../..//buil
d/st.o] Error 1

BUILD FAILED
C:\Users\Jon\Documents\JavaDev\jruby\build.xml:1520: exec returned: 2

On Thu, Aug 12, 2010 at 4:37 PM, Jon <jon.forums@gmail.com> wrote:
> Almost...Windows doesn't like many of build.xml's <exec> and doesn't like the CC in Makefile.

Ahh, I didn't even notice you were on Windows. Yeah...that's a challenge 
:)

> The attached patch to build.xml got me a bit farther, but I still get the following build failure...barfs that it's missing sys/select.h.  Will try to get some more time to look at it later.

Ok, I'll make sure Tim sees this and we'll get it incorporated into
the branch. We obviously want C exts to work on Windows like they do
anywhere else, though we'll probably build and ship a pre-made binary
for release (but of course, we have to be able to build that binary
too).

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> too).
Let me find some time to get my head around this and get you a real 
patch.

This is the known missing sys/select.h issue on mingw.  It's been such a 
long time that I've forgotten the fix but I think it's solved by 
including winsock2.h from mingw.  Have to check and experiment.  I'm 
hoping you or Tim are open to #ifdef's  or something similar :)

I agree that a pre-made binary (linked against msvcrt.dll) for release 
would be great.

I would like to get our new DevKit from RubyInstaller to be able to work 
with your build.xml to compile everything by simply doing something like 
"c:\devkit\devkitvars.bat" and ensuring "ant" and "%JAVA_HOME%\bin" are 
on the PATH.  Then Windows users can simply clone your repo, do the 
super easy install of the DevKit, and not only build JRuby w/cext from 
source, but also do things like "jruby -S gem install rdiscount 
--platform=ruby" and have things just work.  I think we're close, let's 
see :)

Roger Pack...looking for something else into for a bit? ;)

Jon


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

On 13 August 2010 11:27, Jon <jon.forums@gmail.com> wrote:
> This is the known missing sys/select.h issue on mingw.  It's been such a long time that I've forgotten the fix but I think it's solved by including winsock2.h from mingw.  Have to check and experiment.  I'm hoping you or Tim are open to #ifdef's  or something similar :)

I'd just wrap that in #ifndef _WIN32 .. #endif

It is most likely there because some extensions expect it to be
included by default.  if not, then it should only be included by the C
file that needs it.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

I've looked at how ruby core gets around the select() call but haven't 
had time to find a Windows machine I can play on. I might have one ready 
tonight (in about 10h time), but if you get a patch ready (preferrably 
git format) I'll apply it.

On Aug 13, 2010, at 3:27 AM, Jon wrote:

>> too).
> 
> Jon
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
> 
>    http://xircles.codehaus.org/manage_email
> 
> 


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> I've looked at how ruby core gets around the select() call but haven't had time to find a Windows machine I can play on. I might have one ready tonight (in about 10h time), but if you get a patch ready (preferrably git format) I'll apply it.

This from ruby core seems to address regular internet sockets

http://github.com/ruby/ruby/blob/trunk/include/ruby/win32.h#L239
http://github.com/ruby/ruby/blob/trunk/include/ruby/win32.h#L554

...but the following from cext makes me wonder if the select used by 
cext's threads use *nix domain sockets to communicate?  If that's true, 
I'm fairly sure that simply #ifndef _WIN32 ... replacing sys/select.h 
with winsock2.h won't fix things and I think you'd have to use something 
like Windows named pipes instead.

http://github.com/jruby/jruby/blob/cext/cext/src/thread.cpp#L19
http://github.com/jruby/jruby/blob/cext/cext/src/thread.cpp#L67

That said, I'm not familiar with how cext's threads behave and 
unfortunately I've run out of time today to look into it more...not 
going to be able to get you a patch for testing tonight :(

Jon

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Well, the select is basically what MRI does for unix systems. I have 
only used this code for now as I didn't have a Windows machine ready 
(not even a VM, it has been a few years since I used Windows, well 
before Vista and friends). They wrap their code in
  #if defined(__CYGWIN__) || defined(_WIN32)
    30 lines of windows code here
  #else
    select in a blocking region
  #endif
and I haven't had time to understand the threading code completely, yet 
(it's one of the last things that is not passing the capi specs, I'm 
down to  8 failures, 2 errors).
I've found a Windows machine at Uni now, which I can use, so I'll look 
into that in the next few hours.

-Tim

On Aug 13, 2010, at 4:49 PM, Jon wrote:

> http://github.com/jruby/jruby/blob/cext/cext/src/thread.cpp#L67
> 
---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Hi

On Aug 13, 2010, at 4:49 PM, Jon wrote:
> This from ruby core seems to address regular internet sockets
> 
> http://github.com/ruby/ruby/blob/trunk/include/ruby/win32.h#L239
> http://github.com/ruby/ruby/blob/trunk/include/ruby/win32.h#L554

I was talking more about this:

http://github.com/ruby/ruby/blob/trunk/thread.c#L2475 .. 
http://github.com/ruby/ruby/blob/trunk/thread.c#L2509

> That said, I'm not familiar with how cext's threads behave and unfortunately I've run out of time today to look into it more...not going to be able to get you a patch for testing tonight :(

Well, I have installed your devkit on a windows machine (btw: is there 
some proper terminal application for windows? The default cmd.exe is 
quite difficult to work with, so to speak) and I have created a few 
patches that, not very nicely, made everything at least compile on my 
the windows machine. I'm still getting a linker error, I'll look into 
this if I can get a remote desktop connection to the windows PC in the 
university network - working for the administrators in my first year was 
a good way to get access to our gateways ;)

-Tim


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> Well, I have installed your devkit on a windows machine (btw: is there some proper terminal application for windows? The default cmd.exe is quite difficult to work with, so to speak) and I have created a few patches that, not very nicely, made everything at least compile on my the windows machine. I'm still getting a linker error, I'll look into this if I can get a remote desktop connection to the windows PC in the university network - working for the administrators in my first year was a good way to get access to our gateways ;)

That's great news!  I'll do a pull.

Now that it compiled, I'm going to be _very_ interested in running the 
<DEVKIT_INSTALL>\dk.rb install script and see if things like "gem 
install rdiscount --platform=ruby" allow the RDiscount native gem to be 
compiled and used with JRuby.

re: cmd.exe replacement, you'll likely creating a windows desktop 
shortcut to <DEVKIT_INSTALL>\msys.bat which will give you a bash-like 
shell experience in that you can use the cmds in <DEVKIT_INSTALL>\bin 
like ls, cp, rm, grep, etc.

Initially the bash-like shell will look horrible but if you left click 
on the upper left icon in the title bar and select "Properties", you can 
make things look better by tweaking the "Font" tab (use TrueType) and 
the "Layout" tag (tweak the Screen Buffer sizes).  You can also edit 
<DEVKIT_INSTALL>\etc\profile (eg - call a ~/.bashrc) and add a 
<DEVKIT_INSTALL>\home\<USER>\.bashrc.

Sounds like your time with the sysadmins was time well spent :)

Jon



FYI - As soon as I get time to update our wiki [1] with usage info, I'm 
going to release the updated DevKit with tweaks to the dk.rb install 
script to make it more user friendly.  Nothing else changed so you 
should be fine using the version you have.  However, if you want to use 
the new version feel free to download it from [2]  In lieu of proper 
documentation for using the dk.rb installer helper script check out [3] 
and the posts near the end.

[1] http://wiki.github.com/oneclick/rubyinstaller/development-kit
[2] http://rubyinstallerdev.heroku.com/downloads/
[3] 
http://groups.google.com/group/rubyinstaller/browse_thread/thread/d9c226735a54679f

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

I forgot to mention a very useful tool for your newly found windows 
machine...btw, which OS version?

It's called 'pik' and it's very similar to 'rvm' on *nix 
http://github.com/vertiginous/pik

I use it all the time to test things out between different Ruby 
implementations...

[C:\Users\Jon\Documents\RubyDev]pik ls
  110: IronRuby 1.1.0.0 on .NET 4.0.30319.1
* 151: jruby 1.5.1 (ruby 1.8.7 patchlevel 249) (2010-06-06 f3a3480) 
(Java H...
  187: ruby 1.8.7 (2010-06-23 patchlevel 299) [i386-mingw32]
  191: ruby 1.9.1p429 (2010-07-02 revision 28523) [i386-mingw32]
  192: ruby 1.9.2dev (2010-08-09 revision 28936) [i386-mingw32]
  193: ruby 1.9.3dev (2010-08-13 trunk 28974) [i386-mingw32]

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

This is a Windows 7 machine. I've gotten the library to compile, but it 
doesn't want to compile the extensions - undefined references.
I'm using the following commands and output:

> cc  -IC:/Users/tim.felgentreff/Desktop/devkit/rubyspec/optional/capi/ext -IC:/Users/tim.felgentreff/Desktop/devkit/jruby/lib/native/include/ruby  -fno-omit-frame-pointer -fno-strict-aliasing  -fexceptions -fPIC -c C:/Users/tim.felgentreff/Desktop/devkit/rubyspec/optional/capi/ext/array_spec.c -o C:/Users/tim.felgentreff/Desktop/devkit/rubyspec/optional/capi/ext/array_spec.o"

C:/Users/tim.felgentreff/Desktop/devkit/rubyspec/optional/capi/ext/array_spec.c:1:0: 
warning: -fPIC ignored for target (all code is position independent)

> cc -shared  C:/Users/tim.felgentreff/Desktop/devkit/rubyspec/optional/capi/ext/array_spec.o -LC:/Users/tim.felgentreff/Desktop/devkit/rubyspec/optional/capi/ext   -o C:/Users/tim.felgentreff/Desktop/devkit/rubyspec/optional/capi/ext/array_spec.dll"

C:/Users/tim.felgentreff/Desktop/devkit/rubyspec/optional/capi/ext/array_spec.o: 
array_spec.c:(.text+0x1e): undefined reference to `rb_num2long'
  ... many more undefined references follow
collect2: ld returned 1 exit status

Under linux I'm using the same compile and link flags, so I'm not sure 
what's wrong here. I vaguely remember that "-shared" didn't work under 
mingw as it did with linux. I remember that I had to compile e.g. E17 
completely statically under Windows. Cannot remember why, though. Any 
thoughts?

Oh, and could you add a "cc" link to gcc in your devkit?

-Tim

On Aug 13, 2010, at 11:16 PM, Jon wrote:

>  191: ruby 1.9.1p429 (2010-07-02 revision 28523) [i386-mingw32]
>  192: ruby 1.9.2dev (2010-08-09 revision 28936) [i386-mingw32]
>  193: ruby 1.9.3dev (2010-08-13 trunk 28974) [i386-mingw32]
> 
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
> 
>    http://xircles.codehaus.org/manage_email
> 
> 


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> This is a Windows 7 machine. I've gotten the library to compile, but it doesn't want to compile the extensions - undefined references.

Unfortunately I couldn't get it compile on my Win7 Ultimate 32-bit 
machine.  The attached patch allows me to build and part of the problem 
was the build.xml patch I sent you caused the new make.bat to fail 
because it didn't handle "make CC=gcc" correctly.  I'm not sure all the 
mods I made to Makefile are needed, but we can tune later. Does this 
patch work for you when running "ant clean cext"?


> Oh, and could you add a "cc" link to gcc in your devkit?

I've added the following to my C:\DevKit\mingw\bin\cc.bat file
@ECHO OFF
gcc.exe %*

I think this is fine and needs to go into the release version of the new 
DevKit, but I want a couple of the other RubyInstaller contributors to 
double check.  Good catch, thanks.

Even with all of the above, I'm still not able to install the native 
rdiscount gem and it looks like there's some mkmf.rb work still left to 
do.  The good news is that the gem pre_install hook seems to be working 
fine on JRuby like we've seen on other impl's....making progress :)

C:\Users\Jon\Documents\CDev\rdiscount>jruby -S gem install 
rdiscount-1.6.5.jm1.gem
JRuby limited openssl loaded. http://jruby.org/openssl gem install 
jruby-openssl for full support.
Temporarily enhancing PATH to include DevKit...
Building native extensions.  This could take a while...
ERROR:  Error installing rdiscount-1.6.5.jm1.gem:
        ERROR: Failed to build gem native extension.

C:/Users/Jon/Documents/JavaDev/jruby/bin/jruby.exe extconf.rb
WARNING: JRuby does not support native extensions or the `mkmf' library 
very well.
Check http://kenai.com/projects/jruby/pages/Home for alternatives.
C:/Users/Jon/Documents/JavaDev/jruby/lib/ruby/1.8/mkmf.rb:31 warning: 
already initialized constant RUBY_PLATFORM
checking for random()... no
checking for srandom()... no
checking for rand()...
make
make: *** No rule to make target `ruby.h', needed by `basename.o'. 
Stop.

Jon

Hi

On Aug 14, 2010, at 4:54 AM, Jon wrote:

>> This is a Windows 7 machine. I've gotten the library to compile, but it doesn't want to compile the extensions - undefined references.
> 
> Unfortunately I couldn't get it compile on my Win7 Ultimate 32-bit machine.  The attached patch allows me to build and part of the problem was the build.xml patch I sent you caused the new make.bat to fail because it didn't handle "make CC=gcc" correctly.  I'm not sure all the mods I made to Makefile are needed, but we can tune later. Does this patch work for you when running "ant clean cext"?

I don't use the "ant cext" target because I couldn't get both javac and 
devkit to run in the same shell on windows, so I'm doing ant jar in one 
shell and a direct make in the cext/src in another.

I tried it with deactivating the dependency of cext on the jar, and that 
built the cext code fine, so yes, it kinda works.

> Even with all of the above, I'm still not able to install the native rdiscount gem and it looks like there's some mkmf.rb work still left to do.  The good news is that the gem pre_install hook seems to be working fine on JRuby like we've seen on other impl's....making progress :)

The generated Makefile is not really different from the one MRI 
generates, save for the include paths, and the same as on unix.
I checked the include flags, they look fine, for some reason the 
dependency on ruby.h and defines.h is not checked against those paths on 
windows. Ah well, I removed the dependencies, we don't really need them, 
anyway, I just wanted to keep the diff against MRI's mkmf small.

> C:\Users\Jon\Documents\CDev\rdiscount>jruby -S gem install rdiscount-1.6.5.jm1.gem

Doing a "jruby -S gem install rdiscount" now gives me the same linker 
error as the capi specs.

C:\Users\tim.felgentreff\Desktop\devkit\jruby\cext\src>jruby -S gem 
install rdiscount
JRuby limited openssl loaded. http://jruby.org/openssl
gem install jruby-openssl for full support.
Temporarily enhancing PATH to include DevKit...
Building native extensions.  This could take a while...
ERROR:  Error installing rdiscount:
        ERROR: Failed to build gem native extension.

C:/Users/tim.felgentreff/Desktop/devkit/jruby/bin/jruby.exe extconf.rb
WARNING: JRuby does not support native extensions or the `mkmf' library 
very well.
Check http://kenai.com/projects/jruby/pages/Home for alternatives.
C:/Users/tim.felgentreff/Desktop/devkit/jruby/lib/ruby/1.8/mkmf.rb:31 
warning: already initialized constant RUBY_PLATFORM
checking for random()... no
checking for srandom()... no
checking for rand()...
make
cc  -I. -I. 
-IC:/Users/tim.felgentreff/Desktop/devkit/jruby/lib/native/include/ruby 
-I. -DHAVE_RAND -DHAVE_SRAND  -fno-omit-frame-pointer 
-fno-strict-aliasing -fexceptions  -m32 -c basename.c
cc  -I. -I. 
-IC:/Users/tim.felgentreff/Desktop/devkit/jruby/lib/native/include/ruby 
-I. -DHAVE_RAND -DHAVE_SRAND  -fno-omit-frame-pointer 
-fno-strict-aliasing -fexceptions  -m32 -c Csio.c
  ..... more compilation
cc -shared  -o rdiscount.dll basename.o Csio.o css.o docheader.o 
dumptree.o emmatch.o generate.o html5.o markdown.o mkdio.o rdiscount.o 
resource.o tags.o toc.o xml.o -L"." 
-L"/Users/tim.felgentreff/Desktop/devkit/jruby/lib"   -m32
rdiscount.o:rdiscount.c:(.text+0x28): undefined reference to 
`rb_intern2'
rdiscount.o:rdiscount.c:(.text+0x49): undefined reference to 
`rb_funcall'
rdiscount.o:rdiscount.c:(.text+0x58): undefined reference to 
`rb_str_buf_new'
  ..... more undefined references

So the question is, what do I need to tell mingw's gcc to link those 
undefined references dynamically?

-Tim


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

On 14 August 2010 20:36, Tim Felgentreff <tim@nada1.de> wrote:
> cc -shared  -o rdiscount.dll basename.o Csio.o css.o docheader.o dumptree.o emmatch.o generate.o html5.o markdown.o mkdio.o rdiscount.o resource.o tags.o toc.o xml.o -L"." -L"/Users/tim.felgentreff/Desktop/devkit/jruby/lib"   -m32
> rdiscount.o:rdiscount.c:(.text+0x28): undefined reference to `rb_intern2'
> rdiscount.o:rdiscount.c:(.text+0x49): undefined reference to `rb_funcall'
> rdiscount.o:rdiscount.c:(.text+0x58): undefined reference to `rb_str_buf_new'
>        ..... more undefined references
>
> So the question is, what do I need to tell mingw's gcc to link those undefined references dynamically?

jruby-cext.dll should work, assuming it will be found at load time.
We could (maybe should?) do this for unixen as well ... again,
assuming it is in an appropriate place to be located at runtime.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Hi

> jruby-cext.dll should work, assuming it will be found at load time.

I can't tell it to just -ljruby-cext, complains that it cannot find it 
(even though it is in -L path).

> We could (maybe should?) do this for unixen as well ... again,
> assuming it is in an appropriate place to be located at runtime.

Well, the logic is currently that the jruby-cext library, if it is 
within the Jar and thus cannot be loaded by the system directly, is 
extracted to java.io.tmpdir, which can be set via VM parameter and is 
something in /tmp per default. That's what QtJambi does. Is that 
inappropriate?

-Tim
---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Before I head out of town for the weekend, a few quick thoughts...

> > Unfortunately I couldn't get it compile on my Win7 Ultimate 32-bit machine.  The attached patch allows me to build and part of the problem was the build.xml patch I sent you caused the new make.bat to fail because it didn't handle "make CC=gcc" correctly.  I'm not sure all the mods I made to Makefile are needed, but we can tune later. Does this patch work for you when running "ant clean cext"?
> 
> I don't use the "ant cext" target because I couldn't get both javac and devkit to run in the same shell on windows, so I'm doing ant jar in one shell and a direct make in the cext/src in another.

That's ugly and unusable.  I'll add a <DEVKIT_HOME>\devkitvars.sh file 
similiar to purpose of the devkitvars.bat file.  The DevKit _must_ be 
able to be used from either a clean cmd.exe shell or an 
MSYS-bash-using-cmd shell.


> rdiscount.o:rdiscount.c:(.text+0x28): undefined reference to `rb_intern2'
> rdiscount.o:rdiscount.c:(.text+0x49): undefined reference to `rb_funcall'
> rdiscount.o:rdiscount.c:(.text+0x58): undefined reference to `rb_str_buf_new'
>   ..... more undefined references
> 
> So the question is, what do I need to tell mingw's gcc to link those undefined references dynamically?

Looks like library load path issues not being able to find the import 
library or the dll.  Where do these symbols live, a ruby core or cext 
dll/so?

One of the cool things with mingw is that you can link directly with the 
dll and not just the import library.  No time right now to test, but 
check out http://sourceware.org/binutils/docs/ld/WIN32.html#WIN32 which 
may have some info to help.

FYI, when building libs to run with MRI we need to link everything 
against msvcrt.dll and ensure the correct msvcrt-rubyXYZ.dll to prevent 
segfaults, i.e. - extensions running on MRI 1.8 must link against the 
ruby 1.8 and 1.9 against the ruby 1.9.  Don't know if these are issues 
with cext, but I definitely think the 
link-against-msvcrt-for-all-built-extensions is the same.

Regarding the CC issue, adding an cc.bat and cc.sh is going to cause us 
issues for the RubyInstaller project.  Instead I'm going to update the 
operating_system.rb file so that the pre_install hook set's this up 
correctly.  That in combination with the mods to build.xml should fix 
everything without having to resort to cc.bat/cc.sh stubs...can always 
do make CC=gcc when the running from the command line.  Do you see any 
issues with this approach?

Great stuff, good luck, and I'll be back online Monday latest.

Jon

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> > jruby-cext.dll should work, assuming it will be found at load time.
> 
> I can't tell it to just -ljruby-cext, complains that it cannot find it (even though it is in -L path).

I can't duplicate this.  Editing rdiscount's Makefile to the following, 
jruby-cext.dll was found

ldflags  = 
-LC:/Users/Jon/Documents/JavaDev/jruby/lib/native/i386-Windows 
-ljruby-cext


Except for "jruby_str_length" and "jruby_str_cstr" all the other 
undefinded reference errors for rdiscount.o went away when I forced 
linkage against both c:\ruby187\bin\msvcrt-ruby18.dll and 
c:\ruby191\bin\msvcrt-ruby191.dll  I'm sure this isn't how cext is 
supposed to work, but it tells me we're missing a link against the 
equivalent JRuby artifact exporting all those symbols.  Using -lffi-1.0 
and -ljruby (in <JRUBY_ROOT>/bin) didn't do it for me and I've not found 
the right artifact yet.

To confirm, JRuby doesn't have an rbconfig.rb file but generates the 
RbConfig::CONFIG hash from src/org/jruby/libraries/RbConfigLibrary.java 
and whatever mods we end up with will likely require changes there and 
cext/src/Makefile?

Jon

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

On 16 August 2010 06:24, Jon <jon.forums@gmail.com> wrote:

> went away when I forced linkage against both c:\ruby187\bin\msvcrt-ruby18.dll and c:\ruby191\bin\msvcrt-ruby191.dll  I'm sure this isn't how cext is supposed to work, but it tells me we're missing a link against the

Linking against anything from MRI is really, really bad.  jruby-cext
provides the jruby implemention of those symbols, and mixing the two
will make jruby asplode.  The symbols might not be exported correctly
for windows, when jruby-cext.dll is built, it probably needs an
additional linker flag to export all symbols.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> > went away when I forced linkage against both c:\ruby187\bin\msvcrt-ruby18.dll and c:\ruby191\bin\msvcrt-ruby191.dll  I'm sure this isn't how cext is supposed to work, but it tells me we're missing a link against the
> 
> Linking against anything from MRI is really, really bad.  jruby-cext
> provides the jruby implemention of those symbols, and mixing the two
> will make jruby asplode.

Yikes, probably just as fun as mixing MRI 1.8 extensions with 1.9 MRI 
core.  My point was that the link worked better (obviously) when the 
extension found the symbols it wanted.  Too bad it came across as 
suggesting link-cext-against-MRI-artifacts as the way to go.


> The symbols might not be exported correctly
> for windows, when jruby-cext.dll is built, it probably needs an
> additional linker flag to export all symbols.

Agreed.  Using the graphical http://www.dependencywalker.com/ shows 
jruby-cext.dll only exporting symbols (ordinals 1-40) like "JNI_OnLoad" 
and "Java_org_jruby_cext_Native_newRString" and 
"Java_org_jruby_cext_Native_getTrue" but none of the jruby 
implementations of the MRI symbols.

According to http://sourceware.org/binutils/docs/ld/WIN32.html#WIN32 the 
--export-all-symbols is ld's default except if you (a) provide a DEF, or 
(b) any symbol in jruby-cext marked with __declspec(dllexport) 
attribute. Haven't double checked...currently only have time to give to 
cleaning up the DevKi to work with JRuby before officially releasing it. 
Need to drop something else as jruby-cext is a great idea.

From what you've seen so far do you agree that RbConfigLibrary.java will 
likely need to be updated as well as cext's Makefile?

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

On 16 August 2010 08:39, Jon <jon.forums@gmail.com> wrote:
>> The symbols might not be exported correctly
>> for windows, when jruby-cext.dll is built, it probably needs an
>> additional linker flag to export all symbols.
>
> Agreed.  Using the graphical http://www.dependencywalker.com/ shows jruby-cext.dll only exporting symbols (ordinals 1-40) like "JNI_OnLoad" and "Java_org_jruby_cext_Native_newRString" and "Java_org_jruby_cext_Native_getTrue" but none of the jruby implementations of the MRI symbols.

Ok, that gives me a clue.  All of those use JNIEXPORT in their
declaration+definition, which most likely explicitly exports those
symbols - a quick&dirty fix would be to do the same for all the rb_*
and jruby_* functions we export - prefix at least the definition's
with JNIEXPORT and see if that works.

I think you can also have mingw's linker read a list of symbols to
export from a file, which may be another alternative.

>
> From what you've seen so far do you agree that RbConfigLibrary.java will likely need to be updated as well as cext's Makefile?

Sounds like it.  I don't think tweaking those specifically for
win32/64 will be a problem.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> > From what you've seen so far do you agree that RbConfigLibrary.java will likely need to be updated as well as cext's Makefile?
> 
> Sounds like it.  I don't think tweaking those specifically for
> win32/64 will be a problem.

You may not need to prefix with JNIEXPORT as an explicit 
--export-all-symbols in the cext Makefile fixed a lot of things.

We're sooo close....the attached patch (needs scrubbed) allows me to 
build jruby_cext.dll and rdiscount in a normal windows cmd.exe shell 
(not the MSYS shell as it appears my build.xml edits cause problems) 
with a C:\DevKit\cc.bat

...but I get a NoMethodError when trying to use rdiscount :(


C:\Users\Jon\Documents\JavaDev\jruby>ant clean cext

C:\Users\Jon\Documents\CDev\rdiscount>jruby -S gem install 
rdiscount-1.6.5.jm1.gem
JRuby limited openssl loaded. http://jruby.org/openssl
gem install jruby-openssl for full support.
Temporarily enhancing PATH to include DevKit...
Building native extensions.  This could take a while...
Successfully installed rdiscount-1.6.5.jm1
1 gem installed

C:\Users\Jon\Documents\CDev\rdiscount>jruby -rubygems -e "require 
'rdiscount'; puts RDiscount.new('**hello**').to_html"
-e:1: undefined method `to_html' for #<RDiscount:0x37165f 
@text="**hello**"> (NoMethodError)

On 16 August 2010 12:30, Jon <jon.forums@gmail.com> wrote:

> Successfully installed rdiscount-1.6.5.jm1
> 1 gem installed
>
> C:\Users\Jon\Documents\CDev\rdiscount>jruby -rubygems -e "require 'rdiscount'; puts RDiscount.new('**hello**').to_html"
> -e:1: undefined method `to_html' for #<RDiscount:0x37165f @text="**hello**"> (NoMethodError)


Hmmm.  Unless tim has removed the debug, there should be a line like
"calling init(0x12345678)" output to stdout or stderr.  this could
indicate that the native module is not getting loaded.

It might be an idea to try some of the example modules in jruby
(wmeissner-jruby-cext/examples I think, if they're still there).  They
have an alternative, simpler build system, which might make debugging
the init/loading sequence a bit easier.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Hi

On Aug 16, 2010, at 6:01 AM, Wayne Meissner wrote:

>> Temporarily enhancing PATH to include DevKit...
> indicate that the native module is not getting loaded.
It is not removed, rdiscount master does not currently load on JRuby, 
see timfel/rdiscount for a patch (I've already sent a pull request to 
Ryan).
It seems that MRI loads "rdiscount.bundle" (on Mac) even when 'require 
"rdiscount.so"' is in the code. We might have to mirror this behavior, 
as I personally don't like just forcing an .so extension on the cext 
libraries (as in Jon's patch).

>> From what you've seen so far do you agree that RbConfigLibrary.java will likely need to be updated as well as cext's Makefile?
> 
> Sounds like it.  I don't think tweaking those specifically for
> win32/64 will be a problem.


I will apply parts of your patch today.

-Tim


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> > Hmmm.  Unless tim has removed the debug, there should be a line like
> > "calling init(0x12345678)" output to stdout or stderr.  this could
> > indicate that the native module is not getting loaded.
> 
> It is not removed, rdiscount master does not currently load on JRuby, see timfel/rdiscount for a patch (I've already sent a pull request to Ryan).
> It seems that MRI loads "rdiscount.bundle" (on Mac) even when 'require "rdiscount.so"' is in the code. We might have to mirror this behavior, as I personally don't like just forcing an .so extension on the cext libraries (as in Jon's patch).

Does the patch incorrectly determine the platform type before deciding 
on an extension?  My intention was to change from .dll to .so only on 
windows and not touch the Mac stuff.  And the .so change on windows was 
because I _thought_ I remembered MRI 1.9 needed it's extensions named 
.so...this may be completely wrong _and_ irrelevant to JRuby.


> I will apply parts of your patch today.

After seeing that jruby-cext.dll exports 652 symbols of which ~45% start 
with either rb, ruby, or jruby I think that --export-all-symbols gives 
too much surface area and Wayne's initial idea of explicitly exporting 
symbols is the way to go.

FYI, I'm moving back to focusing on polishing thingw to work better with 
the DevKit as I want to release the new DevKit ASAP _and_ I want it to 
have JRuby support.  Specifically, (a) remove the need for cc.bat and 
cc.sh stubs as CC should be overridden by operating_system.rb and 
devkit.rb similar to how PATH is temporarily updated, and (b) cleanup 
build.xml so that it works in both standard cmd.exe and MSYS shells. 
That said, don't hesitate to ping me if you'd like another set of 
testing eyes on Win7.

Jon

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> > C:\Users\Jon\Documents\CDev\rdiscount>jruby -rubygems -e "require 'rdiscount'; puts RDiscount.new('**hello**').to_html"
> > -e:1: undefined method `to_html' for #<RDiscount:0x37165f @text="**hello**"> (NoMethodError)
> 
> 
> Hmmm.  Unless tim has removed the debug, there should be a line like
> "calling init(0x12345678)" output to stdout or stderr.  this could
> indicate that the native module is not getting loaded.

Would the successfull rdiscount require and the RDiscount.new indicate 
the native module is getting loaded?  The error message makes it look 
like the instance is being created but then something is going wrong in 
the method lookup logic.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

On Aug 16, 2010, at 3:20 PM, Jon wrote:

> 
>>> C:\Users\Jon\Documents\CDev\rdiscount>jruby -rubygems -e "require 'rdiscount'; puts RDiscount.new('**hello**').to_html"
>>> -e:1: undefined method `to_html' for #<RDiscount:0x37165f @text="**hello**"> (NoMethodError)
>> 
>> 
>> Hmmm.  Unless tim has removed the debug, there should be a line like
>> "calling init(0x12345678)" output to stdout or stderr.  this could
>> indicate that the native module is not getting loaded.
> 
> Would the successfull rdiscount require and the RDiscount.new indicate the native module is getting loaded?  The error message makes it look like the instance is being created but then something is going wrong in the method lookup logic.

No, the native module in RDiscount is loaded after the RDiscount class 
is defined in Ruby, the native codes basically only adds the to_html 
method. If that isn't there, the native extension hasn't been loaded.
---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Just a short note, gotta catch some sleep:
I pushed parts of Jon's patch today and added explicit dllexports for 
the build and defined dllimports in the ruby.h for extensions using it. 
I was then able to completely build cext support and run Thin on my 
Windows machine here. I haven't gotten the specs to work, but I already 
know the fix for that, just didn't have the time, yet.
Jon, could you test? I have e.g. native mongrel and rfuzz with passing 
specs here, so those should definitely run. You could also try curb, 
eventmachine, racc, ruby-pg which I have mostly working here (mostly 
some spec failures here and there)
-Tim


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> I pushed parts of Jon's patch today and added explicit dllexports for the build and defined dllimports in the ruby.h for extensions using it. I was then able to completely build cext support and run Thin on my Windows machine here. I haven't gotten the specs to work, but I already know the fix for that, just didn't have the time, yet.

cext built without error and exports 331 symbols rather than 652 with 
--export-all-symbols :)


> Jon, could you test? I have e.g. native mongrel and rfuzz with passing specs here, so those should definitely run. You could also try curb, eventmachine, racc, ruby-pg which I have mostly working here (mostly some spec failures here and there)
> -Tim

* RedCloth fails due to missing rb_obj_clone symbol
* EventMachine fails due to unrecognized -EHs and -GR options
* should -fexceptions be removed from RbConfigLibrary.java?
* attached patch removes -mimpure-text (Solaris only, now Win32?) and 
makes CC usage a bit more flexible.
* FYI, I still haven't found a way to remove cc.bat and cc (shell 
script) from DevKit so that the Makefile's built from 
RbConfigLibrary.java can have their "CC=cc" overridden.


=== FAILURE DETAILS ===

C:\Users\Jon\Documents\CDev\rdiscount>gem install RedCloth 
--platform=ruby
JRuby limited openssl loaded. http://jruby.org/openssl
gem install jruby-openssl for full support.
Temporarily enhancing PATH to include DevKit...
Building native extensions.  This could take a while...
ERROR:  Error installing RedCloth:
        ERROR: Failed to build gem native extension.

C:/Users/Jon/Documents/JavaDev/jruby/bin/jruby.exe extconf.rb
WARNING: JRuby does not support native extensions or the `mkmf' library 
very well.
Check http://kenai.com/projects/jruby/pages/Home for alternatives.
C:/Users/Jon/Documents/JavaDev/jruby/lib/ruby/1.8/mkmf.rb:31 warning: 
already initiali
zed constant RUBY_PLATFORM
creating Makefile

make
cc  -I. -IC:/Users/Jon/Documents/JavaDev/jruby/lib/native/include 
-IC:/Users/Jon/Docum
ents/JavaDev/jruby/lib/native/include/ruby -I.  -fno-omit-frame-pointer 
-fno-strict-al
iasing  -fexceptions -O2   -m32 -march=native -mtune=native -c 
redcloth_attributes.c
cc  -I. -IC:/Users/Jon/Documents/JavaDev/jruby/lib/native/include 
-IC:/Users/Jon/Docum
ents/JavaDev/jruby/lib/native/include/ruby -I.  -fno-omit-frame-pointer 
-fno-strict-al
iasing  -fexceptions -O2   -m32 -march=native -mtune=native -c 
redcloth_inline.c
cc  -I. -IC:/Users/Jon/Documents/JavaDev/jruby/lib/native/include 
-IC:/Users/Jon/Docum
ents/JavaDev/jruby/lib/native/include/ruby -I.  -fno-omit-frame-pointer 
-fno-strict-al
iasing  -fexceptions -O2   -m32 -march=native -mtune=native -c 
redcloth_scan.c
cc  -shared  -o redcloth_scan.dll redcloth_attributes.o 
redcloth_inline.o redcloth_sca
n.o -L"." -L"/Users/Jon/Documents/JavaDev/jruby/lib" 
-LC:/Users/Jon/Documents/JavaDev/
jruby/lib/native/i386-Windows -ljruby-cext 
-Wl,--enable-auto-image-base,--enable-auto-
import -m32 -march=native -mtune=native
redcloth_scan.o:redcloth_scan.c:(.text+0xa33b): undefined reference to 
`rb_obj_clone'
collect2: ld returned 1 exit status
make: *** [redcloth_scan.dll] Error 1


C:\Users\Jon\Documents\CDev\rdiscount>gem install eventmachine 
--platform=ruby
JRuby limited openssl loaded. http://jruby.org/openssl
gem install jruby-openssl for full support.
Temporarily enhancing PATH to include DevKit...
Building native extensions.  This could take a while...
ERROR:  Error installing eventmachine:
        ERROR: Failed to build gem native extension.

C:/Users/Jon/Documents/JavaDev/jruby/bin/jruby.exe extconf.rb
WARNING: JRuby does not support native extensions or the `mkmf' library 
very well.
Check http://kenai.com/projects/jruby/pages/Home for alternatives.
C:/Users/Jon/Documents/JavaDev/jruby/lib/ruby/1.8/mkmf.rb:31 warning: 
already initiali
zed constant RUBY_PLATFORM
checking for rb_trap_immediate in ruby.h,rubysig.h... no
checking for rb_thread_blocking_region()... conftest.c:2:0: fatal error: 
when writing
output to : Broken pipe
compilation terminated.
checking for inotify_init() in sys/inotify.h... no
checking for __NR_inotify_init in sys/syscall.h... no
checking for writev() in sys/uio.h... no
checking for rb_thread_check_ints()... no
checking for rb_time_new()... conftest.c:2:0: fatal error: when writing 
output to : In
valid argument
compilation terminated.
checking for windows.h... yes
checking for winsock.h... yes
checking for main() in -lkernel32... yes
checking for main() in -lrpcrt4... yes
checking for main() in -lgdi32... yes
checking for main() in -lssleay32... no
creating Makefile

make
g++ -I. -I. 
-IC:/Users/Jon/Documents/JavaDev/jruby/lib/native/include/ruby -I. 
-DBUILD
_FOR_RUBY -DHAVE_RB_THREAD_BLOCKING_REGION -DHAVE_TBR -DHAVE_RB_TIME_NEW 
-DOS_WIN32 -D
HAVE_WINDOWS_H -DHAVE_WINSOCK_H -EHs -GR -DWITHOUT_SSL 
-fno-omit-frame-pointer -fno-s
trict-aliasing  -fexceptions  -m32 -march=native -mtune=native  -c 
binder.cpp
g++.exe: unrecognized option '-EHs'
g++.exe: unrecognized option '-GR'
g++ -I. -I. 
-IC:/Users/Jon/Documents/JavaDev/jruby/lib/native/include/ruby -I. 
-DBUILD
_FOR_RUBY -DHAVE_RB_THREAD_BLOCKING_REGION -DHAVE_TBR -DHAVE_RB_TIME_NEW 
-DOS_WIN32 -D
HAVE_WINDOWS_H -DHAVE_WINSOCK_H -EHs -GR -DWITHOUT_SSL 
-fno-omit-frame-pointer -fno-s
trict-aliasing  -fexceptions  -m32 -march=native -mtune=native  -c 
cmain.cpp
cmain.cpp: In function 'int evma_send_file_data_to_connection(long 
unsigned int, const
 char*)':
cmain.cpp:752:20: error: 'fstat' was not declared in this scope
g++.exe: unrecognized option '-EHs'
g++.exe: unrecognized option '-GR'
make: *** [cmain.o] Error 1

On 18 August 2010 11:03, Jon <jon.forums@gmail.com> wrote:
> * should -fexceptions be removed from RbConfigLibrary.java?

If its used when compiling the c extensions, then definitely not.  Its
needed so C++ exceptions (which is how rb_raise() is implemented) get
propagated thru the C extension code.  Without -fexceptions, as soon
as an exception is raised, the jvm terminates ... which is usually not
what one wants.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> * FYI, I still haven't found a way to remove cc.bat and cc (shell script) from DevKit so that the Makefile's built from RbConfigLibrary.java can have their "CC=cc" overridden.

Something's very wrong, either in JRuby's mkmf.rb or my understanding.

When I run, using either 'gcc' or 'gcc.exe' for CC...

C:\Users\Jon\Documents\CDev\rdiscount>set CC=gcc.exe && gem install 
rdiscount-1.6.5.jm1.gem --platform=ruby -- --environment-overrides

...this should override any CC in the extensions Makefile created from 
running extconf.rb.  Not so.

As "make -p | grep CC" shows make uses "CC = cc" by default, I tried 
deleting

http://github.com/jruby/jruby/blob/master/src/org/jruby/libraries/RbConfigLibrary.java#L289

in hopes the extension Makefile simply _wouldn't_ define CC similar to 
how CXX is currently undefined.  Unfortunately the generated Makefile 
had "CC =" and I couldn't override it from the environment.

I'm loathe to start spelunking through JRuby 1.8's custom mkmf.rb...any 
other ideas or something I'm overlooking?

Jon

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> Jon, could you test? I have e.g. native mongrel and rfuzz with passing specs here, so those should definitely run. You could also try curb, eventmachine, racc, ruby-pg which I have mostly working here (mostly some spec failures here and there)

Based upon failures attempting to install curb, I think JRuby's RubyGems 
may be causing some of the problems I'm seeing when trying to override 
CC and will cause problems if one tries to install a native gem and 
needs to explicitly point to headers/libs.

The following works with the MRI 1.9.2 branch and MRI trunk (1.9.3). 
Notice that in jruby's call to extconf.rb, none of the build args given 
to "gem install" are passed.

Can you replicate what I'm seeing with a clean PATH like?

C:\Users\Jon\Documents>echo %PATH%
c:\Users\Jon\Documents\JavaDev\jruby\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem


C:\Users\Jon\Documents>gem install curb --platform=ruby -- 
--with-curl-lib="C:/gnuwin32/curl/bin" 
--with-curl-include="C:/gnuwin32/curl/include"
JRuby limited openssl loaded. http://jruby.org/openssl
gem install jruby-openssl for full support.
Temporarily enhancing PATH to include DevKit...
Building native extensions.  This could take a while...
ERROR:  Error installing curb:
        ERROR: Failed to build gem native extension.

c:/Users/Jon/Documents/JavaDev/jruby/bin/jruby.exe extconf.rb
WARNING: JRuby does not support native extensions or the `mkmf' library 
very well.
Check http://kenai.com/projects/jruby/pages/Home for alternatives.
c:/Users/Jon/Documents/JavaDev/jruby/lib/ruby/1.8/mkmf.rb:31 warning: 
already initialized constant RUBY_PLATFORM
" -fno-omit-frame-pointer -fno-strict-aliasing  -fexceptions"
checking for curl-config... no
checking for main() in -lcurl... no
*** extconf.rb failed ***


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

On Aug 18, 2010, at 3:30 PM, Jon wrote:

> As "make -p | grep CC" shows make uses "CC = cc" by default, I tried deleting
> 
> http://github.com/jruby/jruby/blob/master/src/org/jruby/libraries/RbConfigLibrary.java#L289
> 
> in hopes the extension Makefile simply _wouldn't_ define CC similar to how CXX is currently undefined.  Unfortunately the generated Makefile had "CC =" and I couldn't override it from the environment.
> 
> I'm loathe to start spelunking through JRuby 1.8's custom mkmf.rb...any other ideas or something I'm overlooking?

It's not actually that 'custom'... the modifications are only at the 
very top for working around things which work differently in RbConfig, 
in "have_func" and in the removal of the dependency on defines.h and 
ruby.h

The line 
http://github.com/jruby/jruby/blob/cext/lib/ruby/1.8/mkmf.rb#L1348 is 
the reason why the CC variable is always set in the Makefile (see 
http://github.com/ruby/ruby/blob/trunk/lib/mkmf.rb#L1507 where it's done 
in the same way, only they also include the CXX compiler).

> Based upon failures attempting to install curb, I think JRuby's RubyGems may be causing some of the problems I'm seeing when trying to override CC and will cause problems if one tries to install a native gem and needs to explicitly point to headers/libs.

Okay, there are two problems here, first rubygems isn't passing the 
paramters to the extconf.rb and secondly the paramter 
'environment-overrides' is ignored entirely. Do you know where MRI 
handles the latter? I couldn't find a reference to it in mkmf.rb. 
Additionally, I have seen a few extconf.rb's which handle this 
themselves ( ENV["CC"] || CONFIG["CC"] ).

-Tim


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> It's not actually that 'custom'... the modifications are only at the very top for working around things which work differently in RbConfig, in "have_func" and in the removal of the dependency on defines.h and ruby.h
> 
> The line http://github.com/jruby/jruby/blob/cext/lib/ruby/1.8/mkmf.rb#L1348 is the reason why the CC variable is always set in the Makefile (see http://github.com/ruby/ruby/blob/trunk/lib/mkmf.rb#L1507 where it's done in the same way, only they also include the CXX compiler).

Thanks for the pointers, I'll start investigating.


> Okay, there are two problems here, first rubygems isn't passing the paramters to the extconf.rb and secondly the paramter 'environment-overrides' is ignored entirely. Do you know where MRI handles the latter? I couldn't find a reference to it in mkmf.rb. Additionally, I have seen a few extconf.rb's which handle this themselves ( ENV["CC"] || CONFIG["CC"] ).
> 

I think those params are gathered up by rubygems and then made part of a 
"jruby extconf.rb --my-build-option --another-build-option" backtick 
call.  Haven't had a chance to track it down and verify yet.

I've also played around with adding "RbConfig::CONFIG['CC'] = 'gcc'" to 
the DevKit-installed operating_system.rb with no luck.  Also tried 
http://github.com/luislavena/sqlite3-ruby/blob/master/ext/sqlite3/extconf.rb#L7 
with no luck.

I want to check whether manually running "jruby extconf.rb --blah --blah 
--blah" from inside the curb ext folder works.  That should tell us 
whether JRuby's mkmf.rb is guilty of ignoring its parameters.

Jon

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> I've also played around with adding "RbConfig::CONFIG['CC'] = 'gcc'" to the DevKit-installed operating_system.rb with no luck.  Also tried http://github.com/luislavena/sqlite3-ruby/blob/master/ext/sqlite3/extconf.rb#L7 with no luck.

In mkmf the Makefile values are taken from RbConfig::CONFIG hash, not 
MAKEFILE_CONFIG hash. That's why I "merge!" MAKEFILE_CONFIG over CONFIG 
in the header of mkmf.rb. This is why setting RbConfig::CONFIG['CC'] = 
'gcc' in the operating_system.rb won't do, as the mkmf.rb is loaded 
afterwards and just overrides this. In the sqlite3/extconf.rb it's the 
opposite: MAKEFILE_CONFIG is modified, but that is ignored here, as it 
has been previously force-merged into CONFIG for this execution and is 
ignored further down.

-Tim
---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

> In mkmf the Makefile values are taken from RbConfig::CONFIG hash, not MAKEFILE_CONFIG hash. That's why I "merge!" MAKEFILE_CONFIG over CONFIG in the header of mkmf.rb. This is why setting RbConfig::CONFIG['CC'] = 'gcc' in the operating_system.rb won't do, as the mkmf.rb is loaded afterwards and just overrides this. In the sqlite3/extconf.rb it's the opposite: MAKEFILE_CONFIG is modified, but that is ignored here, as it has been previously force-merged into CONFIG for this execution and is ignored further down.

I'm hoping we find a robust mod to the current behavior that will allow 
either RbConfig::CONFIG['CC'] or ENV['CC'] overrides in 
operating_system.rb's pre_install hook and 
lib/ruby/site_ruby/shared/devkit.rb to work similar to how I override 
ENV['PATH'] to bring in the DevKit in only very specific situations, 
i.e. 'gem install' and 'jruby -rdevkit extconf.rb'

Things are going to be crazy busy at work for me starting today through 
next week but I'll reply with any new info I might unearth.

Did you discover anything on the missing build args issue with the 
"jruby extconf.rb" call?  I'm going to try the "jruby -rdevkit 
extconf.rb --with-curl-lib --with-curl-include" with curb to see if 
mkmf.rb is really ignoring them.

Jon

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Building this morning with bf3e11a using "ant clean cext" gives me the 
following failure:

     [exec] gcc -DNDEBUG -O2 -fno-omit-frame-pointer 
-fno-strict-aliasing -W -Wa
ll -Wno-unused -Wno-parentheses -Werror -Wundef 
-I"/c/Users/Jon/Documents/JavaDe
v/jruby/cext/src/../..//build" 
-I"/c/Users/Jon/Documents/JavaDev/jruby/cext/src"
 -I"/c/Users/Jon/Documents/JavaDev/jruby/cext/src/../..//build"/jni 
-I"/c/Users/
Jon/Documents/JavaDev/jruby/cext/src"/include 
-I"/c/Users/Jon/Documents/JavaDev/
jruby/cext/src"/include/ruby  -I"C:\Program 
Files\Java\jdk1.6.0_21/include" -I"C
:\Program Files\Java\jdk1.6.0_21/include/win32"  -D_REENTRANT 
-D_LARGEFILE64_SOU
RCE -D_GNU_SOURCE -march=native -mtune=native -D_JNI_IMPLEMENTATION_ 
-DRUBY_DLLS
PEC="__declspec(dllexport)" -c 
/c/Users/Jon/Documents/JavaDev/jruby/cext/src/st.
c -o /c/Users/Jon/Documents/JavaDev/jruby/cext/src/../..//build/st.o
     [exec] In file included from 
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/
include/ruby.h:2:0,
     [exec]                  from 
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/
st.c:5:
     [exec] 
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/include/ruby/ruby.h:90
8:33: error: external linkage required for symbol 
'rb_thread_stop_timer_thread'
because of 'dllexport' attribute
     [exec] 
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/include/ruby/ruby.h:90
9:33: error: external linkage required for symbol 
'rb_thread_start_timer_thread'
 because of 'dllexport' attribute
     [exec] 
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/include/ruby/ruby.h:91
0:33: error: external linkage required for symbol 'rb_thread_stop_timer' 
because
 of 'dllexport' attribute
     [exec] 
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/include/ruby/ruby.h:91
1:33: error: external linkage required for symbol 
'rb_thread_start_timer' becaus
e of 'dllexport' attribute
     [exec] make: *** 
[/c/Users/Jon/Documents/JavaDev/jruby/cext/src/../..//buil
d/st.o] Error 1

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

"ant clean cext" now works for me with 9a285a3.

Running the following indicates JRuby's mkmf.rb handles the extconf.rb 
args OK and something in jruby (rubygems?) is not sending the args from 
the "gem install" command.  Didn't have time to look into the CC 
override issue today.

FYI, I just released the new DevKit at 
http://rubyinstaller.org/downloads/ with usage info at 
http://wiki.github.com/oneclick/rubyinstaller/development-kit and I'll 
release another version once we're both happy with the mods needed for 
things to work flawlessly with JRuby cext.

Jon


C:\Users\Jon\Documents\JavaDev\jruby\lib\ruby\gems\1.8\gems\curb-0.7.8\ext>jruby 
-rdevkit extconf.rb --with-curl-lib="C:/gnuwin32/curl/bin" 
--with-curl-include="C:/gnuwin32/curl/include"
Temporarily enhancing PATH to include DevKit...
WARNING: JRuby does not support native extensions or the `mkmf' library 
very well.
Check http://kenai.com/projects/jruby/pages/Home for alternatives.
C:/Users/Jon/Documents/JavaDev/jruby/lib/ruby/1.8/mkmf.rb:31 warning: 
already initiali
zed constant RUBY_PLATFORM
" -fno-omit-frame-pointer -fno-strict-aliasing  -fexceptions"
checking for curl-config... no
checking for main() in -lcurl... yes
checking for curl/curl.h... yes
checking for curlinfo_redirect_time... yes
checking for curlinfo_response_code... yes
checking for curlinfo_filetime... yes
checking for curlinfo_redirect_count... yes
checking for curlinfo_os_errno... yes
checking for curlinfo_num_connects... yes
checking for curlinfo_ftp_entry_path... yes
checking for curl_version_ssl... yes
checking for curl_version_libz... yes
checking for curl_version_ntlm... yes
checking for curl_version_gssnegotiate... yes
checking for curl_version_debug... yes
checking for curl_version_asynchdns... yes
checking for curl_version_spnego... yes
checking for curl_version_largefile... yes
checking for curl_version_idn... yes
checking for curl_version_sspi... yes
checking for curl_version_conv... yes
checking for curlproxy_http... yes
checking for curlproxy_socks4... yes
checking for curlproxy_socks5... yes
checking for curlauth_basic... yes
checking for curlauth_digest... yes
checking for curlauth_gssnegotiate... yes
checking for curlauth_ntlm... yes
checking for curlauth_anysafe... yes
checking for curlauth_any... yes
checking for curle_tftp_notfound... yes
checking for curle_tftp_perm... yes
checking for curle_tftp_diskfull... yes
checking for curle_tftp_illegal... yes
checking for curle_tftp_unknownid... yes
checking for curle_tftp_exists... yes
checking for curle_tftp_nosuchuser... yes
checking for curle_send_fail_rewind... yes
checking for curle_ssl_engine_initfailed... yes
checking for curle_login_denied... yes
checking for curlmopt_maxconnects... yes
checking for curle_conv_failed... yes
checking for curle_conv_reqd... yes
checking for curle_ssl_cacert_badfile... yes
checking for curle_remote_file_not_found... yes
checking for curle_ssh... yes
checking for curle_ssl_shutdown_failed... yes
checking for curle_again... yes
checking for curle_ssl_crl_badfile... yes
checking for curle_ssl_issuer_error... yes
checking for curlopt_username... yes
checking for curlopt_password... yes
checking for curlinfo_primary_ip... yes
checking for curlauth_digest_ie... yes
checking for curlftpmethod_multicwd... yes
checking for curlftpmethod_nocwd... yes
checking for curlftpmethod_singlecwd... yes
checking for curlm_bad_socket... yes
checking for curlm_unknown_option... yes
checking for curl_multi_timeout()... yes
checking for curl_multi_fdset()... yes
checking for curl_multi_perform()... yes
checking for Ruby 1.9 Hash... no
checking for Ruby 1.9 st.h... no
checking for curl_easy_escape... yes
creating curb_config.h
creating Makefile

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

FYI, a few tweaks to RbConfigLibrary.java (attached) and the following 
mods to the DevKit helper files and I'm able to override CC correctly 
for "jruby -rdevkit extconf.rb" cases.

There's still something in JRuby/RubyGems that's preventing CC overrides 
for the "gem install my_ext --platform=ruby" cases and I think we still 
have the problem of the build options given to "gem install" not being 
passed to the "jruby extconf.rb" call.

Jon


# FILE: lib/ruby/site_ruby/shared/devkit.rb
# enable RubyInstaller DevKit usage as a vendorable helper library
unless ENV['PATH'].include?('C:\\DevKit\\mingw\\bin') then
  require 'rbconfig'
  puts 'Temporarily enhancing PATH to include DevKit...'
  ENV['PATH'] = 'C:\\DevKit\\bin;C:\\DevKit\\mingw\\bin;' + ENV['PATH']
  RbConfig::MAKEFILE_CONFIG['CC'] = 'gcc'
end

# FILE: lib/ruby/site_ruby/1.8/rubygems/defaults/operating_system.rb
# override 'gem install' to enable RubyInstaller DevKit usage
Gem.pre_install do |i|
  require 'rbconfig'
  unless ENV['PATH'].include?('C:\\DevKit\\mingw\\bin') then
    puts 'Temporarily enhancing PATH to include DevKit...'
    ENV['PATH'] = 'C:\\DevKit\\bin;C:\\DevKit\\mingw\\bin;' + 
ENV['PATH']
  end
  RbConfig::MAKEFILE_CONFIG['CC'] = 'gcc'
end