Forum: IronRuby Very Very initial CMake support

I have started playing around with cmake to see whether it could help 
out
iron* and dlr.  I have therefore started implementing CMake makefiles to 
build
the dlr (Runtime) part of ironruby, install those dlls into the gac and
generate *.pc files for them.  I have tested it on mono-2.8 (requires 
mono-2.8)
and am aware that windows .NET support is broken (but very possible)

cmake also supports creating deb's and rpm's which will be done in the 
future
hopefully.  The potential is also there for it to generate csproj files
(already supports other VS file types)

To play around with it

git clone git://github.com/alistair/ironruby.git
mkdir build
mkdir install
cd build  (out of tree builds,   oh how I love them )
cmake ../repo/Runtime/
make
make DESTDIR=../install install


After this you should have

install $ find
.
./usr
./usr/lib64
./usr/lib64/mono
./usr/lib64/mono/Microsoft.Scripting.Metadata
./usr/lib64/mono/Microsoft.Scripting.Metadata/Microsoft.Scripting.Metadata.dll
./usr/lib64/mono/Microsoft.Dynamic
./usr/lib64/mono/Microsoft.Dynamic/Microsoft.Dynamic.dll
./usr/lib64/mono/Microsoft.Scripting
./usr/lib64/mono/Microsoft.Scripting/Microsoft.Scripting.dll
./usr/lib64/mono/Microsoft.Scripting.Core
./usr/lib64/mono/Microsoft.Scripting.Core/Microsoft.Scripting.Core.dll
./usr/lib64/mono/gac
./usr/lib64/mono/gac/Microsoft.Scripting.Metadata
./usr/lib64/mono/gac/Microsoft.Scripting.Metadata/1.1.0.10__7f709c5b713576e1
./usr/lib64/mono/gac/Microsoft.Scripting.Metadata/1.1.0.10__7f709c5b713576e1/Microsoft.Scripting.Metadata.dll
./usr/lib64/mono/gac/Microsoft.Dynamic
./usr/lib64/mono/gac/Microsoft.Dynamic/1.1.0.10__7f709c5b713576e1
./usr/lib64/mono/gac/Microsoft.Dynamic/1.1.0.10__7f709c5b713576e1/Microsoft.Dynamic.dll
./usr/lib64/mono/gac/Microsoft.Scripting
./usr/lib64/mono/gac/Microsoft.Scripting/1.1.0.10__7f709c5b713576e1
./usr/lib64/mono/gac/Microsoft.Scripting/1.1.0.10__7f709c5b713576e1/Microsoft.Scripting.dll
./usr/lib64/mono/gac/Microsoft.Scripting.Core
./usr/lib64/mono/gac/Microsoft.Scripting.Core/1.1.0.10__7f709c5b713576e1
./usr/lib64/mono/gac/Microsoft.Scripting.Core/1.1.0.10__7f709c5b713576e1/Microsoft.Scripting.Core.dll
./usr/local
./usr/local/lib
./usr/local/lib/pkgconfig
./usr/local/lib/pkgconfig/microsoft.scripting.metadata.pc
./usr/local/lib/pkgconfig/microsoft.scripting.core.pc
./usr/local/lib/pkgconfig/microsoft.dynamic.pc
./usr/local/lib/pkgconfig/microsoft.scripting.pc

still lots of work to do,   but hopefully you enjoy.

Alistair.

Having multiple building systems is counterproductive, it's hard to 
maintain
all of them.

Is cmake using xbuild to buil the dlls or does it all the file linking 
by
itself?

> Having multiple building systems is counterproductive, it's hard to
> maintain all of them.
>

In General I agree. thats is why ultimately I would like cmake to 
support
creating of csproj files.   That would need to be implemented,  but has 
already
been proven possible by cmakes other VS proj support.

For this project I don't believe that maintaining cmake will be all that
difficult.  it supports wildcards (Dir/*.cs) when selecting which files 
to build
which means that in most cases no changes will need to be made.  There 
is an
exception to this which im planning on raising in a separate thread.
Currently that work I have done with cmake has encountered a large 
number of
"abandoned" files within the repo.   It seems these classes have been 
migrated
into System.* but only removed from the proj file.   This is really 
really
messy.

> Is cmake using xbuild to buil the dlls or does it all the file linking by
> itself?

Nope it calls the compiler directly.   Sadly I think it would be very 
difficult
to maintain xbuild to do what I want it to do.  Ultimately we need to 
get the
Runtime/ directory out of the tree (or at least make linking against it
optional).  There are also other benefits.

* It also makes pkg-config support very easy.
* Allows multiple (as many as you want) out of tree builds so that you 
can for
example run tests on standard debug build as well as a silverlight build
simultaneously.  You also don't have to be constantly switch configs. 
you can
easy target multiple versions of mono/.NET etc etc etc.
* supports creating of debs and rpms directly.

Please don't add yet another build system.

Generating .csproj files is not the right way to go. The project files 
are and should be the primary metadata storage for the build system. If 
you work in VS and add a new file VS will add it into csproj.
If you would like to build some packages (like rpms) you can write a 
.proj file that does that. Like we do for building .msis (see 
Msi\Installer.proj). If there is something msbuild/xbuild doesn't 
support you can write a custom task that does that.
If there is some bug in xbuild that prevents you to do what you need the 
bug needs to be fixed in xbuild.

Other than supporting packages, what's exactly is the scenario that 
doesn't work today that you want to support? I don't understand what do 
you mean by "Allows multiple (as many as you want) out of tree builds 
...". Can you be more specific?

Tomas

On Wed, Nov 3, 2010 at 7:56 PM, Tomas Matousek 
<Tomas.Matousek@microsoft.com
> wrote:

> Please don't add yet another build system.
>

I completely agree with that, though I'm interested why there are 3 sets 
of
all csproj in every project directory? As far

Which directories do you have on mind? I only see a single csproj file 
here

https://github.com/ironruby/ironruby/tree/8acfd2f7...

There are two here:
https://github.com/ironruby/ironruby/tree/8acfd2f7...

one set up to build 32bit only exe and the other with AnyPlatform (64bit 
on 64bit machines).

If you see *.build.csproj files somewhere let me know. Those were part 
of an internal Microsoft infrastructure and should have been deleted, 
but maybe I missed some.

Tomas

From: ironruby-core-bounces@rubyforge.org 
[mailto:ironruby-core-bounces@rubyforge.org] On Behalf Of Andrius 
Bentkus
Sent: Wednesday, November 03, 2010 5:16 PM
To: ironruby-core@rubyforge.org
Subject: Re: [Ironruby-core] Very Very initial CMake support


On Wed, Nov 3, 2010 at 7:56 PM, Tomas Matousek 
<Tomas.Matousek@microsoft.com<mailto:Tomas.Matousek@microsoft.com>> 
wrote:
Please don't add yet another build system.

I completely agree with that, though I'm interested why there are 3 sets 
of all csproj in every project directory? As far

Generating .csproj files is not the right way to go. The project files 
are and should be the primary metadata storage for the build system. If 
you work in VS and add a new file VS will add it into csproj.
If you would like to build some packages (like rpms) you can write a 
.proj file that does that. Like we do for building .msis (see 
Msi\Installer.proj). If there is something msbuild/xbuild doesn't 
support you can write a custom task that does that.
If there is some bug in xbuild that prevents you to do what you need the 
bug needs to be fixed in xbuild.

Other than supporting packages, what's exactly is the scenario that 
doesn't work today that you want to support? I don't understand what do 
you mean by "Allows multiple (as many as you want) out of tree builds 
...". Can you be more specific?

Tomas

> Please don't add yet another build system.
>
> Generating .csproj files is not the right way to go. The project files are
> and should be the primary metadata storage for the build system. If you
> work in VS and add a new file VS will add it into csproj.

It can [1] be automatically available to cmake to consume.   Only those 
cases
where files are only applicable for silverlight would there be any 
changes
required and you already have to hand crank the csproj files anyway.

> If you would
> like to build some packages (like rpms) you can write a .proj file that
> does that. Like we do for building .msis (see Msi\Installer.proj). If
> there is something msbuild/xbuild doesn't support you can write a custom
> task that does that. If there is some bug in xbuild that prevents you to
> do what you need the bug needs to be fixed in xbuild.
>

Currently that sounds like more work, not less.

> Other than supporting packages, what's exactly is the scenario that doesn't
> work today that you want to support?

Mainly the complete and easy separation of dlr from ironruby.  The 
ablility to
install ironruby into a location of my pleasing with a simple command. 
easy
pkg-config support.  The ability to develop and compile against multiple
versions of mono/moonlight/etc.  Probably a few more but I haven't 
thought of
them :)

> I don't understand what do you mean
> by "Allows multiple (as many as you want) out of tree builds ...". Can you
> be more specific?

maybe "out of source" would explain it more.

if we take this example here

git clone git://github.com/alistair/ironruby.git
mkdir build
mkdir install
cd build
cmake ../repo/Runtime/
make
make DESTDIR=../install install

the build dir is an "out of tree/source build".  after executing make 
(aka a
compile task) no changes have been made to the git repo.  No new dll's, 
etc,
etc. They are all located within build.  rm -R build is the equivilent 
of
running a clean task.

this build directory also is a configuration cache.  So currently the 
csproj
have mulitple PropertyGroup conditional on 
$(Configuration)|$(Platform)'.  eg.
Debug|AnyCPU, v2Debug|AnyCPU ( Release/v2Release and Silverlight 
versions as
well).

with cmake you just go.

mkdir silverlight-build-debug; cd silverlight-build-debug;
cmake -D CONF_VAR_NAME=SilverlightDebug ../repo/Runtime/
and you are then able to build both debug (from above) and silverlight 
debug
config targets (compile, test, etc) side-by-side simultantiously.  No 
building
one and then selecting from a drop down to build the next configuration. 
No
need to have .gitignore either.

- Alistair.


[1]  I say "can" because the Runtime tree (at least) is a mess at the 
moment.
Far to many redundant files left over.  Sadly you just sync (copy is a 
more
appropriate term) entire directories between projects repos so now even 
if I
fix it within the ironruby tree I have to worry about them coming back 
in the
next sync.
> > Having multiple building systems is counterproductive, it's hard to
> thread. Currently that work I have done with cmake has encountered a large
> linking against it optional).  There are also other benefits.
>
> * It also makes pkg-config support very easy.
> * Allows multiple (as many as you want) out of tree builds so that you can
> for example run tests on standard debug build as well as a silverlight
> build simultaneously.  You also don't have to be constantly switch
> configs. you can easy target multiple versions of mono/.NET etc etc etc. *
> supports creating of debs and rpms directly.
>
> > On Wed, Nov 3, 2010 at 11:54 AM, Alistair Bush <ali_bush@gentoo.org>

I should also add that im going to continue because

1)  I want to learn more cmake
2)  there is also the possiblity to generate cmake files from csproj 
files :)

"Mainly the complete and easy separation of dlr from ironruby"

I don't think we need cmake to make this happen. In fact, I don't even 
think it is desirable to do so (at least not short term). I'm still 
convinced that having all source code in a single repo (perhaps split 
into multiple submodules) is the best thing to do given that we want to 
be able to easily debug cross-language interop and evolve DLR.

"The ablility to install ironruby into a location of my pleasing with a 
simple command"

This doesn't need to be part of a build. You can write a few lines of a 
script (even Ruby script!) that builds the binaries by running "xbuild 
Ruby.sln /p:Configuration=Release" first and then copy the binaries from 
the bin\Release dir to wherever you want them to be. This script can be 
completely customized as you want and doesn't even need to be checked 
into the repository. But I guess it could be if multiple contributors 
would find it useful.

"easy pkg-config support"
The above solution seems to be sufficient for this too. A script using 
the binaries produced by xbuild should do. Similarly, any other 
packaging can be done post build if you don't want to integrate with 
msbuild/xbuild. You don't need to run the packaging scripts every time a 
source code changes so there is no need for a dependency tracking.

" The ability to develop and compile against multiple versions of 
mono/moonlight/etc"

Having to too many different builds/flavors for different platforms is 
not desirable. Ideally we would have a single one. Today we have 4: 
desktop CLR 3.5, desktop CLR 4.0, Silverlight 4 and Windows Phone 7 and 
that's too many already. There are places in the source code that need 
to be #if-defed for these platforms. We have been supporting 3.5 only to 
make IronRuby work on Mono < 2.8. Now that 2.8 is out there we can get 
rid of this flavor and clean up the code a bit. In some (hopefully not 
so distant) future Windows Phone might sync with the latest Silverlight 
and thus we won't need a special flavor for WP either. We should only 
introduce a new flavor if a) none of the current binaries works on the 
target platform b) it would be too messy to adjust the behavior at 
runtime based upon the value of Environment.OSVersion, sizeof(IntPtr) 
etc. What versions of Mono do you want to target? The current target is 
2.8 since it supports features of .NET 4.0 and has many bug fixes for i
 ssues hit by IronRuby.

"out of source"
Sure, we can change the output directory to be outside of the repo. But 
is that really an issue? What's wrong with a few lines in .gitignore? 
Having a .gitignore file certainly requires much less effort than 
understanding and maintaining cmake files.

" So currently the csproj have multiple PropertyGroup conditional on 
$(Configuration)|$(Platform)"
Right. You have to have some kind of configuration - a set of 
flags/properties passed to the compiler. Certain symbols are defined 
based upon the chosen configuration, references to assemblies are 
different, etc. Sure, you can probably creat cmake files with all that. 
But then you'll need to keep it in sync with the project files. It's not 
just "msc /recurse:*.cs" done. And besides, I can select the 
configuration I want to debug and hit F5 in VS. I also get the 
IntelliSense based upon what's really available in given configuration. 
And so on.


In general, I would suggest focusing on features and bug fixes rather 
than wasting time with constructing another build system and trying to 
hack it so that it works with .csproj files. Scripts that would create 
various packages are indeed welcome. But please keep them simple and 
based upon the current build system.

Tomas

MSBuild is a pretty powerful build language, other than being ugly as
sin I don't see any reason why xbuild shouldn't be the way to go on
linux platforms. *shrugs* Just my two copper.



On Thu, Nov 4, 2010 at 4:07 AM, Tomas Matousek
<Tomas.Matousek@microsoft.com> wrote:
>
>
>
>> like to build some packages (like rpms) you can write a .proj file
>
> git clone git://github.com/alistair/ironruby.git
> Debug|AnyCPU, v2Debug|AnyCPU ( Release/v2Release and Silverlight
> [1] I say "can" because the Runtime tree (at least) is a mess at the moment.
>> Subject: Re: [Ironruby-core] Very Very initial CMake support
>> files to build which means that in most cases no changes will need to be made.
>> difficult to maintain xbuild to do what I want it to do. Ultimately
>> > On Wed, Nov 3, 2010 at 11:54 AM, Alistair Bush <ali_bush@gentoo.org>
>> > > cmake also supports creating deb's and rpm's which will be done in
>> > > make
>> > > ./usr/lib64/mono/Microsoft.Scripting.Metadata
>> > > ./usr/lib64/mono/gac/Microsoft.Scripting.Metadata
>> > >
>> > > e1
>> > >
>>
> Ironruby-core mailing list
> Ironruby-core@rubyforge.org
> http://rubyforge.org/mailman/listinfo/ironruby-core
>



--
Michael Letterle
IronRuby MVP
http://blog.prokrams.com