# Some benchmarks with MinGW, JRuby, MSVC

**URL:** <https://www.ruby-forum.com/t/some-benchmarks-with-mingw-jruby-msvc/154492>\
**Category:** Ruby\
**Created:** [November 30, 2008, 6:20pm UTC](https://www.ruby-forum.com/t/some-benchmarks-with-mingw-jruby-msvc/154492 "2008-11-30T18:20:40Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![jpedrosa](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/ecae2f/32.png) [@jpedrosa](https://www.ruby-forum.com/u/jpedrosa)\
**Post date:** [November 30, 2008, 6:20pm UTC](https://www.ruby-forum.com/t/some-benchmarks-with-mingw-jruby-msvc/154492/1 "2008-11-30T18:20:40Z")

</div>

So, I have managed to build several different versions of Ruby to  
compare all  
in some benchmarks. I am going to let the numbers talk for themselves.  
It was  
all tested on an old desktop computer with Windows XP, so I am not  
testing  
anything too impressive.

Of importance is the seemingly poor performance of Ruby 1.8.7 for  
whatever  
reason. It’s as if Ruby 1.8.7 were like Windows ME, a version between  
major  
versions, as an analogy. 🙂

Also, I was not able to run the Mongrel gem on Ruby 1.9.1.

I am going to order the data by fastest to make it easier to compare  
it on text.  
I have collected the data on a Google Spreadsheet:

> **[ruby\_benchmark\_on\_windows](https://docs.google.com/spreadsheets/d/1S8j0LPbq0OSpEMIGv3SR8G_b62ssSNpiwH2DSw5cbbE/edit)**
>
> Sheet1
> 
> Ruby 1.8.6 OneClick Installer,Ruby 1.8.6 RubyInstaller - custom built,Ruby 1.8.6 MinGW custom built,Ruby 1.8.6 MSVC custom built,Ruby 1.8.7 MinGW custom built,Ruby 1.8.7 MSVC custom built,Ruby 1.9.1 MinGW custom built,Ruby 1.9.1 MSVC custom...

I am going to keep it short here and for more detail on versions and  
whatnot  
please refer to the spreadsheet.

== WEBrick (req/s)

22.39 - jruby  
16.95 - jruby --server  
12.82 - ruby oneclick 1.8.6  
12.49 - ruby msvc 1.8.6  
12.45 - ruby rubyinstaller 1.8.6  
12.43 - ruby mingw 1.8.6  
11.94 - ruby msvc 1.9.1  
11.48 - ruby mingw 1.9.1  
3.42 - ruby msvc 1.8.7  
3.40 - ruby mingw 1.8.7

== Mongrel (req/s)

357.14 - ruby mingw 1.8.6  
347.45 - ruby rubyinstaller 1.8.6  
326.70 - ruby msvc 1.8.6  
313.57 - ruby oneclick 1.8.6  
274.68 - ruby mingw 1.8.7  
232.39 - ruby msvc 1.8.7  
71.24 - jruby  
62.22 - jruby --server  
(crash when starting up) - ruby mingw 1.9.1  
(error when building gem) - ruby msvc 1.9.1

== bench\_pythag.rb (s)

19.09 - jruby --server  
24.35 - jruby  
31.72 - ruby mingw 1.9.1  
31.91 - ruby mingw 1.8.6  
32.73 - ruby mingw 1.8.7  
33.94 - ruby msvc 1.9.1  
35.67 - ruby rubyinstaller 1.8.6  
35.83 - ruby msvc 1.8.7  
39.79 - ruby msvc 1.8.6  
48.30 - ruby oneclick 1.8.6

== bench\_fractal.rb (s)

3.22 - jruby --server  
3.89 - jruby  
5.67 - ruby msvc 1.9.1  
5.75 - ruby mingw 1.9.1  
9.80 - ruby mingw 1.8.6  
9.90 - ruby mingw 1.8.7  
10.11 - ruby rubyinstaller 1.8.6  
10.30 - ruby msvc 1.8.6  
10.64 - ruby msvc 1.8.7  
15.17 - ruby oneclick 1.8.6

== bench\_float\_math.rb (s)

103.44 - jruby --server  
125.89 - jruby  
158.64 - ruby msvc 1.9.1  
175.41 - ruby mingw 1.9.1  
176.61 - ruby mingw 1.8.6  
179.94 - ruby mingw 1.8.7  
184.73 - ruby rubyinstaller 1.8.6  
195.37 - ruby msvc 1.8.7  
197.39 - ruby msvc 1.8.6  
279.62 - ruby oneclick 1.8.6

Since I collected these data, the Ruby 1.9.1 branch has had a number  
of updates  
for one thing. 🙂

Benchmarks can be kind of tough to do right and there are all  
possibilities when it  
comes to doing them.

If anyone has an algorithm that he would like to test on all those  
Ruby implementations  
I would be glad to test it. With the exception of testing Rails, which  
I have never used  
and don’t know enough about it to willingly make myself available when  
testing it. 🙂

Cheers,  
Joao

---

<div class="post-metadata">

**Author:** ![jpedrosa](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/ecae2f/32.png) [@jpedrosa](https://www.ruby-forum.com/u/jpedrosa)\
**Post date:** [December 1, 2008, 12:02am UTC](https://www.ruby-forum.com/t/some-benchmarks-with-mingw-jruby-msvc/154492/2 "2008-12-01T00:02:16Z")

</div>

According to my own tests, “Thin” should be better than your current  
best, which is Mongrel. Furthermore, TudbcJRubyServlet should help JRuby  
win over everyone.

[http://www.tudbc.org/tudbcjrubyservlet](http://www.tudbc.org/tudbcjrubyservlet)

You should also give more details about how you test.

---

<div class="post-metadata">

**Author:** ![jpedrosa](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/ecae2f/32.png) [@jpedrosa](https://www.ruby-forum.com/u/jpedrosa)\
**Post date:** [December 1, 2008, 1:19am UTC](https://www.ruby-forum.com/t/some-benchmarks-with-mingw-jruby-msvc/154492/3 "2008-12-01T01:19:41Z")

</div>

Joao P. wrote:

> (crash when starting up) - ruby mingw 1.9.1  
> (error when building gem) - ruby msvc 1.9.1

I’d love to figure out what makes mongrel slow for us, when all other  
cases are faster 😉

GlassFish gem would be a nice one to test, but of course there’s no  
equivalent comparison for the others.

- Charlie

---

<div class="post-metadata">

**Author:** ![jpedrosa](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/ecae2f/32.png) [@jpedrosa](https://www.ruby-forum.com/u/jpedrosa)\
**Post date:** [December 1, 2008, 8:23pm UTC](https://www.ruby-forum.com/t/some-benchmarks-with-mingw-jruby-msvc/154492/4 "2008-12-01T20:23:57Z")

</div>

Benchmarking ruby on windows is fairly useless. MRI 1.8.\* is known to  
be very slow on windows and harldy anyone deploys production apps to  
windows because of this. If you want your benches to be taken  
seriously you need to run them on linux, bsd or osx where ruby  
performs much better. Also make sure to compile ruby with --disable-  
pthreads for best perf.

Cheers-  
Ezra Z.  
removed\_email\_address@domain.invalid

---

<div class="post-metadata">

**Author:** ![jpedrosa](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/ecae2f/32.png) [@jpedrosa](https://www.ruby-forum.com/u/jpedrosa)\
**Post date:** [December 1, 2008, 8:49pm UTC](https://www.ruby-forum.com/t/some-benchmarks-with-mingw-jruby-msvc/154492/5 "2008-12-01T20:49:53Z")

</div>

On Mon, Dec 1, 2008 at 2:17 PM, Ezra Z.  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)wrote:

> removed\_email\_address@domain.invalid  
> Though it might be slow, I really don’t think it’s useless. Where I  
> work, we  
> are stuck in windows no matter the cost or benefit of another platform.  
> Because of this these windows numbers actually matter to me.

---

<div class="post-metadata">

**Author:** ![jpedrosa](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/ecae2f/32.png) [@jpedrosa](https://www.ruby-forum.com/u/jpedrosa)\
**Post date:** [December 1, 2008, 11:51pm UTC](https://www.ruby-forum.com/t/some-benchmarks-with-mingw-jruby-msvc/154492/6 "2008-12-01T23:51:42Z")

</div>

Post Tudbc wrote:

> According to my own tests, “Thin” should be better than your current  
> best, which is Mongrel. Furthermore, TudbcJRubyServlet should help JRuby  
> win over everyone.
> 
> [http://www.tudbc.org/tudbcjrubyservlet](http://www.tudbc.org/tudbcjrubyservlet)
> 
> You should also give more details about how you test.

I notice that you use the best of 5 runs to report numbers, which I  
think it is going to vary a lot. You can use my free tool to do web  
benchmarking, which I think is much more accurate.  
[http://www.tudbc.org/tudbcwebbenchmark](http://www.tudbc.org/tudbcwebbenchmark)

I think your Machine is about same as my old test machine. Here are some  
numbers:

Using “Echo” test: (copied from [http://www.tudbc.org/jrubywebapp](http://www.tudbc.org/jrubywebapp))

0.98ms TudbcJRubyServlet\_RB  
1.73ms Thin  
4.04ms Mongrel (estimated, because Mongrel crashed in the middle)  
8.51ms WEBrick

Using your “time now+version” test:

1.11ms TudbcJRubyServlet\_RB  
1.56ms Thin  
4.00ms Mongrel (estimated, because Mongrel crashed in the middle)  
8.18ms WEBrick

As you can see in your “time now” test on your machine, your Mongrel  
results were probably inaccurate, because you used the “best of 5” to  
measure.

Both my “Echo” and your “Time now+version” tests seem to produce similar  
benchmarks. As you can see, TudbcJRubyServlet\_RB still win easily, and  
Thin is faster than Mongrel by a wide margin.

Note: The default measurement of TudbcWebBenchmark is average of last 10  
of 20 runs (with each run for 10 seconds). Because Mongrel crashed, so I  
only estimate it by 3rd run of 5 seconds each.

Note: Ruby version used: JRuby 1.1.5 and Ruby 1.8.7-p72 on Windows

---

<div class="post-metadata">

**Author:** ![jpedrosa](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/ecae2f/32.png) [@jpedrosa](https://www.ruby-forum.com/u/jpedrosa)\
**Post date:** [December 2, 2008, 12:24am UTC](https://www.ruby-forum.com/t/some-benchmarks-with-mingw-jruby-msvc/154492/7 "2008-12-02T00:24:39Z")

</div>

Oops! Forgot to paste the codes I used for TudbcJRubyServlet

## --------- timeversion.rb -------- a = [] a \<\< ‘Hello Rack!’ a \<\< Time.new a \<\< RUBY\_PLATFORM a \<\< RUBY\_VERSION ctx.print(a.join(’ ’))

Your “time now+version” uses 5 array ops. My guess is that maybe JRuby  
array processing is relatively not as efficient, so that ratio between  
TudbcJRubyServlet and Thin for “time now+version” is not as great as  
“Echo” test.

Note: My 4ms means your 1000/4 or 250 reqs/sec.