What do you guys use to connect to mongo?
I found this: GitHub - guyboertje/jmongo: UNMAINTAINED, consider Moped instead of JMongo
Not sure if this is production worthy though?
In general any gem that has/requires connection pooling and threading
should be re-written for jruby right?
On Mar 12, 2012, at 4:44 PM, S Ahmed wrote:
What do you guys use to connect to mongo?
I found this: GitHub - guyboertje/jmongo: UNMAINTAINED, consider Moped instead of JMongo
Not sure if this is production worthy though?
In general any gem that has/requires connection pooling and threading should be
re-written for jruby right?
I was the original author of jmongo. When I stopped supporting it, it
was taken over by other folks in the Ruby community. I’m sure it’s a
solid project.
I originally wrote it because the mongo Ruby driver performed poorly
under JRuby. JMongo, being a thin wrapper around the official Java
driver, was much faster. However, a native Java extension for JRuby was
added to the official Ruby driver. Based on the benchmarks that were
important to me for my production use, the Ruby driver was faster than
jmongo.
I don’t know if that is still the case or not. I would highly recommend
that you benchmark the Ruby driver versus jmongo and verify the
performance.
Regarding your question about rewriting gems for JRuby if they use
threads, the answer is no, that is not necessary.
cr
Chuck, thanks for your reply (and work on jmongo).
I found this benchmark but havent’ tested things myself just yet:
I’m actually stuck on how to go about setting the connection using the
mongo driver, can you show me how you set your db connection and then
how
you go about using it? (is it pooled?)
I just downloaded the mongo.jar and was going to use that, but if you
are
saying the ruby driver was good enough I’d rather stick with that for
now.
On Tue, Mar 13, 2012 at 2:13 PM, S Ahmed [email protected] wrote:
user system total real
~/dev/test/jruby>
Interesting results, thank for sharing them.
Could you please share how you modified the script on the connection
part using jmongo ?
–
Christian
Here are my results running the test 3 times, java mongo driver is way
faster (3x) for this specific benchmark:
~/dev/test/jruby>jruby mongo_bench.rb
user system total real
java driver (find) 5.962000 0.000000 5.962000 ( 5.963000)
ruby driver (find) 7.530000 0.000000 7.530000 ( 7.530000)
java driver (create) 0.969000 0.000000 0.969000 ( 0.969000)
ruby driver (create) 2.700000 0.000000 2.700000 ( 2.700000)
~/dev/test/jruby>jruby mongo_bench.rb
user system total real
java driver (find) 1.784000 0.000000 1.784000 ( 1.784000)
ruby driver (find) 7.723000 0.000000 7.723000 ( 7.723000)
java driver (create) 0.983000 0.000000 0.983000 ( 0.983000)
ruby driver (create) 2.667000 0.000000 2.667000 ( 2.667000)
~/dev/test/jruby>jruby mongo_bench.rb
user system total real
java driver (find) 1.757000 0.000000 1.757000 ( 1.756000)
ruby driver (find) 7.897000 0.000000 7.897000 ( 7.898000)
java driver (create) 0.965000 0.000000 0.965000 ( 0.965000)
ruby driver (create) 2.410000 0.000000 2.410000 ( 2.410000)
~/dev/test/jruby>
On Mar 12, 2012, at 8:41 PM, S Ahmed wrote:
Chuck, thanks for your reply (and work on jmongo).
I found this benchmark but havent’ tested things myself just yet:
# MongoDB Java driver vs Ruby Driver with JRuby 1.3.1## user system - Pastebin.comI’m actually stuck on how to go about setting the connection using the mongo
driver, can you show me how you set your db connection and then how you go about
using it? (is it pooled?)I just downloaded the mongo.jar and was going to use that, but if you are saying
the ruby driver was good enough I’d rather stick with that for now.
The benchmark in that pastebin is rather poor. It’s dealing with
extremely simple documents. I highly recommend that you tailor the
benchmark to work with documents similar (or exactly like) what you will
use in your actual production application.
I made several benchmarks all of which stressed the system using a much
more complex document (think a hash that contained embedded arrays and
other hashes) and the Ruby driver was faster on JRuby. Be sure to
compare apples to apples.
cr
On Wed, Mar 14, 2012 at 12:07 AM, Chuck R. [email protected]
wrote:
about using it? (is it pooled?)
I made several benchmarks all of which stressed the system using a much more
complex document (think a hash that contained embedded arrays and other
hashes) and the Ruby driver was faster on JRuby. Be sure to compare apples
to apples.cr
I have a real life testcase, pretty big, to test out
mongo-2.6.5.gb1.jar (present in jmongo).
All I was missing was how to initiate the connection properly, which S
Ahmed just shared (thx!).
I’ve to adapt my code, since I’m not using jruby in 1.9 mode (I cannot
use jmongo gem out of the box).
I will report some metrics tomorrow here. ~5x speedup is not something
I can let go, if confirmed 
–
Christian
require “java”
require “benchmark”
require “rubygems”
require “mongo”
require “lib/mongo.jar”
class MongoJava
import “com.mongodb.Mongo”
import “com.mongodb.DBCollection”
import “com.mongodb.BasicDBObject”
import “com.mongodb.DBObject”
def initialize
@db = Mongo.new().get_db(“mongojavatest”)
@coll = @db.get_collection(“testCollection”)
@coll.drop
doc = BasicDBObject.new
doc.put "name", "MongoDB"
@coll.insert(doc)
end
def create
doc = BasicDBObject.new
doc.put “name”, “MongoDB”
@coll.insert(doc)
end
def find
@coll.find_one
end
end
class MongoRuby
def initialize
@db = Mongo::Connection.new.db(“mongorubytest”)
@coll = @db.collection(“testCollection”)
@coll.insert({ “name” => “MongoDB” })
end
def create
@coll.insert({ “name” => “MongoDB” })
end
def find
@coll.find_one
end
end
jdriver = MongoJava.new
rdriver = MongoRuby.new
TIMES = 10_000
Benchmark.bm do |x|
x.report(“java driver (find)”) { TIMES.times { |n| jdriver.find } }
x.report(“ruby driver (find)”) { TIMES.times { |n| rdriver.find } }
x.report(“java driver (create)”) { TIMES.times { |n| jdriver.create }
}
x.report(“ruby driver (create)”) { TIMES.times { |n| rdriver.create }
}
end
On Tue, Mar 13, 2012 at 4:29 PM, Christian MICHON <
On Wed, Mar 14, 2012 at 12:19 AM, Christian MICHON
[email protected] wrote:
All I was missing was how to initiate the connection properly, which S
Ahmed just shared (thx!).I’ve to adapt my code, since I’m not using jruby in 1.9 mode (I cannot
use jmongo gem out of the box).
Interesting findings: it would seem the speedup between jmongo and
mongo on jruby is coming from the same binary/java extension, just
used differently.
jmongo and mongo/bson(1.6.0 jruby) contain both the same extension,
maybe only different versions.
So my guess would be that inside the mongo/bson gems, some extra
cycles are consumed by business logic, that we do not see if we’re
directly using the extension. This is what the benchmark does…
I isolated the speedy behavior to these few lines (no mongo gem loaded):
require “java”
@db = Java::ComMongodb::Mongo.new().get_db(“mongojavatest”)
@coll = @db.get_collection(“testCollection”)
@coll.drop
doc = Java::ComMongodb::BasicDBObject.new
doc.put “name”, “MongoDB”
@coll.insert(doc)
It seems the trick of going through BasicDBObject does save some time.
In the mongo/bson approach, a hash is used, not a BasicDBObject…
I will apply this approach as mentioned on a real life testcase
tomorrow… Wait and see…
–
Christian
On Mar 13, 2012, at 6:45 PM, Christian MICHON wrote:
mongo on jruby is coming from the same binary/java extension, just
I will apply this approach as mentioned on a real life testcase
tomorrow… Wait and see…
That’s why earlier in this thread I suggested using an actual document
for the benchmark instead of the very simple one key/value “name” =>
“MongoDB” hash (or BasicDBObject). If your code is Ruby (and I assume it
is), you’ll have to convert from Ruby objects to the BasicDBObject
(which jmongo handles for you) and that has a speed penalty. It’s also a
better real-world test than using the BasicDBObject directly.
cr
As the current maintainer I can give some history.
In the beginning of the jmongo dev, JRUBY’s decoration of HashMaps
was not as complete as it is now so I paid attention to java → ruby
conversion. This has a time penalty. Times are different now.
I tried to make it as API compatible as the ruby mongo driver as I
could - not a simple wrapper around the java driver.
The java driver handles master/slave differently/tranparently while
the ruby driver is more explicit.
The cursor in jmongo is lazy but the ruby driver caches the
results. AFAIK, if you have the mongodb on the same machine then with
mongo you will have the results in mongo memory and in ruby memory but
not with jmongo. However if you have mongodb on other machines then
jmongo incurs a network overhead to fetch subsequent batches of
results.
The ruby driver did not have a java based BSON implementation as it
has now.
The mongo-2.6.5.gb1.jar (present in jmongo) is my patched version
to overcome https://jira.mongodb.org/browse/JAVA-441
The mongoid team has a slim all ruby driver called moped - tested
under jruby. Using moped might be better than writing a thinner
wrapper around the java driver.
Guy
On Wed, Mar 14, 2012 at 9:46 AM, Guy B. [email protected]
wrote:
The mongoid team has a slim all ruby driver called moped - tested
under jruby. Using moped might be better than writing a thinner
wrapper around the java driver.Guy
But this gem is only 1.9 compliant. My current dev stack is 1.8.7
compliant only, so I could not test it.
Thanks for the pointers.
–
Christian
On Wed, Mar 14, 2012 at 11:10 AM, Christian MICHON
[email protected] wrote:
I’m almost finished in my benchmarks. I’ll post the results shortly.
Here’s the use case: ~1.6GB of packed metadata, unpacked and stored
into ~25 tables.
I compared MongoDB with both drivers/methods discussed in this thread
versus my H2 database.
Please find the results in seconds, minutes and relative speed
improvement over the slowest (H2/tcp):
BasicDBObject 335s 6mn 4x
mongo/bson 594s 10mn 2.4x
h2 embedded 924s 16mn 1.5x
h2 tcp 1420s 24mn 1x
Only drawbacks I saw were:
(I’ve not tried with mongo-sequel though)
–
Christian
Since the java mongo driver has pooling built-in, what is the best way
to
create a instance of this?
This is what I am doing currently, opinions?
class MongoService
def self.mongodb_connection
@@cached_mongodb_connection ||=
Mongo.new().get_db(‘errorlogger’)
end
def self.get_blah_by_id(some_id)
coll = mongodb_connection.get_collection(“my_collection”)
…
…
end
end
On Wed, Mar 14, 2012 at 8:36 AM, Christian MICHON <
On Wed, Mar 14, 2012 at 4:14 AM, Chuck R. [email protected]
wrote:
It seems the trick of going through BasicDBObject does save some time.
In the mongo/bson approach, a hash is used, not a BasicDBObject…I will apply this approach as mentioned on a real life testcase
tomorrow… Wait and see…That’s why earlier in this thread I suggested using an actual document for the
benchmark instead of the very simple one key/value “name” => “MongoDB” hash (or
BasicDBObject). If your code is Ruby (and I assume it is), you’ll have to convert
from Ruby objects to the BasicDBObject (which jmongo handles for you) and that has
a speed penalty. It’s also a better real-world test than using the BasicDBObject
directly.
unfortunately jmongo works only in 1.9 mode, and I’ve to stick to
1.8.7 for the time being. My conversion to BasicDBObject is not
optimal, but it’s almost a 1-liner. I need to look at the constructor
of BasicDBObject to see if I could make it even faster.
I’m almost finished in my benchmarks. I’ll post the results shortly.
–
Christian
Hi, Ahmed, which version of jruby are you using?
I was hoping new versions are doing better, and ran your benchmark with
jruby-1.6.7, the result is quite similar:
$ jruby -S bench_mongo.rb
user system total real
java driver (find) 1.354000 0.000000 1.354000 ( 1.354000)
ruby driver (find) 5.956000 0.000000 5.956000 ( 5.956000)
java driver (create) 0.702000 0.000000 0.702000 ( 0.702000)
ruby driver (create) 1.864000 0.000000 1.864000 ( 1.864000)
$ jruby -S bench_mongo.rb
user system total real
java driver (find) 1.320000 0.000000 1.320000 ( 1.320000)
ruby driver (find) 6.411000 0.000000 6.411000 ( 6.412000)
java driver (create) 0.752000 0.000000 0.752000 ( 0.752000)
ruby driver (create) 2.395000 0.000000 2.395000 ( 2.395000)
$ jruby -S bench_mongo.rb
user system total real
java driver (find) 1.335000 0.000000 1.335000 ( 1.334000)
ruby driver (find) 6.294000 0.000000 6.294000 ( 6.294000)
java driver (create) 0.733000 0.000000 0.733000 ( 0.733000)
ruby driver (create) 2.331000 0.000000 2.331000 ( 2.331000)
And I also tried the comparing with jmongo, it is really faster than the
ruby driver on jruby
$ jruby -S bench_mongo.rb
user system total real
java driver (find) 1.328000 0.000000 1.328000 ( 1.328000)
jmongo driver (find) 4.111000 0.000000 4.111000 ( 4.111000)
java driver (create) 0.449000 0.000000 0.449000 ( 0.449000)
jmongo driver (create) 1.195000 0.000000 1.195000 ( 1.195000)
$ jruby -S bench_mongo.rb
user system total real
java driver (find) 1.323000 0.000000 1.323000 ( 1.323000)
jmongo driver (find) 4.137000 0.000000 4.137000 ( 4.137000)
java driver (create) 0.463000 0.000000 0.463000 ( 0.462000)
jmongo driver (create) 1.182000 0.000000 1.182000 ( 1.183000)
$ jruby -S bench_mongo.rb
user system total real
java driver (find) 1.337000 0.000000 1.337000 ( 1.337000)
jmongo driver (find) 4.267000 0.000000 4.267000 ( 4.267000)
java driver (create) 0.445000 0.000000 0.445000 ( 0.445000)
jmongo driver (create) 1.195000 0.000000 1.195000 ( 1.195000)
Cheers,
Larry Zhao
On Mar 15, 2012, at 1:49 AM, Larry Zhao wrote:
And I also tried the comparing with jmongo, it is really faster than the ruby
driver on jruby
Please don’t draw any conclusions from that particular benchmark. It’s
terrible. The main problem with it is that it avoids the conversion from
a Ruby data structure (e.g. Hash) to the BasicDBObject equivalent.
cr
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.
Sponsor our Newsletter | Privacy Policy | Terms of Service | Remote Ruby Jobs