Sorry for the non-descriptive title but I have no clue where to even begin looking for a cause here. I've only just started with JRuby, and have attempted to use it with a short script that makes use of the AWS Ruby SDK. It simply attempts to list the entire contents of an S3 bucket: https://gist.github.com/4119222 Not long after starting it through JRuby, it starts spewing exceptions from each thread. Here is a representative example: https://gist.github.com/4119264 I am using the latest AWS SDK for Ruby gem (1.7.1), OpenJDK 7 (have also tried 6) and have tried JRuby 1.7.0, 1.6.8 and jruby-head. All appear to fail with the same error. Any help is appreciated. Best Regards, Oliver
on 2012-11-20 20:49
on 2012-11-20 22:16
Oliver - I admit not to have a clue, but here are some questions I would ask myself in your situation: If I launch only one thread, would the problem persist? I do see that the AWS gem is supposed to be thread safe here (https://forums.aws.amazon.com/thread.jspa?messageI...), but that was a while back. I searched Amazon's forum for help on this and only found your message, at https://forums.aws.amazon.com/thread.jspa?messageI.... I'm posting the link here so that we can look it up. Another question, if I had the time, would be, do I also experience this problem using Oracle's JDK? - Keith
on 2012-11-21 10:20
On 20 November 2012 22:14, Keith Bennett <email@example.com> wrote: > I think you copied and pasted the wrong URL here, but I recall the thread you are talking about and I believe it is this one: https://forums.aws.amazon.com/thread.jspa?messageID=290781 Indeed, reducing the concurrency back down to one thread seems to have removed the problem which seems to suggest that the library is in fact not thread safe. Guess my little adventure with that library and JRuby ends here. > I searched Amazon's forum for help on this and only found your > message, at > https://forums.aws.amazon.com/thread.jspa?messageI.... > I'm posting the link here so that we can look it up. > > Another question, if I had the time, would be, do I also experience > this problem using Oracle's JDK? I would probably try this option, but given the above results I don't think it's important. Correct me if this is the wrong assumption.
on 2012-11-26 17:49
it's unlikely to be the cause of your problem, but I'd suggest you avoid creating all those threads and instead use java's executor api to create a thread pool. check out my tip on jrubytips.com to se how: http://jrubytips.com/posts/8-use-java-s-concurrenc... since the S3 does a lot of IO you can create a thread pool with more threads than you have cpus, try a few different numbers and see what gives you the best performance. also, creating an S3 client for each thread would probably wouldn't hurt. I don't trust the thread safety of any ruby libraries. T#
on 2012-11-26 22:22
Good suggestions Theo, I'll definitely give them a try. I have not as yet made use of any thread pooling but I can already see I'm suffering from this decision (aside from any other performance problem). My context switch rate is accordingly very high, so I'll also implement that.