Forum: JRuby Exceptions thrown when using JRuby 1.7.0 with Amazon SDK

Sorry for the non-descriptive title but I have no clue where to even 
begin
looking for a cause here. I've only just started with JRuby, and have
attempted to use it with a short script that makes use of the AWS Ruby 
SDK.
It simply attempts to list the entire contents of an S3 bucket:

https://gist.github.com/4119222

Not long after starting it through JRuby, it starts spewing exceptions 
from
each thread. Here is a representative example:

https://gist.github.com/4119264

I am using the latest AWS SDK for Ruby gem (1.7.1), OpenJDK 7 (have also
tried 6) and have tried JRuby 1.7.0, 1.6.8 and jruby-head. All appear to
fail with the same error. Any help is appreciated.

Best Regards,
Oliver

Oliver -

I admit not to have a clue, but here are some questions I would ask
myself in your situation:

If I launch only one thread, would the problem persist? I do see that
the AWS gem is supposed to be thread safe here
(https://forums.aws.amazon.com/thread.jspa?messageI...),
but that was a while back.

I searched Amazon's forum for help on this and only found your
message, at 
https://forums.aws.amazon.com/thread.jspa?messageI....
 I'm posting the link here so that we can look it up.

Another question, if I had the time, would be, do I also experience
this problem using Oracle's JDK?

- Keith

On 20 November 2012 22:14, Keith Bennett <keithrbennett@gmail.com> 
wrote:

> I think you copied and pasted the wrong URL here, but I recall the thread
you are talking about and I believe it is this one:

https://forums.aws.amazon.com/thread.jspa?messageID=290781

Indeed, reducing the concurrency back down to one thread seems to have
removed the problem which seems to suggest that the library is in fact 
not
thread safe. Guess my little adventure with that library and JRuby ends
here.


> I searched Amazon's forum for help on this and only found your
> message, at
> https://forums.aws.amazon.com/thread.jspa?messageI....
>  I'm posting the link here so that we can look it up.
>
> Another question, if I had the time, would be, do I also experience
> this problem using Oracle's JDK?


I would probably try this option, but given the above results I don't 
think
it's important. Correct me if this is the wrong assumption.

it's unlikely to be the cause of your problem, but I'd suggest you avoid
creating all those threads and instead use java's executor api to create 
a
thread pool. check out my tip on jrubytips.com to se how:
http://jrubytips.com/posts/8-use-java-s-concurrenc...

since the S3 does a lot of IO you can create a thread pool with more
threads than you have cpus, try a few different numbers and see what 
gives
you the best performance.

also, creating an S3 client for each thread would probably wouldn't 
hurt. I
don't trust the thread safety of any ruby libraries.

T#

Good suggestions Theo, I'll definitely give them a try. I have not as 
yet
made use of any thread pooling but I can already see I'm suffering from
this decision (aside from any other performance problem). My context 
switch
rate is accordingly very high, so I'll also implement that.