Hello there, I'm running into some concurrency issues and need some help. I'm currently using JRuby 1.7.12 and Rails 4.1.1. Every now and then i'm getting this error: ActiveRecord::ConnectionTimeoutError (could not obtain a database connection within 15.000 seconds (waited 15.001 seconds)): vendor/bundle/jruby/1.9/gems/activerecord-4.1.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:190:in `wait_poll' Our database pool is set to 20 and a checkout_timeout of 15. Has anyone seen this issue before?
on 2015-01-19 22:29
on 2015-01-19 22:44
It sounds like the twenty database connection you are allowing are all in use for more than 15 seconds. Is your Rails server under much load? (Are there many, concurrent requests? Are a bunch of those requests taking a long time (several seconds) to complete?) Is there a chance that your application is leaking database connections? That is, might their be code paths where a connection is taken from the database connection pool, but never returned to the pool? Can you increase the size of your database connection pool? - Bruce P.S. For what it's worth, I recommend a low checkout_timeout, specifically one second. I don't what a user request waiting around for resources. Fifteen seconds is a very long time for a user to wait for a web request. I'd rather give the user (and my own error logs) a message quickly when I run out of resources. With an error message, I get earlier notice that something isn't quite right.
on 2015-01-19 23:12
Hey Bruce, Thanks for the quick response. 1. The rails server is only taking 2-5 connections at this point in time. We're testing atm. * No there are not many concurrent connections. * The requests do take a significant amount of time 20+ seconds. (Returning json data hitting many different apis) 2. I am not aware of any AR calls outside of the typical rails controller actions. * There may be a db leak but how do I tell? Is there a variable, etc I can debug to print that info out? Remaining connections perhaps? 3. I could increase the pool higher but with the info above there is most likely a leak or something going wrong with the connection pool. We are also using threading in the action that is causing the issue. But we have used the "with_connection" block before making any database calls in the thread.
on 2015-01-19 23:34
Chason - Interesting. I've done some fun stuff with concurrency under Rails, but my stuff didn't deal with database connections in threads. So, I don't don't have direct experience to help you here. This (long) blog post, https://bibwild.wordpress.com/2014/07/17/activerec..., looks like it might be relevant to what you are seeing. - Bruce