We’re using nginx with HAProxy and mongrel as well as monit and we’re
integrating with SAP using Piers H.'s sapnwrfc gem. It contains a
C-extension which calls itself an SAP provided C-library for RFC
calls. In general it works great.
But there are SAP RFCs with very long runtimes, e.g. orders with a lot
of items, searches with not restricted enough search criterias, but
also sporadic hickups in SAP which can happen to any RFC call. They
can block a mongrel for more than 2 minutes. We therefore need a
timeout solution.
The 2 available solutions we found (SystemTimer and Terminator) just
wouldn’t work, no matter what we tried. We therefore wrote an old-
fashioned custom solution: each request writes a file initially and
deletes it again at the end of processing. A monit process which runs
periodically (in our case every 10s) checks whether there are any
files older than a specific timeout period. If it finds one, it kills
the mongrel process (no soft kill) and starts a new one. When the
mongrel gets killed, nginx receives a 504 error. We assume that this
will happen mostly (only ?) in timeout cases, therefore we modified it
to redirect to our app to a page which has an error message about the
timeout.
This solution works perfectly so far. The only weird phenomenon we saw
is that in one case the user never gets the error page/redirect, but
the browser just hangs (firefox and IE).
Comments ? Ideas why SytemTimer and Terminator would not work ?
Improvements to the current solution ?
Adrian Z.
www.b2b2dot0.com
The solution details:
-
Setting a global constant with the mongrel port during
initialization:
begin
ObjectSpace.each_object(Mongrel::HttpServer) do |i|
Const::App.port = i.port
end
rescue
Const::App.port = ‘3000’ # when testing etc.
end -
Creating/deleting the file in before/after filters:
class ApplicationController < ActionController::Base
before_filter :write_timeout_file
after_filter :delete_timeout_file
def write_timeout_file
File.new(“tmp/pids/mongrel.#{Const::App.port}.req”, “w”).close
end
def delete_timeout_file
File.delete(“tmp/pids/mongrel.#{Const::App.port}.req”)
end
end -
Monit Configuration:
check file mongrel.3000.req path /var/www/apps/b2b2dot0/current/tmp/
pid /mongrel.3000.req
if timestamp > 90 seconds
then exec “/export/admin-scripts/kill_mongrel.sh 3000”
mode passive
group mongrel_timeout