I posted this at the mongrel forum, but got no response.,
I believe the problems isn’t in mongrel itself, so maybe somebody here
has some info on this.
Anyway, here is the message:
We have a server with 10 running mongrel_cluster instances with apache
in front of them, and every now and then one or some of them hang.
No activity is seen in the database (we’re using activerecord sessions).
Mysql with innodb tables. show innodb status shows no locks. show
processlist shows nothing.
The server is linux debian 4.0
Ruby is: ruby 1.8.6 (2008-03-03 patchlevel 114) [i486-linux]
Rails is: Rails 1.1.2 (yes, quite old)
We’re using the native mysql connector (gem install mysql)
“strace -p PID” gives the following in a loop for the hung mongrel
process:
gettimeofday({1219834026, 235289}, NULL) = 0
select(4, [3], [0], [], {0, 905241}) = -1 EBADF (Bad file descriptor)
gettimeofday({1219834026, 235477}, NULL) = 0
select(4, [3], [0], [], {0, 905053}) = -1 EBADF (Bad file descriptor)
gettimeofday({1219834026, 235654}, NULL) = 0
select(4, [3], [0], [], {0, 904875}) = -1 EBADF (Bad file descriptor)
gettimeofday({1219834026, 235829}, NULL) = 0
select(4, [3], [0], [], {0, 904700}) = -1 EBADF (Bad file descriptor)
gettimeofday({1219834026, 236017}, NULL) = 0
select(4, [3], [0], [], {0, 904513}) = -1 EBADF (Bad file descriptor)
gettimeofday({1219834026, 236192}, NULL) = 0
select(4, [3], [0], [], {0, 904338}) = -1 EBADF (Bad file descriptor)
gettimeofday({1219834026, 236367}, NULL) = 0
…
I used lsof and found that the process used 67 file descriptors (lsof -p
PID |wc -l)
Is there any other way I can debug this, so that I could for example
determine which file descriptor is “bad”?
Any other info or suggestions? Anybody else seen this?
The site is fairly used, but not overly so, load averages usually around
0.3.
Some additional info. I installed mongrel_proctitle to show what the
hung processes were doing, and it seems they are hanging on a method
that displays images using file_column / images from the database /
rmagick to resize and make the images greyscale. Not conclusive the
problem is here, but it is a suspicion.
Is there something obviously wrong with the following? The method
displays a static image if the order doesn’t contain an image, else the
image resized from the order. The cache stuff is so that the image gets
updated in the browser every time. The image is inserted in the page
with a normal image tag.
code:
def preview_image
@order = session[:order]
if @order.image.nil?
@headers[‘Pragma’] = ‘no-cache’
@headers[‘Cache-Control’] = ‘no-cache, must-revalidate’
send_data(EMPTY_PIC.to_blob, :filename => “img.jpg”, :type =>
“image/jpeg”, :disposition => “inline”)
else
@pic = Image.read(@order.image)[0]
if (@order.crop)
@pic.crop!(@order.crop[:x1].to_i, @order.crop[:y1].to_i,
@order.crop[:width].to_i, @order.crop[:height].to_i, true)
end
@pic.resize!(103,130)
@pic = @pic.quantize(256, Magick::GRAYColorspace)
@headers[‘Pragma’] = ‘no-cache’
@headers[‘Cache-Control’] = ‘no-cache, must-revalidate’
send_data(@pic.to_blob, :filename => “img.jpg”, :type =>
“image/jpeg”, :disposition => “inline”)
end
end