Large(er) File Uploads


#1

Howdy Folks,

My Rails app allows file uploads in several places. Most of the time the
files are pretty small (under 1MB) and we never have any trouble.

There is one section where the files are a bit larger, about (10-15MB).
This
section uses the same code as other places and works with small files.

It breaks with larger files… This is what happens on both server
(FCGI/Apache) and local (WebBrick):

Upload the file and click ‘Submit’. The gears turn, looks fine… Then
it
just chugs along, forever. Nothing happens, it just sits there. Apache
will
go forever, WebBrick eventually dies with an error that it can’t find
the
action (strange error).

So, does anyone have any suggestions (other than ‘don’t use HTTP for
uploads
of this size’). I realize that FTP and other solutions are better for
large
files but in my opinion this file isn’t all that large and I’m pretty
sure
that other folks have accomplished this… Strongspace is a Rails app
and I
believe handles this situation.

Any advice is very much appreciated.

Cheers.


#2

If you’re using MySQL the default max_allowed_packet size is set to 1M,
meaning that
you cannot transfer/receive records larger than 1 megabyte. Resetting
this
in the my.ini file to something like 8M and restarting the db may fix
your problem.

max_allowed_packet=8M


#3

Thanks.

I’m actually not on MySQL, we’re on Postgres… But these files are
being
stored in the filesystem, not the database so I don’t think that’s
coming
into play.


#4

if you’re running apache you may want to check directives:

KeepAlive
KeepAliveTimeout
MaxKeepAliveRequests
RLimitCPU max
RLimitMEM max


#5

Thanks, I will.

I do notice the following errors in my FCGI crash log:

[09/Dec/2005:10:40:51 :: 20136] Dispatcher failed to catch: protocol
error
(FCGI::Stream::ProtocolError)
/usr/local/lib/ruby/1.8/cgi.rb:1015:in read' /usr/local/lib/ruby/1.8/cgi.rb:1015:inread_multipart’
/usr/local/lib/ruby/1.8/cgi.rb:983:in loop' /usr/local/lib/ruby/1.8/cgi.rb:983:inread_multipart’

/usr/local/lib/ruby/gems/1.8/gems/actionpack-1.11.1/lib/action_controller/cg
i_ext/raw_post_data_fix.rb:20:in initialize_query' /usr/local/lib/ruby/1.8/cgi.rb:2269:ininitialize’
(eval):16:in initialize' /usr/local/lib/ruby/gems/1.8/gems/fcgi-0.8.6.1/./fcgi.rb:600:innew’
/usr/local/lib/ruby/gems/1.8/gems/fcgi-0.8.6.1/./fcgi.rb:600:in
each_cgi' /usr/local/lib/ruby/gems/1.8/gems/fcgi-0.8.6.1/./fcgi.rb:597:ineach’
/usr/local/lib/ruby/gems/1.8/gems/fcgi-0.8.6.1/./fcgi.rb:597:in
each_cgi' /usr/local/lib/ruby/gems/1.8/gems/rails-0.14.4/lib/fcgi_handler.rb:52:inprocess!’
/usr/local/lib/ruby/gems/1.8/gems/rails-0.14.4/lib/fcgi_handler.rb:22:in
`process!’
/home/www/myapp/apps/myapp/public/dispatch.fcgi:24
almost killed by this error

This occurs when the upload is in process.