I have an app that is using sendfile to return large files to customers.
We can’t use regular Apache to handle this static content since the
files are what the customers are paying for - so it wouldn’t be cool for
anyone else to get access to them.
The issue is that the mongrel mem footprint gets bloated when the files
are transferred and this mem does not get returned to the system.
I’ve looked into x-sendfile, but it doesn’t appear to be compatible with
Apache 2.2.
Now the question, does the current mongrel 1.1.1 have this same issue
with sendfile and memory bloating? (I’m currently running 1.0.1)
Are there any other recommendations for handling this situation?
And wouldn’t it be possible to intercept all download requests and force
use of a cookie/session authorization requirement? I do it on one of my
websites. The X_SendFile feature doesn’t actually send the server
location of the file and if you used a url such as …
/download/ is an action in one of my controllers and it requires a valid
login session. I do a query on the filename and if you’re not
authorized for the file you don’t get to download. All downloads are
stored outside the website directory so no direct linking is available.
Then I use the x_send_file feature to have apache serve up the file
without going through mongrel.
There’s also a nice x_send_file plugin that replaces “send_file” with an
“x_send_file” call, so code updates should be at a minimum.
That’s also where I got mine. I’m running Apache 2.2.6 on FreeBSD 6.x
and compiled it as instructed on that page.
apxs -cia mod_xsendfile.c
My linux-fu is moderate at best so I was lucky I didn’t experience any
errors compiling. Otherwise I’d be spending a day in Google finding
answers. Good luck.
Before I discovered x-sendfile, I used to handle file uploads with
PHP – talking to the same database as my main Rails app. PHP doesn’t
bloat when dumping files across the wire like Ruby does.
Remember: you don’t need to use Rails / Mongrel for everything –
sometimes switching to another platform to solve a particular problem
can be the easiest, most robust solution.
But in this case: x-sendfile FTW. Deny from all in your files
directory, and apache will send via x-sendfile but won’t allow
clients to download them directly.
Before I discovered x-sendfile, I used to handle file uploads with
PHP – talking to the same database as my main Rails app. PHP doesn’t
bloat when dumping files across the wire like Ruby does.
No change in the memory footprint, despite running almost a gigabyte
of file data through it.
One should see essentially the same thing with Mongrel’s send_file()
method, because it reads the file in Const::CHUNK_SIZE pieces (16k).
However…if you are using send_file() from within Rails, then Rails
has to read the entire file into the response before it ever goes back
to Mongrel, and THAT is what is causing the memory bloat. Not Ruby.
Not Mongrel. Rails.
I agree with the other pieces of advice that have been given, though.
Leverage Apache’s x-sendfile since you are on Apache. Alternately,
write a dedicated mongrel handler that will deal with authentication &
file delivery outside of Rails (which also means outside of the Rails
mutex lock, which is a BIG win with regard to scalability of
downloads, at least compared to doing it with send_file from inside of
Rails).
One should see essentially the same thing with Mongrel’s send_file()
method, because it reads the file in Const::CHUNK_SIZE pieces (16k).
However…if you are using send_file() from within Rails, then Rails
has to read the entire file into the response before it ever goes back
to Mongrel, and THAT is what is causing the memory bloat. Not Ruby.
Not Mongrel. Rails.
The rails mongrel handler actually. It buffers all the content from
rails and sends it after the request is done. Rails supports
streaming just fine with FCGI afaik. This is actually something I’d
like to fix (and bring the rails handler into rails) actually, but I’m
not totally clear why this was done. A thread for another time…
+1 for x-send-file or accel-redirect if you’re using nginx.
Matt how do i force apche to handle downlaod request??
Wow. This is an ancient thread that you just hijacked.
This is 100000% an Apache configuration issue. It doesn’t have
anything to do with Mongrel, or with Ruby in the other thread on
ruby-talk that you’ve been beating Luis up on.
I use X_SendFile with Apache 2.2 just fine. �FYI.
Matt how do i force apche to handle downlaod request??
Wow. This is an ancient thread that you just hijacked.
This is 100000% an Apache configuration issue. It doesn’t have
anything to do with Mongrel, or with Ruby in the other thread on
ruby-talk that you’ve been beating Luis up on.
Kirk H.
Kirk do u have any idea ,how should resolved this problem
how do configure apache to handle this download request???
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.