# Webrick fails with large files **URL:** \ **Category:** Ruby\ **Created:** [February 27, 2007, 10:36pm UTC](https://www.ruby-forum.com/t/webrick-fails-with-large-files/87063 "2007-02-27T22:36:26Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![rogerdpack](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@rogerdpack](https://www.ruby-forum.com/u/rogerdpack)\ **Post date:** [February 27, 2007, 10:36pm UTC](https://www.ruby-forum.com/t/webrick-fails-with-large-files/87063/1 "2007-02-27T22:36:26Z")
So…apparently webrick does the following for serving pages: loads the page into a string object then sends that string to the requestor. Unfortunately this means that if it is a very large file being served that that buffer string will get very large. I.e. with a 700MB file it will fail because Ruby runs out of memory. This fact also defeats the ‘streaming’ aspect of some ruby functions, for example the RoR send\_file has an option to send a file ‘a chunk at a time’ to the client–however these chunks are all conglomerated within webrick then sent–so it doesn’t stream out as it hoped to be doing. I therefore see this as a bug in Webrick and was wondering what others thought. Cheers! -Roger ---
**Author:** ![rogerdpack](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@rogerdpack](https://www.ruby-forum.com/u/rogerdpack)\ **Post date:** [February 28, 2007, 12:27am UTC](https://www.ruby-forum.com/t/webrick-fails-with-large-files/87063/2 "2007-02-28T00:27:53Z")
On Feb 27, 2007, at 13:36, Roger P. wrote: > So…apparently webrick does the following for serving pages: > loads the page into a string object > then sends that string to the requestor. > Unfortunately this means that if it is a very large file being served > that that buffer string will get very large. Don’t dynamically build and send large strings. Use an IO instead. WEBrick knows the difference and responds accordingly. ---
**Author:** ![rogerdpack](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@rogerdpack](https://www.ruby-forum.com/u/rogerdpack)\ **Post date:** [February 28, 2007, 12:34am UTC](https://www.ruby-forum.com/t/webrick-fails-with-large-files/87063/3 "2007-02-28T00:34:10Z")
Eric H. wrote: > On Feb 27, 2007, at 13:36, Roger P. wrote: > > > So…apparently webrick does the following for serving pages: > > loads the page into a string object > > then sends that string to the requestor. > > Unfortunately this means that if it is a very large file being served > > that that buffer string will get very large. > > Don’t dynamically build and send large strings. Use an IO instead. > WEBrick knows the difference and responds accordingly. I believe that rails must use strings, then–any ideas for that case? Thanks! ---
**Author:** ![rogerdpack](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@rogerdpack](https://www.ruby-forum.com/u/rogerdpack)\ **Post date:** [February 28, 2007, 2:00am UTC](https://www.ruby-forum.com/t/webrick-fails-with-large-files/87063/4 "2007-02-28T02:00:59Z")
> > Don’t dynamically build and send large strings. Use an IO instead. > > WEBrick knows the difference and responds accordingly. > > I believe that rails must use strings, then–any ideas for that case? > Thanks! Don’t use webrick for any serious hosting - sending out a 700mb file constitutes serious hosting. I’d say your best approach would be to setup a mongrel cluster running behind apache and then either a: a) let apache serve the file and never hit ruby code or b) install mod\_x\_sendfile into apache and just have your code set the appropriate header to let apache do the lifting. You could also do the same thing behind lighttpd. There’s many howtos about on lighttpd+mongrel and apache+mongrel - google about. Dan. ---
**Author:** ![rogerdpack](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@rogerdpack](https://www.ruby-forum.com/u/rogerdpack)\ **Post date:** [February 28, 2007, 9:45am UTC](https://www.ruby-forum.com/t/webrick-fails-with-large-files/87063/5 "2007-02-28T09:45:41Z")
On Wed, Feb 28, 2007 at 12:53:18PM +0900, Mat S. wrote: > > has an option to send a file ‘a chunk at a time’ to the client– > > however > > these chunks are all conglomerated within webrick then sent–so it > > doesn’t stream out as it hoped to be doing. I therefore see this as a > > bug in Webrick and was wondering what others thought. > > Cheers! > > -Roger > > I doubt Webrick was ever really intended for that sort of work. You > could try Mongrel, although it may yield the same result. Rock solid > ruby deployment is still something of a work in progress, I feel. Or run your code as a fastcgi under Apache. This doesn’t stop you from trying to read a 700MB file into a string of course, but it does give you the option to simply open the file, read it chunk at a time and squirt it to STDOUT. I believe Rails will run happily under fastcgi. You’ll just need to tell it not to render anything if you’re generating your own HTTP headers and body directly. Fastcgi also has the advantage of automatically making your program thread safe, since each instance is a separate process. The downside is that if you’re handling (say) 5 concurrent client requests, you’ll have five Ruby+Rails processes spawned from scratch, each with their own memory footprint. Regards, Brian. ---
**Author:** ![rogerdpack](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@rogerdpack](https://www.ruby-forum.com/u/rogerdpack)\ **Post date:** [February 28, 2007, 4:54am UTC](https://www.ruby-forum.com/t/webrick-fails-with-large-files/87063/6 "2007-02-28T04:54:17Z")
On Feb 27, 2007, at 4:36 PM, Roger P. wrote: > however > these chunks are all conglomerated within webrick then sent–so it > doesn’t stream out as it hoped to be doing. I therefore see this as a > bug in Webrick and was wondering what others thought. > Cheers! > -Roger I doubt Webrick was ever really intended for that sort of work. You could try Mongrel, although it may yield the same result. Rock solid ruby deployment is still something of a work in progress, I feel. -Mat