Large memory spike when posting XML file

Hi all, I’m trying to track down a memory spike in my application that
happens when processing an XML file that’s uploaded.

I created a new rails 2.3.10 app with a single index action that does
nothing other than rendering some text, and posted a 1.1 meg XML file
to the action. Immediately, my rails passenger process jumps from 85
megs to 1.1 gigs, even though my application is doing absolutely
nothing with the file. If I upload a 150 meg file, it jumps to over 4
gigs of memory, and after I cancel the upload, the memory is not
reclaimed. This is extremely worrying, since it looks like it would
be very easy for someone to launch a DOS attack just by posting a
large file to an action.

Is there something I’m overlooking? In our production app, we’ve got
cancan which is restricting access, but the memory spike happens
before cancan even has a chance to enforce the authorization
restrictions. It seems as though the mere fact of uploading a file
causes rails to parse the data into params which leads to the memory
spike.

Is this normal rails behaviour? Is there a way to prevent this memory
spike from occurring, so that people can’t crash our servers by
posting large files to random actions?

I’m using the following curl command to post the XML:

curl -X POST http://large_file_tester -d @/home/mike/tmp/Product
Upload\ Example\ XMLs/products.1000.xml --user mike:password --header
“Content-Type: application/xml”

My end goal is to figure out a better way of handling these XML files,
maybe Rails Metal or a direct apache upload or something else to avoid
going through rails. If anyone has any suggestions, please let me
know! Thanks,

Mike

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs