# \[scalability, performance, DoS\] To or not to process images at runtime **URL:** \ **Category:** Rails\ **Created:** [August 7, 2008, 10:03am UTC](https://www.ruby-forum.com/t/scalability-performance-dos-to-or-not-to-process-images-at-runtime/145265 "2008-08-07T10:03:55Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![mhfs](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/b2d939/32.png) [@mhfs](https://www.ruby-forum.com/u/mhfs)\ **Post date:** [August 7, 2008, 10:03am UTC](https://www.ruby-forum.com/t/scalability-performance-dos-to-or-not-to-process-images-at-runtime/145265/1 "2008-08-07T10:03:55Z")
Hello, My next project will be a kind of online photo viewer. All of these photos will need to have watermark applied to them. The problem is that, depending on the picture, different watermarks need to be applied, exclusivelly (not all in the same image). The easiest solution would be to process these picture at runtime using RMagick, apply the watermark(s) and serve them. The other approach, would be to pre-process them (also using RMagick or the ImageMagick CLI tool) and create different copies on the disk for each watermark, the obvious advantage being that it could be served directly via the webserver (apache) using x-sendfile, but, it would be much harder to manage (need to fix a watermark error? Re-process and re-create the images on the disk…) and would take much more disk space (which could probably be solved by using a distributed filesystem, though). One thing that I thought could work is to have a dedicated (or a cluster) server to process the images, this server (or cluster) would receive a request from the web server(s), process the desired image and return the stream of bytes to the web server that would in turn delegate it back to the client with the right mime/type (image/jpg). The software to process the images could be written in C to better optmize things. This, I think, could solve the CPU/RAM problem for processing the images per-request at runtime, but would result in a rather expensive backend. However, I’m not sure if passing the stream of bytes through Rails is optimal and I think it could be a major bottleneck, since I’ve heard of memory leaks and other hairy bugs when you try to serve big files using Rails - is that true? I would rather process them at runtime (more flexibility (code can decide, per request, which watermark(s) to apply), much less disk space (in the other approach I would have to keep several versions of the pictures for each watermark(s)) per HTTP request, however, this site will probably have lots of traffic. So, I’ve reached a deadend. Could someone share his/her experiences and thoughts and help me decide? 🙂 PS: I’ve put DoS in the subject tagline meaning Denial of Service as I think that maybe dynamic processing of images VS lots of request could result in DoS. Thanks in advance, Marcelo. ---
**Author:** ![mhfs](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/b2d939/32.png) [@mhfs](https://www.ruby-forum.com/u/mhfs)\ **Post date:** [August 7, 2008, 5:38pm UTC](https://www.ruby-forum.com/t/scalability-performance-dos-to-or-not-to-process-images-at-runtime/145265/2 "2008-08-07T17:38:46Z")
Hi, It all depends on amount of visitors/serving images you’ll get. You could try to measure it somehow, to find out if your resources are enough for your app… but for me - if the image would be served more than once, I would save it to disk somewhere and serve using apache/ send\_file for better performance. Especially when images are big and processing time grows fast. Best, H. On Aug 7, 10:03 am, “Marcelo de Moraes S.” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) ---
**Author:** ![mhfs](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/b2d939/32.png) [@mhfs](https://www.ruby-forum.com/u/mhfs)\ **Post date:** [August 8, 2008, 1:23am UTC](https://www.ruby-forum.com/t/scalability-performance-dos-to-or-not-to-process-images-at-runtime/145265/3 "2008-08-08T01:23:03Z")
On Thu, Aug 7, 2008 at 4:03 AM, Marcelo de Moraes S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > on the disk…) and would take much more disk space (which could probably be > solved by using a distributed filesystem, though). Store just the raw images and cache watermarked images, expire the cache as necessary? [http://scottstuff.net/presentations/rails-caching/](http://scottstuff.net/presentations/rails-caching/) "From the Sparklines code in Typo: fragment\_cache = read\_fragment(fragmentname) if(not fragment\_cache) fragment\_cache = Sparklines.plot(ary,params) write\_fragment(fragmentname,fragment\_cache) end send\_data(fragment\_cache, :disposition =\> ‘inline’, :type =\> ‘image/png’)" (slide 25) ---
**Author:** ![mhfs](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/b2d939/32.png) [@mhfs](https://www.ruby-forum.com/u/mhfs)\ **Post date:** [August 8, 2008, 1:34am UTC](https://www.ruby-forum.com/t/scalability-performance-dos-to-or-not-to-process-images-at-runtime/145265/4 "2008-08-08T01:34:45Z")
Wow, this cache thing could work. Can I cache the images for as long as I want? ---
**Author:** ![mhfs](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/b2d939/32.png) [@mhfs](https://www.ruby-forum.com/u/mhfs)\ **Post date:** [August 8, 2008, 9:07pm UTC](https://www.ruby-forum.com/t/scalability-performance-dos-to-or-not-to-process-images-at-runtime/145265/5 "2008-08-08T21:07:35Z")
On Thu, Aug 7, 2008 at 8:33 PM, Marcelo de Moraes S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Wow, this cache thing could work. Can I cache the images for as long as I > want? I don’t see why not. On Thu, Aug 7, 2008 at 8:58 PM, Marcelo de Moraes S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Also, caching these assets (the images) would result serving the data as > fast as x-sendfile ? I have not used the caching capabilities of Rails; my guess is that it might be slightly slower than x-sendfile, but still much faster than dynamically uncached. 🙂 ---
**Author:** ![mhfs](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/b2d939/32.png) [@mhfs](https://www.ruby-forum.com/u/mhfs)\ **Post date:** [August 8, 2008, 2:59am UTC](https://www.ruby-forum.com/t/scalability-performance-dos-to-or-not-to-process-images-at-runtime/145265/6 "2008-08-08T02:59:00Z")
Also, caching these assets (the images) would result serving the data as fast as x-sendfile ? Thanks, Marcelo. On Thu, Aug 7, 2008 at 8:33 PM, Marcelo de Moraes S. \ **Author:** ![mhfs](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/b2d939/32.png) [@mhfs](https://www.ruby-forum.com/u/mhfs)\ **Post date:** [August 8, 2008, 10:45pm UTC](https://www.ruby-forum.com/t/scalability-performance-dos-to-or-not-to-process-images-at-runtime/145265/7 "2008-08-08T22:45:58Z") Thanks, I will have a deeper look into Rails caching. Cheers, Marcelo.