# Re: Image Hosting

**URL:** <https://www.ruby-forum.com/t/re-image-hosting/197453>\
**Category:** NGINX\
**Created:** [October 14, 2010, 4:55pm UTC](https://www.ruby-forum.com/t/re-image-hosting/197453 "2010-10-14T16:55:15Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![okkezSS](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/o/85f322/32.png) [@okkezSS](https://www.ruby-forum.com/u/okkezSS)\
**Post date:** [October 14, 2010, 4:55pm UTC](https://www.ruby-forum.com/t/re-image-hosting/197453/1 "2010-10-14T16:55:15Z")

</div>

Hi Maxim,

With this config, my iowait in increasing. 38.4%wa

before that, maximum is about 15%wa

* * *

From: Indo P. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
To: removed\_email\_address@domain.invalid  
Sent: Thu, October 14, 2010 9:39:29 PM  
Subject: Re: Image Hosting

Hi Maxim,

I’m trying your suggestions

Here’s the config that I used right now  
worker\_processes 10;  
worker\_rlimit\_nofile 50000;  
error\_log /dev/null;

pid /etc/nginx/logs/nginx.pid;

events {  
worker\_connections 50000;  
use epoll;  
}

http {  
include mime.types;  
default\_type application/octet-stream;

```
    client_header_timeout 1m;
     client_body_timeout 1m;
    client_header_buffer_size 24k;
    large_client_header_buffers 32 256k;
    send_timeout 1m;
    ignore_invalid_headers on;
    keepalive_timeout 5;
    output_buffers 1 128k;
    directio 1m;
    server_tokens off;
    location ~*

```

^.+.(jpg|jpeg|png|gif|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js|swf)$  
{  
aio on;  
directio 1;  
access\_log off;  
expires max;  
}

```
    location / {
            aio on;
            directio 1;
            try_files $uri $uri/ /index.php?$uri;
    }

    location ~ \.php$ {
            fastcgi_pass 127.0.0.1:9000;
            fastcgi_index index.php;
            fastcgi_param SCRIPT_FILENAME $document_root/index.php;
             include /etc/nginx/conf/fastcgi_params;
    }

```

* * *

From: Maxim D. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
To: removed\_email\_address@domain.invalid  
Sent: Thu, October 14, 2010 8:46:57 PM  
Subject: Re: Image Hosting

Hello!

On Thu, Oct 14, 2010 at 12:26:59PM +0000, removed\_email\_address@domain.invalid wrote:

> Actually the image size is arround 100KB each. The server is running in 250Mbps  
> traffic.
> 
> I already described the disk I’m using is scsi 15K RPM in raid 0

Basic tunings you have to apply when serving static which doesn’t  
fit into memory are:

If you use sendfile:

- Make sure your OS uses appropriate read-ahead for sendfile to  
avoid trashing disks with small requests (and seeks). For FreeBSD  
8.1+ it should be enough to set read\_ahead directive in nginx  
config (0.8.18+).

[http://sysoev.ru/nginx/docs/http/ngx\_http\_core\_module.html#read\_ahead](http://sysoev.ru/nginx/docs/http/ngx_http_core_module.html#read_ahead)  
(in Russian)

Using bigger socket buffers (listen … sndbuf=… in nginx  
config) may help too.

- If serving large files - make sure you use appropriate  
sendfile\_max\_chunk to avoid blocking nginx worker on disk for too  
long.

- Consider switching sendfile off if you can’t persuade it to read  
large blocks from disk.

If not using sendfile:

- Tune output\_buffers (again, to avoid trashing disks with small  
requests and seeks). Default is 2x32k, which will result in 4  
disk requests for 100k file. Changing it to 1x128k would result  
in 2x memory usage but 4x less disk requests, this is probably  
good thing to do if you are disk-bound.

In both cases:

- Using aio may help a lot (“aio sendfile” is only available under  
FreeBSD) by adding more concurency to disk load and generally  
improving nginx interactivity. Though right now it requires  
patches to avoid socket leaks, see here:

[http://nginx.org/pipermail/nginx-devel/2010-October/000498.html](http://nginx.org/pipermail/nginx-devel/2010-October/000498.html)

- Using directio may help to improve disk cache effictiveness by  
excluding large files (if you have some) from cache. Though keep  
in mind that it disables sendfile so if you generally tune for  
sendfile - you may have to apply output\_buffers tunings as well.

It’s hard to say anything more than this without knowing lots of  
details.

Maxim D.

---

<div class="post-metadata">

**Author:** ![Indo\_P](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/ce7236/32.png) [@Indo\_P](https://www.ruby-forum.com/u/Indo_P)\
**Post date:** [October 14, 2010, 7:37pm UTC](https://www.ruby-forum.com/t/re-image-hosting/197453/2 "2010-10-14T19:37:12Z")

</div>

Hello!

On Thu, Oct 14, 2010 at 07:54:14AM -0700, Indo P. wrote:

> Hi Maxim,
> 
> With this config, my iowait in increasing. 38.4%wa
> 
> before that, maximum is about 15%wa

The question isn’t iowait, but how well your server behaves, i.e.  
how many requests it can serve. You may want to monitor  
throughput and latency instead.

On the other hand, under linux using aio may be bad idea if your  
disk cache isn’t completely trashed and still able to cache  
something as aio on linux requires directio.

Maxim D.

---

<div class="post-metadata">

**Author:** ![Indo\_P](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/ce7236/32.png) [@Indo\_P](https://www.ruby-forum.com/u/Indo_P)\
**Post date:** [October 15, 2010, 3:13am UTC](https://www.ruby-forum.com/t/re-image-hosting/197453/3 "2010-10-15T03:13:17Z")

</div>

Hi Maxim,

Is it the time for me to move to FreeBSD?