Re: Image Hosting

Hi Maxim,

With this config, my iowait in increasing. 38.4%wa

before that, maximum is about 15%wa


From: Indo P. [email protected]
To: [email protected]
Sent: Thu, October 14, 2010 9:39:29 PM
Subject: Re: Image Hosting

Hi Maxim,

I’m trying your suggestions

Here’s the config that I used right now
worker_processes 10;
worker_rlimit_nofile 50000;
error_log /dev/null;

pid /etc/nginx/logs/nginx.pid;

events {
worker_connections 50000;
use epoll;
}

http {
include mime.types;
default_type application/octet-stream;

    client_header_timeout           1m;
     client_body_timeout             1m;
    client_header_buffer_size       24k;
    large_client_header_buffers     32 256k;
    send_timeout                    1m;
    ignore_invalid_headers          on;
    keepalive_timeout               5;
    output_buffers   1  128k;
    directio   1m;
    server_tokens   off;
    location ~*

^.+.(jpg|jpeg|png|gif|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js|swf)$
{
aio on;
directio 1;
access_log off;
expires max;
}

    location / {
            aio  on;
            directio 1;
            try_files $uri $uri/ /index.php?$uri;
    }


    location ~ \.php$ {
            fastcgi_pass   127.0.0.1:9000;
            fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME $document_root/index.php;
             include        /etc/nginx/conf/fastcgi_params;
    }

From: Maxim D. [email protected]
To: [email protected]
Sent: Thu, October 14, 2010 8:46:57 PM
Subject: Re: Image Hosting

Hello!

On Thu, Oct 14, 2010 at 12:26:59PM +0000, [email protected] wrote:

Actually the image size is arround 100KB each. The server is running in 250Mbps
traffic.

I already described the disk I’m using is scsi 15K RPM in raid 0

Basic tunings you have to apply when serving static which doesn’t
fit into memory are:

If you use sendfile:

  • Make sure your OS uses appropriate read-ahead for sendfile to
    avoid trashing disks with small requests (and seeks). For FreeBSD
    8.1+ it should be enough to set read_ahead directive in nginx
    config (0.8.18+).

http://sysoev.ru/nginx/docs/http/ngx_http_core_module.html#read_ahead
(in Russian)

Using bigger socket buffers (listen … sndbuf=… in nginx
config) may help too.

  • If serving large files - make sure you use appropriate
    sendfile_max_chunk to avoid blocking nginx worker on disk for too
    long.

  • Consider switching sendfile off if you can’t persuade it to read
    large blocks from disk.

If not using sendfile:

  • Tune output_buffers (again, to avoid trashing disks with small
    requests and seeks). Default is 2x32k, which will result in 4
    disk requests for 100k file. Changing it to 1x128k would result
    in 2x memory usage but 4x less disk requests, this is probably
    good thing to do if you are disk-bound.

In both cases:

  • Using aio may help a lot (“aio sendfile” is only available under
    FreeBSD) by adding more concurency to disk load and generally
    improving nginx interactivity. Though right now it requires
    patches to avoid socket leaks, see here:

http://nginx.org/pipermail/nginx-devel/2010-October/000498.html

  • Using directio may help to improve disk cache effictiveness by
    excluding large files (if you have some) from cache. Though keep
    in mind that it disables sendfile so if you generally tune for
    sendfile - you may have to apply output_buffers tunings as well.

It’s hard to say anything more than this without knowing lots of
details.

Maxim D.

Hello!

On Thu, Oct 14, 2010 at 07:54:14AM -0700, Indo P. wrote:

Hi Maxim,

With this config, my iowait in increasing. 38.4%wa

before that, maximum is about 15%wa

The question isn’t iowait, but how well your server behaves, i.e.
how many requests it can serve. You may want to monitor
throughput and latency instead.

On the other hand, under linux using aio may be bad idea if your
disk cache isn’t completely trashed and still able to cache
something as aio on linux requires directio.

Maxim D.

Hi Maxim,

Is it the time for me to move to FreeBSD?

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs