Forum: NGINX Really high disk i/o!

13731c628e20ffd83b6114ff2c406272?d=identicon&s=25 shahzaib mushtaq (shahzaib12)
on 2014-06-16 13:49
(Received via mailing list)
Our server HDD i/o is constant on 8MB/s and i/o utilization + await is
very
high due to which nginx video streaming is really slow and we're
receiving
complains from our users regarding slow streaming of the videos.

We're using 12X3TB SATA HDD Hardware-Raid10 with 16G RAM
OS Centos 6.4

 8MB/s w/r should not be issue for 12X3TB SATA HDD. Maybe i need to
tweak
some nginx buffers or kernels in order to reduce the high io wait ?

Could someone point me to the right direction ? We can't afford SAS
Drives
right now and have to go with the SATA.

Linux 2.6.32-431.17.1.el6.x86_64 (storage17)    06/16/2014
_x86_64_        (8 CPU)

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s
avgrq-sz
avgqu-sz   await  svctm  %util
sda             123.66   336.85  118.87    7.67 19496.19  2759.42
175.88     2.40   18.93   6.27  79.35

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s
avgrq-sz
avgqu-sz   await  svctm  %util
sda              75.00     8.67  108.00    2.00 18117.33    85.33
165.48     1.88   17.05   6.86  75.47

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s
avgrq-sz
avgqu-sz   await  svctm  %util
sda              76.00     0.00   94.00    0.33 17192.00     2.67
182.28     1.50   16.04   7.47  70.47

Any help would be highly appreciated.

Regards.
Shahzaib
Ebfa54c92a50798dfdbbdb4e28ed1f29?d=identicon&s=25 Richard Kearsley (Guest)
on 2014-06-16 16:13
(Received via mailing list)
On 16/06/14 12:49, shahzaib shahzaib wrote:
>
>  8MB/s w/r should not be issue for 12X3TB SATA HDD. Maybe i need to
> tweak some nginx buffers or kernels in order to reduce the high io wait ?
>
if you have a high number of concurrent connections and/or use
limit_rate, then expect hdd (sata or sas) to quickly run out of iops
tip from me: use nmon (linux) to see the real time status of your disks.
If they are wrapped up in iowait then you need a faster seeking disk
(e.g. ssd because sas won't give you a significant amount extra)
if only one of them is in iowait then it's probably a dead/dying disk
D9ddf469df826665be182f45010758cd?d=identicon&s=25 Stefan Parvu (Guest)
on 2014-06-16 19:24
(Received via mailing list)
> If they are wrapped up in iowait then you need a faster seeking disk
> (e.g. ssd because sas won't give you a significant amount extra)

or need to tune something. You don't necessarily need to change always
the
hdw if you have high iowaits. There are many variables around.

--
Stefan Parvu <sparvu@systemdatarecorder.org>
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.