We have a very frustrating problem with a NAS configuration that we
recently set-up. After running a filesharing website on 8 servers for 1+
years we decided to switch to a NAS configuration. We set-up our NAS
(24x1TB Raid 5) to serve files to 9 front end servers over a private
switch - doing 9 servers on 1 disk(logical)… not sure if nginx can
handle this. Since we’ve moved to the NAS BW usage has dropped
significantly even though our traffic has stayed the same. Download
speeds are pretty bad aswell but this is due to IO issues that are being
created by NGINX. so here’s the problem we are facing…
In theory nginx is creating an io bottleneck because it’s opening to
many times without closing. We’re using nginx on our front end servers
to grab files off our nas - each file will stay alive though for 45+
minutes. Instead of opening it for 10 seconds, reading into memory, and
closing - it’s opening it for 10 seconds, reading into memory, streaming
it for 45 minutes, THEN closing. We ran a test with nginx and we were
only able to download at 30kbps on port 182, how-ever over apache we
were able to reach speeds of 20mb/s.
We’ve re-compiled nignx and installed the most recent stable version but
that didn’t fix anything.
Can someone please suggest a fix here? I’ve ran out of ideas…
On Sat, Oct 22, 2011 at 08:29:05PM +0200, Graham D. wrote:
created by NGINX. so here’s the problem we are facing…
We’ve re-compiled nignx and installed the most recent stable version but
that didn’t fix anything.
Can someone please suggest a fix here? I’ve ran out of ideas…
Any ideas guys - I need to get this resolved asap.
Otherwise i need to figure another solution for our NSF NAS Nginx
configuration…
Which HW do you have (Networkcard, …)?
Do you have anything in the messages/syslog file about some limits?
Do you use connection track in the iptables?
Forgive my ignorance, what’s “port 182” in your original email?
In your tests did you use a single nginx instance and a single Apache
against that 1 logical disk on Netapp?
Can you show sections on nginx configuration relevant to disk operations
(also, how many workers per nginx server)?
Do you use any specific tunables to adjust the network settings for
Netapp?
How many files on Netapp partition?
What’s your OS?
Hello Andrew, where not using a high end NAS and our OS is Centos 64 Bit
Aleksandar L. wrote in post #1028318:
Which HW do you have (Networkcard, …)?
Do you have anything in the messages/syslog file about some limits?
Do you use connection track in the iptables?
Cheers
Aleks
Hello Aleksandar, Our NAS is working fine and there are no errors in the
logs. The problem is with nginx it self.
What OS do you use ?
Try the following:
worker_processes 32;
http {
sendfile off;
output_buffers 1 1m;
…
}
–
Igor S.
Hello Igor, we are using Centos 64 Bit. We haven’t tried using
output_buffers yet so we will try that shortly…
Right now nginx is keeping the filehandle open after the file is cached
in ram. So we end up with 3,000 filehandles open for the duration of a
45 minute download.