# Tuning nginx on EC2 **URL:** \ **Category:** NGINX\ **Created:** [December 11, 2010, 6:32am UTC](https://www.ruby-forum.com/t/tuning-nginx-on-ec2/200426 "2010-12-11T06:32:48Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Dobai-Pataky\_BSSSSl](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/77aa72/32.png) [@Dobai-Pataky\_BSSSSl](https://www.ruby-forum.com/u/Dobai-Pataky_BSSSSl)\ **Post date:** [December 11, 2010, 6:32am UTC](https://www.ruby-forum.com/t/tuning-nginx-on-ec2/200426/1 "2010-12-11T06:32:48Z")
Hi, I’ve been doing some benchmarks with nginx on EC2 and have run into a bit of a wall. I’m trying to establish a baseline from which to compare my application against. For this I’m using the simplest possible nginx config: server { listen 80; access\_log off; return 204; } With a large instance (1gbps nic), I’ve only been able to get up to about 12k reqs/sec despite the fact that all resources (cpu/mem/disk/network) appear to be under-utilized. My base platform is Ubuntu 10.04 with nginx 0.8.53. I’ve tried generating load from multiple nodes using a variety of tools, but still can’t seem to push passed 12k. Perhaps there’s just some limit within EC2 that I’m unaware of. I’ve tried tweaking lots of sysctl settings to no end, but it could be I just haven’t found the correct one yet. Has anyone had any luck getting higher throughputs on ec2? regards, chetan ---
**Author:** ![Chetan\_Sarva](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/e5b9ba/32.png) [@Chetan\_Sarva](https://www.ruby-forum.com/u/Chetan_Sarva)\ **Post date:** [December 11, 2010, 3:14pm UTC](https://www.ruby-forum.com/t/tuning-nginx-on-ec2/200426/2 "2010-12-11T15:14:11Z")
On 12/11/2010 06:31 AM, Chetan Sarva wrote: > ``` > return 204; > > ``` > > I’ve tried tweaking lots of sysctl settings to no end, but it could be > I just haven’t found the correct one yet. Has anyone had any luck > getting higher throughputs on ec2? I don’t have any direct experience with services like EC2 but remember that before the traffic hits your machine it first has to pass through various routers and switches in Amazons infrastructure that is shared between probably millions of EC2 instances. You might either hit a bottleneck on these shared components or maybe Amazon just has put some limitations in place to prevent abuse. Regards, Dennis ---
**Author:** ![Chetan\_Sarva](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/e5b9ba/32.png) [@Chetan\_Sarva](https://www.ruby-forum.com/u/Chetan_Sarva)\ **Post date:** [December 11, 2010, 5:54pm UTC](https://www.ruby-forum.com/t/tuning-nginx-on-ec2/200426/3 "2010-12-11T17:54:15Z")
On Sat, Dec 11, 2010 at 9:12 AM, Dennis J. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > I don’t have any direct experience with services like EC2 but remember that > before the traffic hits your machine it first has to pass through various > routers and switches in Amazons infrastructure that is shared between > probably millions of EC2 instances. You might either hit a bottleneck on > these shared components or maybe Amazon just has put some limitations in > place to prevent abuse. With shared resources you would expect to see a range of throughput numbers from when resources are constrained and when they are not. On EC2, disk & network I/O is a shared resource and both can be measured. I’ve done raw network throughput tests and they confirm this. With my nginx tests (and, to be fair, other webservers as well), I consistently get around 12k reqs/sec. ---
**Author:** ![Chetan\_Sarva](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/e5b9ba/32.png) [@Chetan\_Sarva](https://www.ruby-forum.com/u/Chetan_Sarva)\ **Post date:** [December 11, 2010, 8:21pm UTC](https://www.ruby-forum.com/t/tuning-nginx-on-ec2/200426/4 "2010-12-11T20:21:37Z")
Hi! May I ask which type of instance where you using in EC2 ? Thanks, Guzman ---
**Author:** ![Chetan\_Sarva](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/e5b9ba/32.png) [@Chetan\_Sarva](https://www.ruby-forum.com/u/Chetan_Sarva)\ **Post date:** [December 11, 2010, 8:32pm UTC](https://www.ruby-forum.com/t/tuning-nginx-on-ec2/200426/5 "2010-12-11T20:32:17Z")
On Sat, Dec 11, 2010 at 2:20 PM, Guzman B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Hi! > > May I ask which type of instance where you using in EC2 ? A large instance (m1.large) in us-east-1c. I also tried one of the cluster compute instances with 10gbps nic and hit a limit around 22k reqs/sec with similar nginx config (though platform was CentOS this time). ---
**Author:** ![Chetan\_Sarva](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/e5b9ba/32.png) [@Chetan\_Sarva](https://www.ruby-forum.com/u/Chetan_Sarva)\ **Post date:** [December 12, 2010, 11:29am UTC](https://www.ruby-forum.com/t/tuning-nginx-on-ec2/200426/6 "2010-12-12T11:29:42Z")
At Sat, 11 Dec 2010 11:53:17 -0500, Chetan Sarva wrote: > [mailman.nginx.org archive](http://nginx.org/mailman/listinfo/nginx) > There are some things you have to consider, when doing network > benchmarks: 1. bandwidth (as you were told) - you can test with iperf 2. rtt 3. statefull firewall (statefull NAT goes in here too) - big penalty especially when every new request is a new connection and you are using keepalive 4. OS max and per nginx process limits for file descriptors 5. network buffers 6. polling 7. backlog 8. tcp fin timeout 9. nginx number of processes 10. nginx backlog per socket 11. nginx regexp in locations, ifs and so on… 12. nginx keepalive? this is important when you are benchmarking req/s, becase a) 3 way tcp handshake takes some time, closing connections too, b) requests cannot be parallelized per connection good luck with benchmarking 🙂 ---
**Author:** ![Chetan\_Sarva](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/e5b9ba/32.png) [@Chetan\_Sarva](https://www.ruby-forum.com/u/Chetan_Sarva)\ **Post date:** [December 22, 2010, 5:34pm UTC](https://www.ruby-forum.com/t/tuning-nginx-on-ec2/200426/7 "2010-12-22T17:34:07Z")
For the record, it turns out that the limiting factor is the throughput of the virtual NIC itself. Specifically, it can push a max of about 100k packets/sec. Without some improvements in that area, it seems the max requests possible on a standard ec2 instance is about 12k/sec (a 10gbps instance can actually do about double that number).