Forum: NGINX Nginx flv stream gets too slow on 2000 concurrent connections

Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 10:44
(Received via mailing list)
Hello,

        We are using nginx to serve large size of static files i.e 
jpg,flv
and mp4 . Nginx stream works very well on 1000~1500 concurrent 
connections
but whenever connections exceeded to 2000~2200, stream gets too slow. 
We've
five content server with following specification:-

Dual Quard Core (8cores/16threads)
RAM = 32G
HDD = Sas Hard-Raid 10


My nginx.conf config is given below :

user  nginx;
worker_processes  16;
worker_rlimit_nofile 300000; #2 filehandlers for each connection;

#pid        logs/nginx.pid;


events {
    worker_connections  6000;
    use epoll;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    limit_rate 180k;
    client_body_buffer_size 128K;
    sendfile_max_chunk 128k;
    server_tokens off; #Conceals nginx version
    access_log off;
    sendfile        on;
    client_header_timeout  3m;
    client_body_timeout 3m;
    send_timeout     3m;
    keepalive_timeout  0;

If somebody can help me improving nginx config will be helpful to him. I
apologize for bad engish :D
Posted by skechboy (Guest)
on 2013-01-23 12:59
(Received via mailing list)
Have you checked HDD performance on the server in this periods with atop 
or
iostat 1 ?
It's very likely to be related with this since I guess there's a lot's 
of
random reading on 2000 connections.

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,235447,235456#msg-235456
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 13:11
(Received via mailing list)
Hello,

       Following is the output of vmstat 1 on 1000+ concurrent 
connections
:-

procs -----------memory---------- ---swap-- -----io---- --system--
-----cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy 
id
wa st
 0  0      0 438364  43668 31548164    0    0    62    23    3    0  5 
0
95  0  0
 0  0      0 437052  43668 31548520    0    0  1292     0 19763 1570  0 
0
99  1  0
 0  1      0 435316  43668 31549940    0    0  1644     0 20034 1537  0 
0
99  1  0
 1  0      0 434688  43676 31551388    0    0  1104    12 19816 1612  0 
0
100  0  0
 0  0      0 434068  43676 31552304    0    0   512    24 20253 1541  0 
0
99  0  0
 1  0      0 430844  43676 31553156    0    0  1304     0 19322 1636  0 
0
99  1  0
 0  1      0 429480  43676 31554256    0    0   884     0 19993 1585  0 
0
99  0  0
 0  0      0 428988  43676 31555020    0    0  1008     0 19244 1558  0 
0
99  0  0
 0  0      0 416472  43676 31556368    0    0  1244     0 18752 1611  0 
0
99  0  0
 2  0      0 425344  43676 31557552    0    0  1120     0 19039 1639  0 
0
99  0  0
 0  0      0 421308  43676 31558212    0    0  1012     0 19921 1595  0 
0
99  0  0


This might be a stupid question ,which section should i focus from above
output to analyze if I/O is performing well or in heavy load?

Thanks
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 13:14
(Received via mailing list)
Sorry for above reply on wrong command. Following are the output of 
iostat
1 :-

Linux 2.6.32-279.19.1.el6.x86_64 (DNTX005.local)        01/23/2013
_x86_64_        (16 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.72    2.94    0.46    0.11    0.00   94.77

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              20.53      1958.91       719.38  477854182  175484870

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.06    0.00    0.13    0.19    0.00   99.62

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              30.00      1040.00      5392.00       1040       5392

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00    0.19    0.25    0.00   99.56

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              24.00      1368.00       104.00       1368        104


Thanks
Posted by Dennis Jacobfeuerborn (Guest)
on 2013-01-23 13:40
(Received via mailing list)
On 01/23/2013 10:43 AM, shahzaib shahzaib wrote:
>
> events {
>     access_log off;
>     sendfile        on;
>     client_header_timeout  3m;
>     client_body_timeout 3m;
>     send_timeout     3m;
>     keepalive_timeout  0;
>
> If somebody can help me improving nginx config will be helpful to him. I
> apologize for bad engish :D

What's the required bandwidth for the flv files? What is the bandwidth 
of
the connection of the system? What is the bandwidth of the uplink to the
Internet?

Regards,
  Dennis
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 14:14
(Received via mailing list)
The average size of each flv video is 60Mb+. We've five content servers
(nginx-1.2.1), each with 1Gbps port and 100TB bandwidth per month. Right
now each server is consuming 10~12 bandwidth per day and we're going to 
run
out of bandwidth on coming last days of month. However we limited every
connection to 180k, you can see limit_rate 180k; in nginx.conf file.

I am newbie to this field. Please correct me if i didn't satisfy your
question regarding bandwidth. :)


On Wed, Jan 23, 2013 at 5:39 PM, Dennis Jacobfeuerborn <
Posted by Geoffrey Hartz (Guest)
on 2013-01-23 14:34
(Received via mailing list)
It will not help you actualy but I had a similar experience.

My issue was due to the system event handle (epoll, kqueue...)

I noticed poor speed when hitting 2000 connections with haproxy. So I
switch to nginx + tcp module proxy. Same results..

But using haproxy + nginx (with two different event handler, I avoid
the speed problem). At the end, I prefered use ESXi and 2/3 VM and
split connections with DNS load balancing

Maybe you should take a look at this event handler problem and do some
tunning on kernel/OS. Nginx (maybe) isn't the actual issue.

2013/1/23 shahzaib shahzaib <shahzaib.cb@gmail.com>:
> On Wed, Jan 23, 2013 at 5:39 PM, Dennis Jacobfeuerborn
>> > We've
>> > worker_processes  16;
>> >     include       mime.types;
>> >     keepalive_timeout  0;
>>
> http://mailman.nginx.org/mailman/listinfo/nginx
--
Geoffrey HARTZ
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 15:14
(Received via mailing list)
Thanks for helping me out guyz but w're already dealing with 3-app 
servers
clustering with haproxy load balancer (For this video streaming site), 
and
can't afford another clustering type things. You're talking about 
Haproxy
load-balancer to split required flv files requests to different servers.
We'll have to buy 5 more content servers for mirroring data between 
every
two servers to split load-balancer requests but the problem is we're
running out of budget. If you can provide me some alternate to this 
problem?
Posted by Reinis Rozitis (Guest)
on 2013-01-23 15:43
(Received via mailing list)
> If somebody can help me improving nginx config will be helpful to him. I
> apologize for bad engish :D

It is not always the nginx that needs tuning. What about OS?

Have you changed the default file descriptor limits? Something like 
“Nginx
stream works very well on 1000~1500 concurrent connections“ might 
indicate
you are hitting the default 1024 limit.
While since some kernel versions linux does autotuning still sometimes
tweaking it a bit helps a lot:
http://www.cyberciti.biz/faq/linux-tcp-tuning/   etc ..


rr
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 15:58
(Received via mailing list)
File-discriptors are tweaked to 700000 in sysctl.conf. However i'll 
follow
above guide to tweak sysctl to better performance and will see if it 
works.
Thanks for guiding me guyz.
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 16:08
(Received via mailing list)
50% of it already been tweaked , i can send you sysctl.conf config if 
you
ask for it. I think the problem is not kernals, it is something else.
Posted by skechboy (Guest)
on 2013-01-23 16:21
(Received via mailing list)
From your output I can see that it isn't IO issue, I wish I could help 
you
more.

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,235447,235476#msg-235476
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 16:30
(Received via mailing list)
Sketchboy, i sent you the output of only 1000 concurrent connections
because it wasn't peak hours of traffic. I'll send you the output of 
iostat
1 when concurrent  connections will hit to 2000+ in next hour. Please 
keep
in touch cause i need to resolve this issue :(
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 17:52
(Received via mailing list)
Following is the output of 3000+ concurrent connections on iostat 1 
command
:-

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.72    2.96    0.47    0.12    0.00   94.73

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              22.47      1988.92       733.04  518332350  191037238

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.39    0.00    0.91    0.20    0.00   98.50

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              22.00      2272.00         0.00       2272          0

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.46    0.00    0.91    0.07    0.00   98.57

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              23.00       864.00        48.00        864         48

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.39    0.00    0.72    0.33    0.00   98.56

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              60.00      3368.00       104.00       3368        104

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.20    0.00    0.65    0.20    0.00   98.95
Posted by Lukas Tribus (Guest)
on 2013-01-23 19:08
(Received via mailing list)
Can you send us a 20+ lines of output from "vmstat 1" under this load? 
Also, what exact linux kernel are you running ("cat /proc/version")?


________________________________
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 20:01
(Received via mailing list)
Following is the output of 2200+ concurrent connections and kernel 
version
is 2.6.32 :-

Linux 2.6.32-279.19.1.el6.x86_64 (DNTX005.local)        01/23/2013
_x86_64_        (16 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.75    3.01    0.49    0.13    0.00   94.63

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              23.27      2008.64       747.29  538482374  200334422

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.97    0.00    1.10    0.19    0.00   97.74

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              30.00      2384.00       112.00       2384        112

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.13    0.00    0.52    0.13    0.00   99.22

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              21.00      1600.00         8.00       1600          8

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.19    0.00    0.45    0.26    0.00   99.10

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              37.00      2176.00         8.00       2176          8

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.45    0.00    0.58    0.19    0.00   98.77

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              24.00      1192.00         8.00       1192          8

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.32    0.00    0.45    0.19    0.00   99.03

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              29.00      2560.00         8.00       2560          8


avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.32    0.00    0.65    0.19    0.00   98.84

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              35.00      2584.00       152.00       2584        152

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.26    0.00    0.39    0.39    0.00   98.96

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              25.00      1976.00         8.00       1976          8

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.32    0.00    0.52    0.39    0.00   98.77

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              33.00      1352.00         8.00       1352          8

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.26    0.00    0.58    0.26    0.00   98.90

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              28.00      2408.00         8.00       2408          8

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.45    0.00    0.65    0.06    0.00   98.84

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              37.00      1896.00         8.00       1896          8

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.71    0.00    0.97    0.13    0.00   98.19

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              33.00      2600.00        64.00       2600         64

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.32    0.00    0.65    0.26    0.00   98.77

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              20.00      1520.00         8.00       1520          8

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.19    0.00    0.39    0.19    0.00   99.22

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              49.00      3088.00        80.00       3088         80

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.26    0.00    0.91    0.26    0.00   98.58

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              48.00      1328.00         8.00       1328          8

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.32    0.00    0.32    0.26    0.00   99.09

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              32.00      1528.00         8.00       1528          8

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.45    0.00    0.58    0.39    0.00   98.58

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              35.00      1624.00        72.00       1624         72

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.39    0.00    0.58    0.19    0.00   98.84
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 20:04
(Received via mailing list)
And also the 20+ lines of vmstat are given below with 2.6.32 kernal :-

procs -----------memory---------- ---swap-- -----io---- --system--
-----cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy 
id
wa st
 0  1      0 259020  49356 31418328    0    0    64    24    0    4  5 
0
95  0  0
 1  0      0 248100  49356 31418564    0    0   704     4 35809 3159  0 
1
99  0  0
 0  0      0 245248  49364 31419856    0    0  1340    48 35114 3217  0 
0
99  0  0
 1  0      0 243884  49364 31421084    0    0   940     4 35176 3106  0 
0
99  0  0
 0  0      0 243512  49364 31422152    0    0   812     4 35837 3204  0 
0
99  0  0
 0  0      0 241608  49364 31423056    0    0  1304     4 35585 3177  1 
1
98  0  0
 1  0      0 241076  49364 31424132    0    0  1004     4 35774 3199  0 
0
99  0  0
 0  0      0 241332  49372 31424644    0    0   724    76 35526 3203  0 
0
99  0  0
 0  0      0 240464  49372 31425376    0    0   776     4 35968 3162  0 
0
99  0  0
 0  1      0 238236  49372 31426244    0    0   652     4 35705 3131  0 
0
99  0  0
 0  0      0 234632  49372 31426924    0    0  1088     4 36220 3309  0 
1
99  0  0
 0  0      0 233640  49372 31428492    0    0   872     4 35663 3235  0 
1
99  0  0
 0  0      0 232896  49376 31429016    0    0  1272    44 35403 3179  0 
0
99  0  0
 1  0      0 231024  49376 31430064    0    0   528     4 34713 3238  0 
0
99  0  0
 0  0      0 239644  49376 31430564    0    0   808     4 35493 3143  0 
1
99  0  0
 3  0      0 241704  49376 31431372    0    0   612     4 35610 3400  1 
1
97  0  0
 1  0      0 244092  49376 31432028    0    0   280     4 35787 3333  1 
1
99  0  0
 2  0      0 244348  49376 31433232    0    0  1260     8 34700 3072  0 
0
99  0  0
 0  0      0 243908  49384 31433728    0    0   512    32 35019 3145  0 
1
99  0  0
 1  0      0 241104  49384 31435004    0    0  1440     4 35586 3211  0 
1
99  0  0
 0  0      0 234600  49384 31435476    0    0   868     4 35240 3235  0 
1
99  0  0
procs -----------memory---------- ---swap-- -----io---- --system--
-----cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy 
id
wa st
 1  0      0 233656  49384 31436376    0    0   704     4 35297 3126  0 
1
99  0  0
 0  0      0 233284  49384 31437176    0    0   192     4 35022 3202  0 
0
99  0  0
 0  0      0 228952  49392 31437336    0    0   868    32 34986 3211  0 
1
99  0  0
 0  0      0 232176  49392 31438124    0    0   448     4 35785 3294  0 
1
99  0  0
 0  0      0 230076  49392 31438664    0    0  1052     4 35532 3297  1 
1
98  0  0
 1  0      0 231184  49392 31439608    0    0   436     4 34967 3177  0 
1
99  0  0
 1  0      0 224300  49392 31440044    0    0   624     4 34577 3216  0 
1
99  0  0
 0  0      0 223748  49396 31440664    0    0   460    44 34415 3155  0 
0
99  0  0
 1  0      0 223260  49396 31441612    0    0   768     4 35287 3194  0 
1
99  0  0
 0  0      0 230464  49396 31441996    0    0   772     4 35140 3208  0 
0
99  0  0
 1  0      0 225504  49396 31442668    0    0   564     4 35316 3133  0 
0
99  0  0



On Thu, Jan 24, 2013 at 12:00 AM, shahzaib shahzaib
Posted by Lukas Tribus (Guest)
on 2013-01-23 20:20
(Received via mailing list)
The box doesn't seem to have problems with that kind of load, not even 
the IO side is struggling, I guess the 31GB of page cache is your life 
saver here.

I would check for recent events in dmesg output. Then I would analyze 
the network side. Like how much is you eth0 loaded (nload)? Perhaps you 
are simply saturating your Gbit/s pipe? what do you see with "ifconfig 
eth0"? Did you talk to the network operators if this kind of load can 
cause drops/packet loss?
Posted by Rainer Duffner (Guest)
on 2013-01-23 20:29
(Received via mailing list)
Am 23.01.2013 um 20:03 schrieb shahzaib shahzaib 
<shahzaib.cb@gmail.com>:

> And also the 20+ lines of vmstat are given below with 2.6.32 kernal :-


There was a thread recently (well, last year sometimes) with a link to a 
blog in Chinese with sysctl-settings etc.
It had tunings for 2k concurrent connections.

Maybe somebody can dig it out?
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 21:22
(Received via mailing list)
The load(nload) of 1500+ concurrent connections with 1Gbps port  is : 
Curr:
988.95 MBit/s
                                                                   ## 
##
##  ##  ##  ##  ##  ##  #  Avg: 510.84 MBit/s
                                                                   ## 
##
##  ##  ##  ##  ##  ##  #  Min: 0.00 Bit/s
                                                                   ## 
##
##  ##  ##  ##  ##  ##  #  Max: 1005.17 MBit/s
                                                                   ## 
##
##  ##  ##  ##  ##  ##  #  Ttl: 10017.30 GByte

What should i see into dmesg to analyse the problem ? I'll also send you
the nload when the traffic will hit to its peak, at this time its 
average
traffic. The following is ifconfig eth0 output :-

eth0      Link encap:Ethernet  HWaddr X:X:X:X:X:X
          inet addr:X.X.X.X  Bcast:X.X.X.X  Mask:255.255.255.192
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3713630148 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7281199166 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:260499010337 (242.6 GiB)  TX bytes:10767156835559 
(9.7
TiB)
          Memory:fbe60000-fbe80000
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 21:28
(Received via mailing list)
No i didn't concerned with network operators yet. And if someone can get 
me
that chinese blog for setting 2k concurrent connections using
sysctl-settings. So far i used this guide to tune kernal.
http://www.cyberciti.biz/faq/linux-tcp-tuning/
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-23 21:40
(Received via mailing list)
I am seeing the following messages on dmesg output :-

TCP: Peer 79.211.64.145:54649/80 unexpectedly shrunk window
347253187:347272955 (repaired)
TCP: Peer 79.211.64.145:54649/80 unexpectedly shrunk window
347253187:347272955 (repaired)
TCP: Peer 79.211.64.145:54649/80 unexpectedly shrunk window
347253187:347272955 (repaired)
TCP: Peer 81.155.221.33:53075/80 unexpectedly shrunk window
1986341072:1986342532 (repaired)
TCP: Peer 81.155.221.33:53075/80 unexpectedly shrunk window
1986341072:1986342532 (repaired)
TCP: Peer 81.155.221.33:53075/80 unexpectedly shrunk window
1986341072:1986342532 (repaired)
TCP: Peer 79.211.64.145:54709/80 unexpectedly shrunk window
1128744179:1128773611 (repaired)
TCP: Peer 79.211.64.145:54709/80 unexpectedly shrunk window
1128744179:1128773611 (repaired)
TCP: Peer 79.211.64.145:54709/80 unexpectedly shrunk window
1128744179:1128773611 (repaired)
TCP: Peer 79.211.64.145:54709/80 unexpectedly shrunk window
1128744179:1128773611 (repaired)
TCP: Peer 79.211.64.145:54709/80 unexpectedly shrunk window
1128744179:1128773611 (repaired)
TCP: Peer 79.211.64.145:54709/80 unexpectedly shrunk window
1128744179:1128773611 (repaired)


If somebody can explain me ? what is that shrunk window thing?
Posted by Geoffrey Hartz (Guest)
on 2013-01-23 21:43
(Received via mailing list)
Just to say... You are already hitting 1Gbits with a 1Gbit port...
It's normal that is slow... no?

2013/1/23 shahzaib shahzaib <shahzaib.cb@gmail.com>:
> TCP: Peer 81.155.221.33:53075/80 unexpectedly shrunk window
> 1128744179:1128773611 (repaired)
> wrote:
>>> The load(nload) of 1500+ concurrent connections with 1Gbps port  is :
>>> What should i see into dmesg to analyse the problem ? I'll also send you
>>> TiB)
>>>>
>>>> nginx@nginx.org
>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>>
>>
>
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx



--
Geoffrey HARTZ
Posted by richardm (Guest)
on 2013-01-23 22:50
(Received via mailing list)
<And if someone can get me that chinese blog for setting 2k concurrent
connections using sysctl-settings.>

Was it this one? It refers to 2M connections and claimed success.
      http://rdc.taobao.com/blog/cs/?p=1062
The blog is in Chinese. I used Chrome and clicked on "Translate" to read 
it.

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,235447,235507#msg-235507
Posted by Rainer Duffner (Guest)
on 2013-01-23 22:52
(Received via mailing list)
Am 23.01.2013 um 22:50 schrieb "richardm" <nginx-forum@nginx.us>:

> <And if someone can get me that chinese blog for setting 2k concurrent
> connections using sysctl-settings.>
>
> Was it this one? It refers to 2M connections and claimed success.
>      http://rdc.taobao.com/blog/cs/?p=1062


Yes, that's it.

2000k, not 2k.
Posted by Jonathan Matthews (Guest)
on 2013-01-23 23:21
(Received via mailing list)
On 23 January 2013 20:42, Geoffrey Hartz <hartz.geoffrey@gmail.com> 
wrote:
> Just to say... You are already hitting 1Gbits with a 1Gbit port...
> It's normal that is slow... no?

+1. I'm not seeing the problem here.

Jonathan
Posted by Lukas Tribus (Guest)
on 2013-01-24 01:04
(Received via mailing list)
>>> The load(nload) of 1500+ concurrent connections with 1Gbps port is : Curr: 
988.95 MBit/s
>> Just to say... You are already hitting 1Gbits with a 1Gbit port...
>> It's normal that is slow... no?
>+1. I'm not seeing the problem here.

Exactly, the issue is crystal clear. You are already hitting your max 
bandwidth with 1500+ concurrent connections, of course with 2000+ 
concurrent connections users will notice severe slowdowns. You have not 
enough bandwidth to serve your clients.

I suggest to monitor your eth0 links carefully and upgrade to either 
multiple bonded 1Gig links, 10gig links or more servers.
Posted by Scott Ribe (Guest)
on 2013-01-24 01:12
(Received via mailing list)
On Jan 23, 2013, at 5:04 PM, Lukas Tribus wrote:

> I suggest to monitor your eth0 links carefully and upgrade to either multiple 
bonded 1Gig links, 10gig links or more servers.

Or do what the cable companies do: compress your video until it fits, 
regardless of how bad the quality gets ;-)

--
Scott Ribe
scott_ribe@elevated-dev.com
http://www.elevated-dev.com/
(303) 722-0567 voice
Posted by Stefan Caunter (Guest)
on 2013-01-24 04:50
(Received via mailing list)
On Wed, Jan 23, 2013 at 7:04 PM, Lukas Tribus <luky-37@hotmail.com> 
wrote:
>
>>>> The load(nload) of 1500+ concurrent connections with 1Gbps port  is : Curr: 
988.95 MBit/s
>>> Just to say... You are already hitting 1Gbits with a 1Gbit port...
>>> It's normal that is slow... no?
>>+1. I'm not seeing the problem here.
>
> Exactly, the issue is crystal clear. You are already hitting your max bandwidth 
with 1500+ concurrent connections, of course with 2000+ concurrent connections 
users will notice severe slowdowns. You have not enough bandwidth to serve your 
clients.
>
> I suggest to monitor your eth0 links carefully and upgrade to either multiple 
bonded 1Gig links, 10gig links or more servers.


bah, you need a CDN.
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-24 07:58
(Received via mailing list)
Thanks for helping me out guyz. I'll tune my content server according to
that chinese guide. Please keep in mind i had only sent the output of 
one
of Five content servers. Other servers load(nload) is not that high and
they just hit 500Mbit/s on 2000 concurrent connections. However i'll
monitor eth0 port more closely on peak time and will let you know the
status. Thanks :)
Posted by shahzaib mushtaq (shahzaib12)
on 2013-01-24 12:31
(Received via mailing list)
I have got an idea of preventing users to download videos from our site, 
so
they just can stream videos and that way will save our bandwidth. We 
have
used one of nginx module "limit_conn 1" so nobody will be able to 
download
stream. But this thing has a major drawback of stream i.e if four users 
are
streaming videos under a LAN network with same ip, other 3 won't be able 
to
stream videos due to 1st user, who's already streaming it and when he'll
finish streaming it'll resume for 2nd user and vice virsa.

Can someone guide me if we can just prevent downloading but stream 
remains
the same ?


On Thu, Jan 24, 2013 at 11:57 AM, shahzaib shahzaib
Posted by Weibin Yao (yaoweibin)
on 2013-01-25 04:56
(Received via mailing list)
Maybe not the box hits the 1Gbit band bandwidth, your switch also 
possible
hits the limit.

2013/1/24 shahzaib shahzaib <shahzaib.cb@gmail.com>
Please log in before posting. Registration is free and takes only a minute.
Existing account (Switch to SSL-encrypted connection)
NEW: Do you have a Google/GoogleMail or Yahoo account? No registration required!
Log in with Google account | Log in with Yahoo account
No account? Register here.