Image serving via nginx are too slow, why?

Server NGINX
1

Hello,

Before posting, i’ve done a lot of google search, and i found other
similar
issue without any solutions.
So, Why nginx is slow serving files in my case ?

Part 1. The hardware

Intel Xeon E5-1620v2 4c/8t 3,7 GHz
32 Go DDR3 ECC 1600MHz
3xSSD Intel 160Go
1 Gbps

The load average is 0.13
I’ve been reproducing the “slow loading image effect” with all kind of
traffic situation from 0 to a lot.
I got 2 new servers load balanced, I disabled the 2nd server, and it had
no
effect at all.

Part2. The software

Nginx 1.6.1
PHP-FPM 5.5.16
Varnish 4, but the cache is disabled for all requests in the .vcl

My nginx configuration can be found on this gist :

My nginx vhost can be found there :

You can “feel” the problem for example here:

It’s not my website, but I took it from a post describing the same
problem.

A 2 Mo PDF takes 8 sec to load on any connection internet (from 20Mbps
@work
to my personal fiber 1Gbps)
http://i.imgur.com/cxYRm8c.png

I’m sure I did tweaked too much without really knowing which side effect
it
could have. Can you help me on this please ?

Posted at Nginx Forum:

2

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Did you try using aio? Try turn off sendfile and tcp_nopush also.
Here is my patch http://fpaste.org/128456/40901547/raw/

On 8/26/2014 03:51, tristanb wrote:

My nginx configuration can be found on this gist :
A 2 Mo PDF takes 8 sec to load on any connection internet (from
[email protected] nginx Info Page

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)

iQEcBAEBAgAGBQJT+97xAAoJEF1+odKB6YIx39YIAJKyIHFf7uGpj2j4vMYOqMZA
BK3dY1IpLQXpJE8WJwghvLmrLguM8WffE2facAo4YZsxvGUIce1+3qFwsWaSqJXy
7RM1pwWs6hDw+LhzyJYdtPLt/xnbdGiXQAPyAKD8OE51U1EFskVkLILr+MwTahLS
UeexniI5+yfLpGP4KEbg2fWu7WJh7wljAzvFwsgI2VqnleI7TQbUNVwylwzMmkd3
luODSyKcDIDW9aCN5Kh0U9DVIpx0nBJE+NyASZsBHgNxawRF8YdPtN9mI7b8Rvb2
DSO2fa8U4rKMwJDGRYA61xuq01AJigLUcgcRs5+hoNnmYpHNzVmEjWg34IbEwac=
=l542
-----END PGP SIGNATURE-----

3

Thanks for your message,

I applyed your patched, restarted varnish, nginx and php5-fpm and it’s
still
the same.
Browing with browser cache off feels like the image are downloaded and
displayed in a “progressive way” because of the slowlyness.

Another example too is this PDF of 3Mo who takes >3 minutes to display :
http://goo.gl/og3xG5

Posted at Nginx Forum:

4

Takes me 2.65 seconds to load the PDF with no caching.

5

Damn, I tested this on 3 different connection from 3 different providers
(all based in France though, where the server are)

  • 20mbps ADSL by Orange
  • 1Gbps fiber by Free
  • 50 Mbps fiber by SFR

Where are you based, what are you connexion specs please ?

Posted at Nginx Forum:

6

Thanks, you really helped me, i’m gonna call my hosting provider to
figure
it out.

I’m glad the nginx is not the cause here :wink:

Posted at Nginx Forum:

7

I’ve been testing from a 10 megabit connection in Netherlands, 100mbit
connection in netherlands, 500 mbit connection in netherlands, 500mbit
connection in France, 100mbit connection in france and a 250 megabit
connection in france, a 20 megabit connection in UK.
Can ask people from Denmark to do the same test.
But seems rather fast for all connections I’ve tested from.

  • Lucas R
8

A last thing, can you provide a traceroute please ?

Posted at Nginx Forum:

9

NL example:

mtr admin.yproximite.fr

HOST: server1 Loss% Snt Last Avg Best Wrst StDev

  1. hosted.by.leaseweb.com 0.0% 2 0.5 0.5 0.5
    0.5 0.0
  2. te0-7-0-3.hvc3.evo.leaseweb. 0.0% 2 0.7 0.7 0.6
    0.7 0.1
  3. ix-5-1-1-0.thar1.HNN-Amsterd 0.0% 2 0.3 0.3 0.3
    0.3 0.0
  4. if-10-2.tcore2.AV2-Amsterdam 0.0% 2 1.3 1.3 1.3
    1.3 0.0
  5. if-2-2.tcore1.AV2-Amsterdam. 0.0% 2 2.9 2.5 2.1
    2.9 0.6
  6. be3044.agr21.ams03.atlas.cog 0.0% 2 2.9 4.1 2.9
    5.4 1.8
  7. be2440.ccr42.ams03.atlas.cog 0.0% 2 2.0 4.8 2.0
    7.5 3.9
  8. be2266.ccr42.par01.atlas.cog 0.0% 2 11.3 12.5 11.3
    13.8 1.7
  9. be2309.ccr21.par04.atlas.cog 0.0% 2 11.5 11.5 11.5
    11.5 0.0
  10. 149.6.164.222 0.0% 2 13.6 13.7 13.6
    13.8 0.1
  11. dedibox-1-t.intf.routers.pro 0.0% 2 13.4 13.5 13.4
    13.6 0.2
  12. 49e-s46-1-a9k2.dc3.poneytele 0.0% 2 12.5 12.1 11.7
    12.5 0.6
  13. 88-190-234-137.rev.poneytele 0.0% 2 13.2 13.0 12.8
    13.2 0.2

France:
HOST: minecraft Loss% Snt Last Avg Best Wrst StDev

  1. 5.135.139.252 0.0% 2 0.4 0.3 0.3
    0.4 0.1
  2. rbx-g1-a9.fr.eu 0.0% 2 6.7 3.8 0.9
    6.7 4.1
  3. th2-g1-a9.fr.eu 0.0% 2 4.5 4.5 4.5
    4.5 0.0
  4. ??? 100.0 2 0.0 0.0 0.0
    0.0 0.0
  5. ??? 100.0 2 0.0 0.0 0.0
    0.0 0.0
  6. cbv-crs8-1-be1005.routers.pr 0.0% 2 8.0 6.8 5.6
    8.0 1.7
  7. bzn-9k-4-be1005.intf.routers 0.0% 2 5.1 5.2 5.1
    5.3 0.1
  8. dedibox-2-t.intf.routers.pro 0.0% 2 5.5 5.3 5.1
    5.5 0.3
  9. 195.154.1.146 0.0% 2 5.2 5.2 5.2
    5.2 0.0
  10. 49e-s46-1-a9k2.dc3.poneytele 0.0% 2 5.1 5.1 5.1
    5.1 0.0
  11. 88-190-234-137.rev.poneytele 0.0% 2 4.8 4.8 4.8
    4.8 0.0

NL2:
HOST: Lucass-MacBook-Pro.local Loss% Snt Last Avg Best Wrst
StDev
1.|-- 10.4.13.252 0.0% 2 1.0 1.2 1.0
1.4 0.3
2.|-- 87.255.57.222 0.0% 2 1.5 1.5 1.5
1.5 0.0
3.|-- adm-b4-link.telia.net 0.0% 2 1.2 1.3 1.2
1.4 0.2
4.|-- adm-bb4-link.telia.net 0.0% 2 1.3 1.4 1.3
1.4 0.1
5.|-- adm-b5-link.telia.net 0.0% 2 1.7 2.1 1.7
2.5 0.6
6.|-- cogent-ic-130765-adm-b3.c 0.0% 2 2.4 3.1 2.4
3.9 1.1
7.|-- be2312.ccr42.ams03.atlas. 0.0% 2 2.9 2.9 2.9
2.9 0.0
8.|-- be2266.ccr42.par01.atlas. 0.0% 2 12.1 12.1 12.1
12.1 0.0
9.|-- be2309.ccr21.par04.atlas. 0.0% 2 13.7 13.9 13.7
14.1 0.3
10.|-- 149.6.165.198 0.0% 2 12.1 12.2 12.1
12.2 0.0
11.|-- dedibox-1-t.intf.routers. 0.0% 2 14.4 13.6 12.7
14.4 1.2
12.|-- 49e-s46-1-a9k2.dc3.poneyt 0.0% 2 12.8 12.7 12.5
12.8 0.2
13.|-- 88-190-234-137.rev.poneyt 0.0% 2 12.2 12.3 12.2
12.4 0.1

NL3:
HOST: api Loss% Snt Last Avg Best Wrst StDev

  1. 82.196.14.1 0.0% 2 1.4 1.0 0.5
    1.4 0.7
  2. 83.231.213.61 0.0% 2 1.4 1.4 1.4
    1.4 0.0
  3. te0-7-0-3.agr21.ams03.atlas. 0.0% 2 0.9 1.0 0.9
    1.1 0.2
  4. be2434.ccr41.ams03.atlas.cog 0.0% 2 1.1 1.3 1.1
    1.5 0.3
  5. be2265.ccr41.par01.atlas.cog 0.0% 2 12.0 12.0 12.0
    12.0 0.1
  6. be2308.ccr21.par04.atlas.cog 0.0% 2 12.5 12.5 12.5
    12.6 0.0
  7. 149.6.165.214 0.0% 2 10.8 10.8 10.7
    10.8 0.0
  8. dedibox-1-t.intf.routers.pro 0.0% 2 12.6 12.6 12.6
    12.7 0.0
  9. 49e-s46-1-a9k2.dc3.poneytele 0.0% 2 11.3 11.3 11.3
    11.3 0.1
  10. 88-190-234-137.rev.poneytele 0.0% 2 12.3 12.3 12.3
    12.3 0.0
10

Thank you so much :wink:

Posted at Nginx Forum: