Forum: NGINX proxy_cache calculating size error under SSD drive but not SATA drive

2974d09ac2541e892966b762aad84943?d=identicon&s=25 feanorknd (Guest)
on 2013-07-18 13:46
(Received via mailing list)
Hello:

I think I am on the right way, but not sure...

The scenario:

- Have 2 drives:
                           -> SSD drive - XFS - default options (with
noatime, discard) - almost empty
                           -> SATA drive - XFS - default options -
almost
empty

- Kernel variables, shared memory, ulimits, max-file, etc... all
correctly
configured

- Nginx and virtualhost correctly configured.

- proxy_cache_path <PATH> levels=1:2 keys_zone=catalogo_fotos:2500m
max_size=2500m inactive=120d;


Ok.. here I am... it depends of the path... the problem is:

- If the path is at SATA drive -> cache is growing until maximum 2500m
are
reached... no problem here. Normal..
- If the path is at SSD drive -> cache is not growing until limit... it
stale... problem... why?

If debug see:

2013/07/18 13:15:52 [debug] 17641#0: http file cache size: 640039
2013/07/18 13:15:52 [debug] 17641#0: http file cache forced expire
2013/07/18 13:15:52 [debug] 17641#0: http file cache forced expire: #0 1
933590ab
2013/07/18 13:15:52 [debug] 17641#0: http file cache expire:
"/DATA/cache/catalogo_fotos/7/3a/c1de7a39f5393c24933590ab20ba43a7"

while:

root@megaserver1 /DATA/cache # du -hs catalogomodacom_fotos/
1.8G    catalogo_fotos/
root@megaserver1 /DATA/cache # find ./catalogo_fotos/ -type f  | wc -l
17134

What happens?

Every 10 seconds, http file cache size detects max size and starts to
force
expire and delete cached files...

What is that "http file cache size: 640039" ???? That number, which way
is
calculated??? What it is?

I have enough shared memory, already properly configured (1Gb for each
app
and 14Gb total).... and the question is if I change path and set it
under
SATA drive there is no problem.

I am thinking at all at this "http file cache size: 640039"..... I think
it
is the key about what is happening... that number is not near the max
size
of 2500m defined for the cache deposit, and is not near the number of
files
actually at cache, around 17134...

Could you help me please?

Thanks so much in advance.

Gino.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,240993,240993#msg-240993
2974d09ac2541e892966b762aad84943?d=identicon&s=25 feanorknd (Guest)
on 2013-07-18 14:32
(Received via mailing list)
Hello:

Each 10 seconds, the core force expires files until http cache size
below
640000   (limits are defined at 2500Mb at config)

grep "cache size" /STORAGE/log.txt
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640339
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640317
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640297
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640262
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640248
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640236
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640226
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640217
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640207
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640197
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640167
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640157
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640141
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640124
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640115
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640057
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 640040
2013/07/18 13:15:22 [debug] 17641#0: http file cache size: 639904
2013/07/18 13:15:32 [debug] 17641#0: http file cache size: 640192
2013/07/18 13:15:32 [debug] 17641#0: http file cache size: 640188
2013/07/18 13:15:32 [debug] 17641#0: http file cache size: 640154
2013/07/18 13:15:32 [debug] 17641#0: http file cache size: 640115
2013/07/18 13:15:32 [debug] 17641#0: http file cache size: 640106
2013/07/18 13:15:32 [debug] 17641#0: http file cache size: 640097
2013/07/18 13:15:32 [debug] 17641#0: http file cache size: 640083
2013/07/18 13:15:32 [debug] 17641#0: http file cache size: 640074
2013/07/18 13:15:32 [debug] 17641#0: http file cache size: 640050
2013/07/18 13:15:32 [debug] 17641#0: http file cache size: 640042
2013/07/18 13:15:32 [debug] 17641#0: http file cache size: 640026
2013/07/18 13:15:32 [debug] 17641#0: http file cache size: 640011
2013/07/18 13:15:32 [debug] 17641#0: http file cache size: 639997
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640397
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640380
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640368
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640343
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640330
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640306
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640158
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640149
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640139
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640130
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640119
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640110
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640101
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640092
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640072
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640062
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640049
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 640025
2013/07/18 13:15:42 [debug] 17641#0: http file cache size: 639944

Hope it helps...

For the proccesor, the max http file cache size is 640000   (but only
for
this proxy_cache... i have more!!!)

Thanks.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,240993,240995#msg-240995
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2013-07-18 16:44
(Received via mailing list)
Hello!

On Thu, Jul 18, 2013 at 07:46:01AM -0400, feanorknd wrote:

> empty
> Ok.. here I am... it depends of the path... the problem is:
>
> - If the path is at SATA drive -> cache is growing until maximum 2500m are
> reached... no problem here. Normal..
> - If the path is at SSD drive -> cache is not growing until limit... it
> stale... problem... why?

Try looking into this ticket:

http://trac.nginx.org/nginx/ticket/157

With XFS, a file size reported on just created files before a file
is closed is incorrect, and this might confuse nginx.

[...]

> What is that "http file cache size: 640039" ???? That number, which way is
> calculated??? What it is?

This is the cache size in blocks.

--
Maxim Dounin
http://nginx.org/en/donation.html
2974d09ac2541e892966b762aad84943?d=identicon&s=25 feanorknd (Guest)
on 2013-07-19 02:55
(Received via mailing list)
Thanks thanks so much!!! :D

I even saw that ticket before posting, but I figured out it was not the
problem just because I do use XFS for my nginx_caches at 4 servers
without
this problem, and also I did test changing the path to point to a XFS
partition on SATA drive, and the problem dissapeared.

If you have a look at the ticket, that user also use a SSD drive and
have
the alloc problem.... it seems like if only at SSD drives, the size
notification is incorrect because of the allocsize, but not on SATA
drives,
although the allocsize is default for both of them, at least in my
case!!!!

Have a look at my first post and see at SATA drive, with XFS defaults,
the
cache gets fullfilled exactly and if I configure 2500Gb, the "du -hs"
when
core starts force expiring objects, is just that 2500Gb..... the
measures al
exact and real.

Not the same for SSD.. do you think people is something to review by
developers somehow?

Thanks.


Maxim Dounin Wrote:
-------------------------------------------------------

> http://nginx.org/en/donation.html
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,240993,241022#msg-241022
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2013-07-19 12:05
(Received via mailing list)
Hello!

On Thu, Jul 18, 2013 at 08:54:48PM -0400, feanorknd wrote:

> although the allocsize is default for both of them, at least in my case!!!!
>
> Have a look at my first post and see at SATA drive, with XFS defaults, the
> cache gets fullfilled exactly and if I configure 2500Gb, the "du -hs" when
> core starts force expiring objects, is just that 2500Gb..... the measures al
> exact and real.
>
> Not the same for SSD.. do you think people is something to review by
> developers somehow?

Depending on media used XFS might apply different defaults, and/or
observed behaviour might be different due to timing reasons.
Quick search suggests XFS currently uses dynamic allocsize by
default, see here:

http://git.kernel.org/cgit/linux/kernel/git/torval...

Try forcing allocsize to something like 4k to see if it helps.

--
Maxim Dounin
http://nginx.org/en/donation.html
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.