# Nginx as reverse caching proxy for CentOS repositories

**URL:** <https://www.ruby-forum.com/t/nginx-as-reverse-caching-proxy-for-centos-repositories/185120>\
**Category:** NGINX\
**Created:** [March 14, 2010, 2:04pm UTC](https://www.ruby-forum.com/t/nginx-as-reverse-caching-proxy-for-centos-repositories/185120 "2010-03-14T14:04:33Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Wouter\_Schoot](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/w/ba9def/32.png) [@Wouter\_Schoot](https://www.ruby-forum.com/u/Wouter_Schoot)\
**Post date:** [March 14, 2010, 2:04pm UTC](https://www.ruby-forum.com/t/nginx-as-reverse-caching-proxy-for-centos-repositories/185120/1 "2010-03-14T14:04:33Z")

</div>

Dear list,

In an attempt to create an offline copy for my homenetwork to quickly  
install CentOS I wanted to use nginx as a reverse caching proxy so that  
nginx stores all the files the that the CentOS installer requests after  
the first time.

However, after most of the startup files (product.img, stage2.img etc)  
are downloaded and cached succesfully, it bails on the chkconfig rpm  
(which happens to be the first file it actually downloads for the  
install, from the CentOS/ folder.

I’m running nginx/0.8.34, compiled with a standard ./configure (only  
using the prefix, so the rest is left as default).

The following relevant changes to the default configuration were made:

proxy\_cache\_path /storage/cache/proxy\_temp/ levels=1:2 keys\_zone=one:10m  
inactive=100d max\_size=1000M;  
proxy\_temp\_path /storage/cache/;

```
     location / {
  proxy_pass http://mirror.oxilion.nl;
         proxy_cache one;
         proxy_cache_key "$scheme$host$request_uri";
         proxy_cache_valid 200 100d;
     }

```

As far as I could google, this seemed only fair. Basic caching works as  
a charm, but it somehow chokes on the chkconfig rpm. However, when I use  
my proxy\_pass mirror (and doing so, bypass nginx), it works like a  
charm. So, the mirror is up2date and up&running.

I’ve tried emptying my cache several times, and it doesn’t matter.

This will be unable to troubleshoot without some debug. I’ve enabled  
caching debug and traced the (requesting) HTTP headers

T 192.168.1.113:33652 -\> 192.168.1.104:80 [AP]  
GET /centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm  
HTTP/1.1…Accept-Encoding: identity…Range: bytes=440-25087…Connection:  
close…Host: 192.168.1.104…User-agent:  
urlgrabber/3.1.0 yum/3.2.22…

Accesslog:

192.168.1.113 - - [13/Mar/2010:19:16:05 +0100] “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” 200  
162614 “-” “urlgrabber/3.1.0 yum/3.2.22”  
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” 200  
143352 “-” “urlgrabber/3.1.0 yum/3.2.22”  
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” 200  
133216 “-” “urlgrabber/3.1.0 yum/3.2.22”  
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” 200  
162614 “-” “urlgrabber/3.1.0 yum/3.2.22”  
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” 200  
162614 “-” “urlgrabber/3.1.0 yum/3.2.22”  
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” 200  
162614 “-” “urlgrabber/3.1.0 yum/3.2.22”  
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” 200  
123238 “-” “urlgrabber/3.1.0 yum/3.2.22”  
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” 200  
162614 “-” “urlgrabber/3.1.0 yum/3.2.22”  
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” 200  
152040 “-” “urlgrabber/3.1.0 yum/3.2.22”  
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” 200  
128872 “-” “urlgrabber/3.1.0 yum/3.2.22”

It tries to redownload the file several times and seems to give up after  
a few failed attempts.

Cache log:

\*\*\*14/Mar/2010:13:56:12 +0100 MISS Cache-Control: - Expires: - “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” (200)  
“urlgrabber/3.1.0 yum/3.2.22”  
\*\*\*14/Mar/2010:13:56:12 +0100 MISS Cache-Control: - Expires: - “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” (200)  
“urlgrabber/3.1.0 yum/3.2.22”  
\*\*\*14/Mar/2010:13:56:12 +0100 HIT Cache-Control: - Expires: - “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” (200)  
“urlgrabber/3.1.0 yum/3.2.22”  
\*\*\*14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” (200)  
“urlgrabber/3.1.0 yum/3.2.22”  
\*\*\*14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” (200)  
“urlgrabber/3.1.0 yum/3.2.22”  
\*\*\*14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” (200)  
“urlgrabber/3.1.0 yum/3.2.22”  
\*\*\*14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” (200)  
“urlgrabber/3.1.0 yum/3.2.22”  
\*\*\*14/Mar/2010:13:56:13 +0100 MISS Cache-Control: - Expires: - “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” (200)  
“urlgrabber/3.1.0 yum/3.2.22”  
\*\*\*14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” (200)  
“urlgrabber/3.1.0 yum/3.2.22”  
\*\*\*14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” (200)  
“urlgrabber/3.1.0 yum/3.2.22”

The file does exists, and downloads succesfully when I simply feed this  
link to firefox or any other tool.

Anaconda (the CentOS installer) gives me a  
“chkconfig-1.3.30.1-2.i386.rpm cannot be opened. This is due to a  
missing file, a corrupt package, or corrupt media.” in the installer  
screen, with no other options than retry and reboot.

For what it’s worth; I’ve also tried nginx-0.7.65 with a comparable  
config, but to no avail.

Any hints or pointers I might have missed? I’ve tried several  
configoptions, but none of them seem to help. If you have a comparable  
setup, I’d love to see the relevant config parts.

Thanks,

Wouter

---

<div class="post-metadata">

**Author:** ![Wouter\_Schoot](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/w/ba9def/32.png) [@Wouter\_Schoot](https://www.ruby-forum.com/u/Wouter_Schoot)\
**Post date:** [March 14, 2010, 9:34pm UTC](https://www.ruby-forum.com/t/nginx-as-reverse-caching-proxy-for-centos-repositories/185120/2 "2010-03-14T21:34:02Z")

</div>

Hi again,

To add something, from the error log:

2010/03/14 21:31:31 [info] 19695#0: \*23 client closed prematurely  
connection, client: 192.168.1.113, server: localhost, request: “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1”,  
host: “192.168.1.104”  
2010/03/14 21:31:31 [info] 19695#0: \*24 client closed prematurely  
connection (104: Connection reset by peer), client: 192.168.1.113,  
server: localhost, request: “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1”,  
host: “192.168.1.104”  
2010/03/14 21:31:31 [info] 19695#0: \*25 client closed prematurely  
connection (104: Connection reset by peer), client: 192.168.1.113,  
server: localhost, request: “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1”,  
host: “192.168.1.104”  
2010/03/14 21:31:31 [info] 19695#0: \*26 client closed prematurely  
connection (104: Connection reset by peer), client: 192.168.1.113,  
server: localhost, request: “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1”,  
host: “192.168.1.104”  
2010/03/14 21:31:31 [info] 19695#0: \*27 client closed prematurely  
connection (104: Connection reset by peer), client: 192.168.1.113,  
server: localhost, request: “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1”,  
host: “192.168.1.104”  
2010/03/14 21:31:32 [info] 19695#0: \*28 client closed prematurely  
connection (104: Connection reset by peer), client: 192.168.1.113,  
server: localhost, request: “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1”,  
host: “192.168.1.104”  
2010/03/14 21:31:32 [info] 19695#0: \*29 client closed prematurely  
connection (104: Connection reset by peer), client: 192.168.1.113,  
server: localhost, request: “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1”,  
host: “192.168.1.104”  
2010/03/14 21:31:32 [info] 19695#0: \*30 client closed prematurely  
connection (104: Connection reset by peer), client: 192.168.1.113,  
server: localhost, request: “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1”,  
host: “192.168.1.104”

I tried the tips on proxy\_buffering from  
[Hanging byte-range requests for remote backend](http://nginx.org/pipermail/nginx/2008-March/003678.html) and

proxy\_ignore\_client\_abort on;  
fastcgi\_ignore\_client\_abort on;

from ["client closed prematurely connection..." - NGINX - Ruby-Forum](http://www.ruby-forum.com/topic/130527) .

Could it be so that, as suggested on the first link, only a partial file  
is received, or sent to the client ?

Wouter

---

<div class="post-metadata">

**Author:** ![Wouter\_Schoot](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/w/ba9def/32.png) [@Wouter\_Schoot](https://www.ruby-forum.com/u/Wouter_Schoot)\
**Post date:** [March 14, 2010, 9:52pm UTC](https://www.ruby-forum.com/t/nginx-as-reverse-caching-proxy-for-centos-repositories/185120/3 "2010-03-14T21:52:10Z")

</div>

Hello!

On Sun, Mar 14, 2010 at 02:03:45PM +0100, Wouter Schoot wrote:

> In an attempt to create an offline copy for my homenetwork to  
> quickly install CentOS I wanted to use nginx as a reverse caching  
> proxy so that nginx stores all the files the that the CentOS  
> installer requests after the first time.
> 
> However, after most of the startup files (product.img, stage2.img  
> etc) are downloaded and cached succesfully, it bails on the  
> chkconfig rpm (which happens to be the first file it actually  
> downloads for the install, from the CentOS/ folder.

[…]

> This will be unable to troubleshoot without some debug. I’ve enabled  
> caching debug and traced the (requesting) HTTP headers
> 
> T 192.168.1.113:33652 → 192.168.1.104:80 [AP]  
> GET /centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm  
> HTTP/1.1..Accept-Encoding: identity..Range:  
> bytes=440-25087..Connection: close..Host: 192.168.1.104..User-agent:  
> urlgrabber/3.1.0 yum/3.2.22…

Most likely your client (incorrectly) assumes that range request  
must by satisfied, and bails out due to getting full reply (which  
happens when url in question isn’t yet cached).

Looking into client code may shed some light on the problem.

[…]

Maxim D.

---

<div class="post-metadata">

**Author:** ![Wouter\_Schoot](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/w/ba9def/32.png) [@Wouter\_Schoot](https://www.ruby-forum.com/u/Wouter_Schoot)\
**Post date:** [March 14, 2010, 10:02pm UTC](https://www.ruby-forum.com/t/nginx-as-reverse-caching-proxy-for-centos-repositories/185120/4 "2010-03-14T22:02:07Z")

</div>

Hi Maxim,

Thanks for your reply!

I do however have replies which state there is a cache-hit, so there’s  
no need to refetch the file:

\*\*\*14/Mar/2010:21:31:32 +0100 HIT Cache-Control: - Expires: - “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” (200)  
“urlgrabber/3.1.0 yum/3.2.22”  
\*\*\*14/Mar/2010:21:31:32 +0100 HIT Cache-Control: - Expires: - “GET  
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1” (200)  
“urlgrabber/3.1.0 yum/3.2.22”

(using the following log-line: log\_format cache '\*\*\*$time\_local ’  
'$upstream\_cache\_status - $upstream\_status - ’  
'Cache-Control: $upstream\_http\_cache\_control ’  
'Expires: $upstream\_http\_expires ')

> Most likely your client (incorrectly) assumes that range request  
> must by satisfied, and bails out due to getting full reply (which  
> happens when url in question isn’t yet cached).
> 
> Looking into client code may shed some light on the problem.

So it probably lies somewhere in the Range header. Is there some setting  
I can use in nginx to override or obey whatever the CentOS installer  
wants? A client-hack in nginx’ config, so to say?

But, now that I think of it.. When nginx has the file cached (HIT  
cachestatus), how does nginx’ reply to the client differ from the  
original one… When I directly fetch the rpm from the backend mirror  
(during install), there’s no problem at all. Only when I put nginx in  
between, with or without the rpm in cache, it fails on me.

Wouter

---

<div class="post-metadata">

**Author:** ![Wouter\_Schoot](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/w/ba9def/32.png) [@Wouter\_Schoot](https://www.ruby-forum.com/u/Wouter_Schoot)\
**Post date:** [March 18, 2010, 10:06am UTC](https://www.ruby-forum.com/t/nginx-as-reverse-caching-proxy-for-centos-repositories/185120/5 "2010-03-18T10:06:44Z")

</div>

Hi,

Is there anyone who can shed a light on this issue? It somehow seems to  
misnegotiate the range headers. Can I somehow (on nginx) ignore this and  
make it all work so that the reverse caching proxy will work for my  
installs?

Wouter

---

<div class="post-metadata">

**Author:** ![Wouter\_Schoot](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/w/ba9def/32.png) [@Wouter\_Schoot](https://www.ruby-forum.com/u/Wouter_Schoot)\
**Post date:** [March 18, 2010, 9:52pm UTC](https://www.ruby-forum.com/t/nginx-as-reverse-caching-proxy-for-centos-repositories/185120/6 "2010-03-18T21:52:16Z")

</div>

Hello!

On Thu, Mar 18, 2010 at 07:30:02PM +0100, Wouter Schoot wrote:

> Could this somehow be related to http 1.1 to the client and 1.0 to  
> the backend? Will that mess up the Range header? Any idea’s ?

Have you actually tried looking into your client’s code as  
I’ve already suggested?

Given the fact that there are multiple requests with Range header  
and some of them replied with 200 (and full content) it’s pretty  
obvious that client which doesn’t handle 200 correctly will get  
garbled result. And you can’t do anything with it without  
actually fixing client (or stopping it somehow from using range  
requests).

Maxim D.

---

<div class="post-metadata">

**Author:** ![Wouter\_Schoot](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/w/ba9def/32.png) [@Wouter\_Schoot](https://www.ruby-forum.com/u/Wouter_Schoot)\
**Post date:** [March 18, 2010, 9:58pm UTC](https://www.ruby-forum.com/t/nginx-as-reverse-caching-proxy-for-centos-repositories/185120/7 "2010-03-18T21:58:42Z")

</div>

Hi Maxim,

Thanks for replying,

> Have you actually tried looking into your client’s code as  
> I’ve already suggested?
> 
> Given the fact that there are multiple requests with Range header  
> and some of them replied with 200 (and full content) it’s pretty  
> obvious that client which doesn’t handle 200 correctly will get  
> garbled result. And you can’t do anything with it without  
> actually fixing client (or stopping it somehow from using range  
> requests).

I understand what you’re saying. However, since “my client” is the  
CentOS/RedHat installer (Anaconda is used during that installer to  
download and install the \*.rpm files), I’m not going to be able to modfy  
it’s code I think. And besides, I’m not that good at reading code.

I’m now, since I’m quite sure it is indeed the installer that goes  
wrong, looking for a way to work around it. Perhaps some server-side  
voodoo or (as I tried tonight) another proxy in between that filters  
some of the headers (I tried pound, but to no avail yet).

Wouter

---

<div class="post-metadata">

**Author:** ![Wouter\_Schoot](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/w/ba9def/32.png) [@Wouter\_Schoot](https://www.ruby-forum.com/u/Wouter_Schoot)\
**Post date:** [March 19, 2010, 10:05am UTC](https://www.ruby-forum.com/t/nginx-as-reverse-caching-proxy-for-centos-repositories/185120/8 "2010-03-19T10:05:02Z")

</div>

On 18-3-2010 21:51, Maxim D. wrote:

> Have you actually tried looking into your client’s code as  
> I’ve already suggested?

It seems it’s a known “bug”:

> **[530900 – Installer blocks on first FTP download because REST FTP command is...](https://bugzilla.redhat.com/show_bug.cgi?id=530900)**

---

<div class="post-metadata">

**Author:** ![Wouter\_Schoot](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/w/ba9def/32.png) [@Wouter\_Schoot](https://www.ruby-forum.com/u/Wouter_Schoot)\
**Post date:** [March 18, 2010, 7:30pm UTC](https://www.ruby-forum.com/t/nginx-as-reverse-caching-proxy-for-centos-repositories/185120/9 "2010-03-18T19:30:35Z")

</div>

Could this somehow be related to http 1.1 to the client and 1.0 to the  
backend? Will that mess up the Range header? Any idea’s ?