Forum: NGINX SPDY + proxy cache = occasional 499 errors

2974d09ac2541e892966b762aad84943?d=identicon&s=25 spdyg (Guest)
on 2012-12-02 00:49
(Received via mailing list)
Hi there,

I have been piloting the SPDY patch on a separate web server pointing to
an
existing HTTP backend.

With just myself using this pilot web server, occasionally static
resources
such as css or images are not loading in Firefox (usually when pressing
F5
to reload a page)

Have done a bit of digging and have learnt this so far:

When issue occurs in Firefox, access log is showing 499 errors for the
missing static resources (when it should be 304)
Refreshing the page will eventually load the resources (and they will
show
304 in the access log)
If I turn off the proxy cache for these static resources, the issue
stops
If I turn off SPDY, the issue stops.
The issue obviously doesn't occur on the production web server running
Nginx
1.2.x

Has this been seen before, or can you think of any reason why this might
happen?

I don't normally use Chrome but have also witnessed this once in Chrome,
so
I don't think it's browser specific (however will try to get more
confidence
of this).

Thanks.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,233461,233461#msg-233461
40b4c848b8fcd63b0cb60b9d170c3a77?d=identicon&s=25 Valentin V. Bartenev (Guest)
on 2012-12-03 14:14
(Received via mailing list)
On Sunday 02 December 2012 03:48:52 spdyg wrote:
>
> happen?
>
[...]

Thank you for testing. There is a reason for this in current spdy
implementation. I'm working on it.

 wbr, Valentin V. Bartenev

--
http://nginx.com/support.html
http://nginx.org/en/donation.html
2974d09ac2541e892966b762aad84943?d=identicon&s=25 spdyg (Guest)
on 2013-04-02 12:55
(Received via mailing list)
I posted this query back in December:

http://forum.nginx.org/read.php?2,233497

And the issue was fixed shortly afterwards in the 1.3.x branch.

I'm now seeing the same issue again (static items coming from proxy
cache +
SPDY not displaying).

However this time, the log does not show 499 error anymore.. it just
shows
successful 304.  However, looking at firefox web console, the first few
static items come back as 304 without problems, then the next set of
static
items come back with no response at all.. (no response header, no body).

The setup looks like this:

Firefox -> Nginx 1.3.15 -> HAproxy -> Tomcat

Has this bug crept back in, or is this a new one?  Also reproduced on
1.3.13.

Can't seem to reproduce the issue if I take out HAproxy in the middle
though.

spdyg

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,233497,238019#msg-238019
40b4c848b8fcd63b0cb60b9d170c3a77?d=identicon&s=25 Valentin V. Bartenev (Guest)
on 2013-04-02 15:17
(Received via mailing list)
On Tuesday 02 April 2013 14:54:30 spdyg wrote:
> successful 304.  However, looking at firefox web console, the first few
> Can't seem to reproduce the issue if I take out HAproxy in the middle
> though.
>

The behavior of "return 499" can result in weird effects if it affects
a bunch of requests at once.

  wbr, Valentin V. Bartenev

--
http://nginx.org/en/donation.html
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2013-04-02 16:46
(Received via mailing list)
Hello!

On Tue, Apr 02, 2013 at 05:16:46PM +0400, Valentin V. Bartenev wrote:

> > However this time, the log does not show 499 error anymore.. it just shows
> >
> > Can't seem to reproduce the issue if I take out HAproxy in the middle
> > though.
> >
>
> The behavior of "return 499" can result in weird effects if it affects
> a bunch of requests at once.

Do you mean "return 444"?

--
Maxim Dounin
http://nginx.org/en/donation.html
40b4c848b8fcd63b0cb60b9d170c3a77?d=identicon&s=25 Valentin V. Bartenev (Guest)
on 2013-04-02 17:22
(Received via mailing list)
On Tuesday 02 April 2013 18:46:01 Maxim Dounin wrote:
> > > I'm now seeing the same issue again (static items coming from proxy
> > > Firefox -> Nginx 1.3.15 -> HAproxy -> Tomcat
> Do you mean "return 444"?
Indeed, I accidentally confused 499 with 444 and as a result I
misunderstood the
issue. Thanks for correcting me.

  wbr, Valentin V. Bartenev

--
http://nginx.org/en/donation.html
2974d09ac2541e892966b762aad84943?d=identicon&s=25 spdyg (Guest)
on 2013-04-03 18:27
(Received via mailing list)
Hi guys,

So perhaps I shouldn't have reused my existing topic and confused
things,
sorry!

I only did so because the symptom is exactly the same as the issue I
originally reported (SOME static content served from proxy cache
occasionally doesn't make it to Firefox when using SPDY).  If you
disable
either SPDY or the proxy cache the problem stops happening.

The originally reported problem stopped occurring after a fix was
implemented, but now I'm seeing it again since I put haproxy as the
upstream
(to get enhanced health checks etc).

The difference this time is that I'm not seeing the 499 errors (or 444)
in
the nginx log.  I'm not returning 444 (via config) in these scenarios
either.

Firefox shows this:

GET https://whatever/postrequest [HTTP/1.1 200 OK 45ms]
GET https://whatever/css1 [HTTP/1.1 304 OK 25ms]
GET https://whatever/css2 [HTTP/1.1 304 OK 260ms]
GET https://whatever/js1 [HTTP/1.1 304 OK 260ms]
GET https://whatever/js2 [HTTP/1.1 304 OK 35ms]
GET https://whatever/image1 [HTTP/1.1 304 OK 40ms]
GET https://whatever/image2 [HTTP/1.1 304 OK 40ms]
GET https://whatever/image3 [HTTP/1.1 304 OK 40ms]
GET https://whatever/image4 [HTTP/1.1 304 OK 40ms]
GET https://whatever/image5 [5ms]
GET https://google.com/analyticsjs [HTTP/1.1 304 Not Modified 50ms]
GET https://whatever/image6 [0ms]
GET https://whatever/image7 [0ms]
GET https://whatever/image8 [0ms]
GET https://whatever/image9 [0ms]
GET https://whatever/image10 [0ms]

So you can see that around the time it loads the 5th image things go
wrong
and the 5th onward images all have no content.  Firefox detail shows no
response headers/content for the 5th image onward.

The Haproxy and Nginx logs show 304 successfully for all of these
requests,
so I guess the connection is dropped somewhere and Nginx isn't picking
it up
and still thinks it worked fine.

Any suggestions on how to proceed further on this?  Happy to provide
specific details/logs via email if required.

Thanks,

spdyg

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,233497,238039#msg-238039
40b4c848b8fcd63b0cb60b9d170c3a77?d=identicon&s=25 Valentin V. Bartenev (Guest)
on 2013-04-04 00:21
(Received via mailing list)
On Wednesday 03 April 2013 02:49:55 spdyg wrote:
> The originally reported problem stopped occurring after a fix was
> GET https://whatever/css1 [HTTP/1.1 304 OK 25ms]
> GET https://whatever/image7 [0ms]
> up and still thinks it worked fine.
>
> Any suggestions on how to proceed further on this?  Happy to provide
> specific details/logs via email if required.
>

Is there anything in the error log?

  wbr, Valentin V. Bartenev

--
http://nginx.org/en/donation.html
2974d09ac2541e892966b762aad84943?d=identicon&s=25 spdyg (Guest)
on 2013-04-04 00:43
(Received via mailing list)
No there's nothing in the error log.  Access log shows 304's for all
requests that failed.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,233497,238073#msg-238073
40b4c848b8fcd63b0cb60b9d170c3a77?d=identicon&s=25 Valentin V. Bartenev (Guest)
on 2013-04-05 12:56
(Received via mailing list)
On Thursday 04 April 2013 02:43:15 spdyg wrote:
> No there's nothing in the error log.  Access log shows 304's for all
> requests that failed.
>

Could you provide a debug log for the issue?
Here is the guide: http://nginx.org/en/docs/debugging_log.html

  wbr, Valentin V. Bartenev

--
http://nginx.org/en/donation.html
2974d09ac2541e892966b762aad84943?d=identicon&s=25 spdyg (Guest)
on 2013-04-08 14:17
(Received via mailing list)
Thanks Valentin.  I have emailed you (off-list) some debug logs /
screenshots.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,233497,238195#msg-238195
87f4999607aa42f5492db95b12a36340?d=identicon&s=25 David Gwynne (Guest)
on 2013-07-22 13:43
(Received via mailing list)
i am also experiencing this problem running 1.4.2.

cheers,
dlg
2974d09ac2541e892966b762aad84943?d=identicon&s=25 spdyg (Guest)
on 2013-07-24 13:36
(Received via mailing list)
David Gwynne Wrote:
-------------------------------------------------------
> i am also experiencing this problem running 1.4.2.

Hi David.  The response I got from my Valentin off-list was that there
is a
known incompatibility with SPDY and proxy cache, due to the way proxy
cache
will kill the connection to prevent it sending the body response (but
before
it has had a chance to finish dealing with all the SPDY requests, I
think).

Only solution now is to either disable SPDY or proxy cache (I went with
SPDY, of course).

I wonder if a note in the documentation might be in order to say there
are
known issues.

There is supposed to be a rewrite coming for proxy cache that will help,
but
not sure when it is likely to appear.

Phil

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,233497,241193#msg-241193
2974d09ac2541e892966b762aad84943?d=identicon&s=25 JStl (Guest)
on 2013-11-14 18:41
(Received via mailing list)
Hi,

I have the same problem (nginx 1.4.3) when spdy is enabled and proxy
cache
too (with apache2 behind).

Any news about this bug ? I would really like to be able to activate
SPDY...

Thanks

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,233497,244667#msg-244667
2974d09ac2541e892966b762aad84943?d=identicon&s=25 spdyg (Guest)
on 2013-11-16 13:03
(Received via mailing list)
No unfortunately, but I have since filed a bug in trac:

http://trac.nginx.org/nginx/ticket/428

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,233497,244737#msg-244737
2974d09ac2541e892966b762aad84943?d=identicon&s=25 ruben.herold (Guest)
on 2013-11-26 21:35
(Received via mailing list)
It would be nice to get some more informations about the current status
and
work.
Since RHEL 6.5 / Centos 6.5 there is openssl 1.0.1e available for me and
I
would
like to activate SPDY on our systems.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,233497,244983#msg-244983
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.