Forum: NGINX error_page and named locations

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Cynix C. (Guest)
on 2008-12-10 16:42
(Received via mailing list)
I have the following config on 0.7.26:

server {
 ...

 location / {
  error_page 404 = @cms;
 }

 location @cms {
  fastcgi_pass 127.0.0.1:1234;
  include fastcgi_params;
  fastcgi_param SCRIPT_FILENAME /www_root/index.php;
  fastcgi_param SCRIPT_NAME /index.php;
 }

 location ~ \.php$ {
  fastcgi_pass 127.0.0.1:1234;
  include fastcgi_params;
  fastcgi_param SCRIPT_FILENAME /www_root$fastcgi_script_name;
  fastcgi_param SCRIPT_NAME $fastcgi_script_name;
  fastcgi_intercept_errors on;
  error_page 404 = @cms;
 }

 ...
}

This should pass all requests to non-existent paths to /index.php so
"clean
URLs" will work and the CMS will display the corresponding pages. Indeed
it
works if I try to access a URI that is first handled by the "location /"
block,
such as http://my.com/2008/12/my-blog-post.

However, when I access a URI that is first handled by the "location ~
\.php$"
block, such as http://my.com/non-existent.php, theoretically nginx will
first
pass this request to FastCGI, then PHP sends back 404 because it can't
find the
non-existent.php file, then nginx should honor the error_page directive
and send
the request to @cms. In reality it results in a 502 Bad Gateway. The
error log
says "upstream sent invalid header while reading response header from
upstream".
But if I used the following:

 location ~ \.php$ {

  error_page 404 = /index.php;
 }

Then it will work as intended.

Why can't I use a named location for error_page if the location block
contains a
fastcgi_pass directive? (I assume this is the case for proxy_pass too,
but I
didn't test that one.)
Cynix C. (Guest)
on 2008-12-10 18:04
(Received via mailing list)
cynix <cynix@...> writes:

>
> But if I used the following:
>
>  location ~ \.php$ {
>
>   error_page 404 = /index.php;
>  }
>
> Then it will work as intended.
>

Just to clarify, I meant

 location ~ \.php$ {
  fastcgi_pass 127.0.0.1:1234;
  include fastcgi_params;
  fastcgi_param SCRIPT_FILENAME /www_root$fastcgi_script_name;
  fastcgi_param SCRIPT_NAME $fastcgi_script_name;
  fastcgi_intercept_errors on;
  error_page 404 = /index.php;
 }

works. I used "..." instead of retyping the whole thing to save a few
keystrokes
but don't know why it didn't show.
Igor S. (Guest)
on 2008-12-10 18:27
(Received via mailing list)
Attachment: patch.fastcgi (0 Bytes)
On Wed, Dec 10, 2008 at 02:32:51PM +0000, cynix wrote:

> I have the following config on 0.7.26:
>
> server {
>  ...
>
>  location / {

You may want to add this here:

    log_not_found   off;

>  location ~ \.php$ {
>
> says "upstream sent invalid header while reading response header from upstream".
> fastcgi_pass directive? (I assume this is the case for proxy_pass too, but I
> didn't test that one.)

The attached patch fixes the bug.
Almir K. (Guest)
on 2008-12-10 18:37
(Received via mailing list)
the problem is that when you request \.php$ the error_page from locatin
/
doesn't get executed. to resolve the problem move the error_page
directive out
of the location / (directly in server {}).
Igor S. (Guest)
on 2008-12-10 19:02
(Received via mailing list)
On Wed, Dec 10, 2008 at 05:28:48PM +0100, Almir K. wrote:

> the problem is that when you request \.php$ the error_page from locatin /
> doesn't get executed. to resolve the problem move the error_page directive out
> of the location / (directly in server {}).

No, there is a bug in nginx.
Cynix C. (Guest)
on 2008-12-11 12:19
(Received via mailing list)
Igor S. <is@...> writes:

>
> You may want to add this here:
>
>     log_not_found   off;
>

Yes I have that line there. Since it's not related to the problem at
hand I left
it out for simplicity's sake.

>
> The attached patch fixes the bug.
>

Thanks Igor, the patch works as intended.

I'd also like your opinion on another way to do the same thing.

 location ~ \.php$ {
  set $script_filename $document_root$fastcgi_script_name;
  if (!-f $script_filename) {
   set $script_filename $document_root/index.php;
  }
  fastcgi_pass   127.0.0.1:1234;
  include        fastcgi_params;
  fastcgi_param  SCRIPT_FILENAME $script_filename;
 }

Is this a better way to handle requests to non-existent PHP files? This
way
nginx takes care of checking if the file exists, and if it doesn't exist
the
original request is not passed to FastCGI at all. Only 1 request is
passed to
FastCGI, but with the addition of file existence checking. Will this
yield
better performance than 2 requests to FastCGI?
Igor S. (Guest)
on 2008-12-11 13:05
(Received via mailing list)
On Thu, Dec 11, 2008 at 10:09:56AM +0000, cynix wrote:

>  }
>
> Is this a better way to handle requests to non-existent PHP files? This way
> nginx takes care of checking if the file exists, and if it doesn't exist the
> original request is not passed to FastCGI at all. Only 1 request is passed to
> FastCGI, but with the addition of file existence checking. Will this yield
> better performance than 2 requests to FastCGI?

I do not recommend to use if/rewrite in nginx as they have some
implementation issues. Nevertheless, your way is better than passing
2 requests to FastCGI.

I'm going to implement the non_existant_request_file directive to use
in similar configurations:

    location ~\.php$ {
        non_existant_request_file  @drupal;

        fastcgi_pass  ..
    }

This directive test a request file existance before passing a request
to fastcgi/proxy.

Could someone suggest better name ?
Igor S. (Guest)
on 2008-12-11 13:22
(Received via mailing list)
On Thu, Dec 11, 2008 at 01:51:51PM +0300, Igor S. wrote:

> >   include        fastcgi_params;
> implementation issues. Nevertheless, your way is better than passing
>
> This directive test a request file existance before passing a request
> to fastcgi/proxy.
>
> Could someone suggest better name ?

Also I want to replace typcal mongrel configuration:

  location / {

      if (-f $request_filename) {
          break;
      }

      if (-f $request_filename/index.html) {
          rewrite (.*) $1/index.html break;
      }

      if (-f $request_filename.html) {
          rewrite (.*) $1.html break;
      }

      if (!-f $request_filename) {
          proxy_pass http://mongrel;
          break;
      }
  }

with something like

     location / {
         match  $request_filename
                $request_filename/index.html
                $request_filename.html;
                @mongrel;
     }

like "index" directive.

Or

         try    $request_filename
                $request_filename/index.html
                $request_filename.html
                /some_fallback_url;
Phillip B Oldham (Guest)
on 2008-12-11 13:57
(Received via mailing list)
Attachment: phill.vcf (0 Bytes)
Igor S. wrote:
> I'm going to implement the non_existant_request_file directive to use
> in similar configurations:
>
>     location ~\.php$ {
>         non_existant_request_file  @drupal;
>
>         fastcgi_pass  ..
>     }
>
> Could someone suggest better name ?
Maybe "when_file_ne"?  Though I'd say you've made it quite succinct
as-is.
--

*Phillip B Oldham*
The Activity People
removed_email_address@domain.invalid 
<mailto:removed_email_address@domain.invalid>

------------------------------------------------------------------------

*Policies*

This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to
anyone.

This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.
Igor S. (Guest)
on 2008-12-11 14:45
(Received via mailing list)
On Thu, Dec 11, 2008 at 02:09:25PM +0300, Igor S. wrote:

> > >   }
> >
> >         fastcgi_pass  ..
>
>       }
>          match  $request_filename
>                 $request_filename/index.html
>                 $request_filename.html
>                 /some_fallback_url;

The single directive can be used for Mongrel type configuration:

      location / {
          file_match  $uri  $uri/index.html  $uri.html
                      @mongrel;
      }

      location @mongrel {
          ...
      }

and Drupal/Joomla type configuration:

      location / {
          file_match  $uri  @drupal;
          # the same as
          #    error_page  404 = @drupal; log_not_found off;
      }

      location ~ \.php$ {
          file_match  $uri  @drupal;

          fastcgi_pass   ...
          fastcgi_param  SCRIPT_FILENAME /path/to$script_filename;
      }

      location @drupal {
          fastcgi_pass   ...
          fastcgi_param  SCRIPT_FILENAME /path/to/index.php;
      }

The file_match iterates parameters and changes URI to the first one
that matches filesystem. An @name matches always and is used as
fallback.
All matches except fallback are handled in the same location context.
Cynix C. (Guest)
on 2008-12-11 14:49
(Received via mailing list)
Igor S. <is@...> writes:

> > I do not recommend to use if/rewrite in nginx as they have some
> > implementation issues. Nevertheless, your way is better than passing
> > 2 requests to FastCGI.

Thanks. Yes I'm aware of the caveats of using "if" (such as only the
last true
if block is processed), that's why I thought of using the error_page
first. But
I guess as long as I keep the if blocks simple it shouldn't be a
problem.


> > I'm going to implement the non_existant_request_file directive to use
> > in similar configurations:

I'd love to see this implemented, I suggest somthing like "if_not_found"
or
"non_exist_fallback".


> Also I want to replace typcal mongrel configuration:
> ...
> with something like
> ...

try is better than match I think.
Jean-Philippe M. (Guest)
on 2008-12-11 16:08
(Received via mailing list)
Igor S. a écrit :
>>> This directive test a request file existance before passing a request
>>                 $request_filename.html;
>>                 /some_fallback_url;
>       }
>           file_match  $uri  @drupal;
> The file_match iterates parameters and changes URI to the first one
> that matches filesystem. An @name matches always and is used as fallback.
> All matches except fallback are handled in the same location context.
>
>

I think "try" is a good name for this feature (which will certainly be
_really_
useful, thanks !).
Jim O. (Guest)
on 2008-12-11 16:17
(Received via mailing list)
I hate to be a nit-picker, but the spelling should be
"non_existent_request_file"



Jim
Igor S. (Guest)
on 2008-12-11 16:35
(Received via mailing list)
On Thu, Dec 11, 2008 at 02:56:16PM +0100, Jean-Philippe M. wrote:

> >>>
> >>                 $request_filename/index.html
> >>                 $request_filename.html
> >           ...
> >       location ~ \.php$ {
> >
> > The file_match iterates parameters and changes URI to the first one
> > that matches filesystem. An @name matches always and is used as fallback.
> > All matches except fallback are handled in the same location context.
>
> I think "try" is a good name for this feature (which will certainly be _really_
> useful, thanks !).

How about "try_file  $uri  $uri/index.html ..." or "test_file  $uri ..."
?
Igor S. (Guest)
on 2008-12-11 18:24
(Received via mailing list)
On Thu, Dec 11, 2008 at 05:18:51PM +0300, Igor S. wrote:

> > >>>         fastcgi_pass  ..
> > >>      location / {
> > >>          try    $request_filename
> > >
> > >       }
> > >           fastcgi_param  SCRIPT_FILENAME /path/to/index.php;
> > >       }
> > >
> > > The file_match iterates parameters and changes URI to the first one
> > > that matches filesystem. An @name matches always and is used as fallback.
> > > All matches except fallback are handled in the same location context.
> >
> > I think "try" is a good name for this feature (which will certainly be _really_
> > useful, thanks !).
>
> How about "try_file  $uri  $uri/index.html ..." or "test_file  $uri ..." ?

Or "use_file" ?
Ezra Z. (Guest)
on 2008-12-11 18:48
(Received via mailing list)
>>> od name for this feature (which will certainly be _really_
>>> useful, thanks !).
>>
>> How about "try_file  $uri  $uri/index.html ..." or "test_file
>> $uri ..." ?
>
> Or "use_file" ?




      location / {
          file_with_fallback  $uri  $uri/index.html  $uri.html
                      @mongrel;
      }

      location @mongrel {
          ...
      }


  Seems to express the idea?

Cheers-

Ezra
mike (Guest)
on 2008-12-11 21:05
(Received via mailing list)
Thu, Dec 11, 2008 at 4:31 AM, Igor S. <removed_email_address@domain.invalid> 
wrote:

>      location / {
>          file_match  $uri  @drupal;
>          # the same as
>          #    error_page  404 = @drupal; log_not_found off;
>      }

This is a very common example - almost anything with "nice names" will
pass the non-existent file requests to a single file handler. That's
how we're designing our own application too (and how I design mine
nowadays)

Is there a reason that a new parameter would need to be invented? Why
is error_page not capable of this? (Note that I was using error_page
in my 0.7.21 or so and the POST data was not coming through to my
PHP/fastcgi scripts, which I thought was fixed a long while back) so
I've been using if (-f $request_filename) for the time being (I don't
want to test for dirs, only exact filenames, otherwise index files get
factored in and I don't want those to be used actually, and I can't
rename them right now)

If this new parameter is affecting only fastcgi-based requests I would
request it be prefixed with fastcgi_ at least for consistency (maybe
fastcgi_fallback_pass or something?). If it is general purpose,
file_try_match maybe. This is hard :)
Igor S. (Guest)
on 2008-12-11 22:12
(Received via mailing list)
On Thu, Dec 11, 2008 at 10:51:59AM -0800, mike wrote:

> how we're designing our own application too (and how I design mine
> nowadays)
>
> Is there a reason that a new parameter would need to be invented? Why
> is error_page not capable of this? (Note that I was using error_page
> in my 0.7.21 or so and the POST data was not coming through to my
> PHP/fastcgi scripts, which I thought was fixed a long while back) so
> I've been using if (-f $request_filename) for the time being (I don't
> want to test for dirs, only exact filenames, otherwise index files get
> factored in and I don't want those to be used actually, and I can't
> rename them right now)

I will look your POST/body issue.

> If this new parameter is affecting only fastcgi-based requests I would
> request it be prefixed with fastcgi_ at least for consistency (maybe
> fastcgi_fallback_pass or something?). If it is general purpose,
> file_try_match maybe. This is hard :)

No, the new directive is general purpose.

The most demonstrably usage is mongrel configuration, where you need
to test several static files existence:

      location / {
          use_file  $uri  $uri/index.html  $uri.html  @mongrel;
      }

      location @mongrel {
          ...
      }

Second usage is testing a script existence before passing it to backend:

      location ~ \.php$ {
          use_file  $uri  @drupal;

          fastcgi_pass   ...
          fastcgi_param  SCRIPT_FILENAME /path/to$script_filename;
      }

And finally, it can be used as simple replacement of

      location / {
          error_page  404 = @drupal; log_not_found off;
      }

as

      location / {
          use_file  $uri  @drupal;
      }

although I have no plan to drop "error_page" support.
The last "use_file" usage is just syntax sugar.

As to names, now I have three variants: use_file, test_file, try_file.
Jean-Philippe M. (Guest)
on 2008-12-11 22:55
(Received via mailing list)
Ezra Z. a écrit :
>                      @mongrel;
>      }
>
>
>     Seems to express the idea?
>
> Cheers-
>
> Ezra
>

'try_file' looks great. I don't think there is a need to refer to
'fallback' in
the feature itself as this is optional.
mike (Guest)
on 2008-12-12 00:30
(Received via mailing list)
or maybe something like "try_first"

will it test files and directories or only files? would not want it to
be try_file if it tests for dirs too :) i can see the need to test for
both files and dirs.

On Thu, Dec 11, 2008 at 12:44 PM, Jean-Philippe M.
Igor S. (Guest)
on 2008-12-12 10:29
(Received via mailing list)
On Thu, Dec 11, 2008 at 02:18:25PM -0800, mike wrote:

> or maybe something like "try_first"
>
> will it test files and directories or only files? would not want it to
> be try_file if it tests for dirs too :) i can see the need to test for
> both files and dirs.

In Unix a directory is a file :)

The new directive
1) tests file(s) existence,
2) and uses first found file as URI.

I want to find a name to express these actions.
mike (Guest)
on 2008-12-12 11:05
(Received via mailing list)
On Fri, Dec 12, 2008 at 12:13 AM, Igor S. <removed_email_address@domain.invalid> 
wrote:

> In Unix a directory is a file :)

ah, I guess I was just making sure, since there is -d -e and -f checks.

> The new directive
> 1) tests file(s) existence,
> 2) and uses first found file as URI.

Yes, I understand.

> I want to find a name to express these actions.

Yup, I'm trying to help find a name too, that's one of the reasons I
asked about dirs vs. files and not limiting it to "file"

Maybe something like "first_uri" ?
Igor C. (Guest)
on 2008-12-12 11:50
Hi Igor,

Igor S. wrote:

> The new directive
> 1) tests file(s) existence,
> 2) and uses first found file as URI.
>
> I want to find a name to express these actions.

If there's an implied

3) falls back to a named location if no file is found

then I think it would be good to have some reference to the fallback
functionality, even if it's optional, as explicit names are more clear,
and for people trying to read documentation, directive names with
'hidden' functionality are hard to find.

The 'uri' bit, on the other hand, can be assumed, as we know from the
fact that we're in a location block that this directive is dealing with
a particular URI, so we need to express what this directive does to
fulfil the request for that URI.

So I'd like to suggest something like 'try_files_with_fallback' - it's
perhaps a bit wordy, but seems to express more clearly what's happening.

Cheers,
Igor C.
Igor S. (Guest)
on 2008-12-12 13:27
(Received via mailing list)
On Fri, Dec 12, 2008 at 10:50:21AM +0100, Igor C. wrote:

> If there's an implied
>
> 3) falls back to a named location if no file is found
>
> then I think it would be good to have some reference to the fallback
> functionality, even if it's optional, as explicit names are more clear,
> and for people trying to read documentation, directive names with
> 'hidden' functionality are hard to find.

Yes, however, fallback is not optional. The directive has at least two
parameters: the last one will be always fallback and it may be
@named_location or /uri/that/should/not/fail.

> The 'uri' bit, on the other hand, can be assumed, as we know from the
> fact that we're in a location block that this directive is dealing with
> a particular URI, so we need to express what this directive does to
> fulfil the request for that URI.
>
> So I'd like to suggest something like 'try_files_with_fallback' - it's
> perhaps a bit wordy, but seems to express more clearly what's happening.

I think fallback should not be mentioned.
Igor C. (Guest)
on 2008-12-12 17:05
Igor S. wrote:
> On Fri, Dec 12, 2008 at 10:50:21AM +0100, Igor C. wrote:
>
>> If there's an implied
>>
>> 3) falls back to a named location if no file is found
>>
>> then I think it would be good to have some reference to the fallback
>> functionality, even if it's optional, as explicit names are more clear,
>> and for people trying to read documentation, directive names with
>> 'hidden' functionality are hard to find.
>
> Yes, however, fallback is not optional. The directive has at least two
> parameters: the last one will be always fallback and it may be
> @named_location or /uri/that/should/not/fail.

Ah, OK.

>> The 'uri' bit, on the other hand, can be assumed, as we know from the
>> fact that we're in a location block that this directive is dealing with
>> a particular URI, so we need to express what this directive does to
>> fulfil the request for that URI.
>>
>> So I'd like to suggest something like 'try_files_with_fallback' - it's
>> perhaps a bit wordy, but seems to express more clearly what's happening.
>
> I think fallback should not be mentioned.

Yes, I think you're right, because it's one of a list of options being
tried, and in a sense it's not actually a fallback, it's just the last
option.

use_* is good because it makes it clear that one of the options /will/
be used; try_* is good because it makes it clear that the arguments are
tried in turn until one /can/ be used. *_file is perhaps less good
because it implies only files can be used in the list.

Maybe use_resource or try_resource? I'd probably go for use_resource,
otherwise, out of the existing proposed names, I'd vote for use_file.

Cheers,
Igor C.
Igor S. (Guest)
on 2008-12-12 17:47
(Received via mailing list)
On Fri, Dec 12, 2008 at 04:05:34PM +0100, Igor C. wrote:

> >> 'hidden' functionality are hard to find.
> >> fulfil the request for that URI.
> use_* is good because it makes it clear that one of the options /will/
> be used; try_* is good because it makes it clear that the arguments are
> tried in turn until one /can/ be used. *_file is perhaps less good
> because it implies only files can be used in the list.
>
> Maybe use_resource or try_resource? I'd probably go for use_resource,
> otherwise, out of the existing proposed names, I'd vote for use_file.

It should be certainly _file to indicate that a real files are tested.

How about try_to_use_file(s) ?
Mark A. (Guest)
on 2008-12-12 18:35
(Received via mailing list)
Igor S. wrote:
> On Fri, Dec 12, 2008 at 10:50:21AM +0100, Igor C. wrote:
>
>> Hi Igor,
>>
>> Igor S. wrote:
>>
>>> The new directive
>>> 1) tests file(s) existence,

tests file(s) existence = valid uri

>>> 2) and uses first found file as URI.

uses first found file as URI = first valid uri


What about first_valid_uri ?


M.
Ezra Z. (Guest)
on 2008-12-12 19:11
(Received via mailing list)
Taking a hint from javacript how about:

try_these   $uri  $uri/index.html  $uri.html  @mongrel;

  Or maybe try_these_files

Cheers-
Ezra Z.
removed_email_address@domain.invalid
Igor C. (Guest)
on 2008-12-13 00:06
Igor S. wrote:

> On Fri, Dec 12, 2008 at 04:05:34PM +0100, Igor C. wrote:
>
> It should be certainly _file to indicate that a real files are tested.

I'm happy to accept your decision on that :-)

> How about try_to_use_file(s) ?

Seems a bit too much, maybe?

On reflection I think try_files is fine. I like Ezra's try_these too,
either would work.

Cheers
Igor C.
KT Walrus (Guest)
on 2008-12-13 00:13
(Received via mailing list)
My suggestion is:

select_uri_file with the default selection being the last file listed
and first file that exists is used.
Merlin (Guest)
on 2009-01-03 09:22
(Received via mailing list)
Having just read the entire conversation and coming perhaps a bit too
late
in the game, I would also like to champion a try_* identifier and I
really
like the try_these identifier for an extremely general facility that
tries
each _file, /uri/that/works, or even http://backends in whatever order I
specify.  I would stay away from the test_* family of directives unless
we
are going to add some kind of testing facility outside of if.
This topic is locked and can not be replied to.