Rewrite help when files do NOT have a ".php" extension

Server NGINX
1

Loving Nginx so far! So much faster and leaner than Apache.

Apologies for this yet another Rewrite rule help after so many threads,
but one of domains has a special need and I am struggling with Rewrite
rules.

The logic is simple:

  1. For anything in /site folder, it should treat the static files as it
    is, but all else is php (file names don’t have .php extension). Still
    further in this folder:

    a. Within /site/private, it should be password protected
    b. Within /site/wordpress, there is a wordpress blog

  2. For the main folder “/” all URLs should be directed to
    “/site/redirect”.

Following is in URI examples, so that it easier to visualize:

(Lines starting with asterisks ** are already working)…

/abc —> /site/redirect?u=abc
/xyz —> /site/redirect?u=xyz

** /site/abc —> /site/abc (as PHP file)
** /site/xyz —> /site/xyz (as PHP file)

** /site/1.gif —> Served as it is, static
** /site/2.png —> Served as it is, static

** /site/private/abc —> Inside http auth, serve as PHP
/site/private/ —> Inside http auth, serve as PHP
/site/private/index

/site/wordpress/… —> This is the usual wordpress thing…
The apache rules used to be:

RewriteEngine On
RewriteBase /site/wordpress/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /site/wordpress/index.php [L]

Here is what I have tried so far. I am almost there:

# SERVE CACHE STATIC FILES FIRST
location ~*

.(xml|mp4|ttf|css|rss|atom|js|jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|doc|xls|exe|ppt)$
{
try_files $uri =404;
expires max;
access_log off;
log_not_found off;
}

# http://nginx.org/en/docs/http/converting_rewrite_rules.html
location / {
    try_files
          $uri
          /site/redirect?u=$uri
          @fromphp
          =404;
}

# FOR ALL THE PHP HANDLING
location @fromphp {
  fastcgi_pass    unix:/dev/shm/php5-fpm.MYDOMAIN.sock;
  fastcgi_index   index;
  include         fastcgi_params;
  fastcgi_param   SCRIPT_FILENAME

$document_root$fastcgi_script_name;
fastcgi_param PATH_TRANSLATED
$document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
}

# FOR THE REST OF THE WEBSITE
location /site/ {

    fastcgi_pass    unix:/dev/shm/php5-fpm.MYDOMAIN.sock;
    fastcgi_index   index;
    include         fastcgi_params;
    fastcgi_param   SCRIPT_FILENAME

$document_root$fastcgi_script_name;
fastcgi_param PATH_TRANSLATED
$document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;

    location ~ /site/private/ {
      auth_basic            "Restricted";
      auth_basic_user_file

/home/MYDOMAIN/.htpasswds/public_html/site/private/passwd;
}

    location ~ \.php$ {
      #NOTE: Set "cgi.fix_pathinfo = 0;" in php.ini
      fastcgi_split_path_info   ^(.+\.php)(/.+)$;
      fastcgi_pass

unix:/dev/shm/php5-fpm.MYDOMAIN.sock;
fastcgi_intercept_errors on;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME
$document_root$fastcgi_script_name;
fastcgi_param PATH_TRANSLATED
$document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME
$fastcgi_script_name;
}

    location ~ /site/wordpress/ {
      # Wordpress stuff
      try_files $uri $uri/ /site/wordpress/index.php?q=$uri&$args;
      if (-f $request_filename)  {  expires 30d;  break; }
      if (-d $request_filename)  {  break; }
      if (!-e $request_filename) {  rewrite ^/site/wordpress/(.+)$

/site/wordpress/index.php?q=$1 last; }
}

}

Basically what is NOT working:

  1. The main directory which is supposed to redirect to
    “/site/redirect?u=URI”
  2. The Wordpress stuff is downloading a text file (with the full PHP
    code, which is scary)

Thanks for any ideas or pointers!

Posted at Nginx Forum:

2

On 1 Jun 2011 13h00 WEST, [email protected] wrote:

is, but all else is php (file names don’t have .php extension). Still
Following is in URI examples, so that it easier to visualize:

RewriteEngine On
RewriteBase /site/wordpress/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /site/wordpress/index.php [L]

location / {
rewrite ^/(.*)$ /site/redirect?u=$1;
}

location /site {
location
/site/.*.(?:xml|mp4|ttf|css|rss|atom|js|jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|doc|xls|exe|ppt)$
expires max;
access_log off;
log_not_found off;

}
location /site/private {
auth_basic “Restricted”;
auth_basic_user_file
/home/MYDOMAIN/.htpasswds/public_html/site/private/passwd;

}

location /site/wordpress {
## Regular PHP processing.
location ~ ^/site/wordpress/(?.+.php)(?<path_info>.*)$ {
include fastcgi.conf;
## The fastcgi_params must be redefined from the ones
## given in fastcgi.conf. No longer standard names
## but arbitrary: named patterns in regex.
fastcgi_param SCRIPT_FILENAME $document_root$script;
fastcgi_param SCRIPT_NAME $script;
fastcgi_param PATH_INFO $path_info;
## Passing the request upstream to the FastCGI
## listener.
fastcgi_pass unix:/dev/shm/php5-fpm.MYDOMAIN.sock;;
}
}
}

location /site/redirect {

}

you need a try_files directive here but your “desired” config is

quite confusing and I don’t know what you need/want. Perhaps:

try_files $uri $uri/ /site/wordpress; ## ???
}

For the fastcgi block use nested locations and named captures like the
above example, always prefixing the regex with the outer location
string, like in the /site/wordpress location.

— appa

3

On 1 Jun 2011 14h23 WEST, [email protected] wrote:

Thanks so much appa. In your suggested example, is there an opening
bracket after "location

/site/.*.(?:xml|mp4|ttf|css|rss|atom|js|jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|doc|xls|exe|ppt)$"

?

Yes there should be one {. I forgot that.

location
/site/.*.(?:xml|mp4|ttf|css|rss|atom|js|jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|doc|xls|exe|ppt)$
{
(…)
}

I got confused a bit with the blocks structure there.

It looks confusing but in fact is clearer since you’re reducing your
exposure to potential config bugs due to unwanted side effects of
regex based locations.

Will try some of this, and get back with results here especially
related to the last “try_files” bit. I didn’t fully comprehend what
you are suggesting, but let me read it some more times and try
stuff.

— appa

4

Thanks so much appa. In your suggested example, is there an opening
bracket after “location
/site/.*.(?:xml|mp4|ttf|css|rss|atom|js|jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|doc|xls|exe|ppt)$”
?

I got confused a bit with the blocks structure there.

Will try some of this, and get back with results here especially related
to the last “try_files” bit. I didn’t fully comprehend what you are
suggesting, but let me read it some more times and try stuff.

Posted at Nginx Forum:

5

On 1 Jun 2011 15h03 WEST, [email protected] wrote:

#----------------
$document_root$fastcgi_script_name;
fastcgi_param PATH_TRANSLATED
$document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
}

try_files $uri $uri/ /site/get?u=$uri =404;

} # END OF SITE BLOCK
#----------------

This won’t work unless you have defined the name of the script that is
to be executed by the upstream fastcgi process.

What is the name of the script that receives the u argument? get.php?
index.php?

Questions:

(1) What am I missing here? The “fastcgi_params” is a file that has
the
usual bits and pieces.

Yes.

(2) Will a more precise regexp like above be faster, or should i
leave
it as “(.*)”?

I don’t think it will make much of a difference. But I’m not privy to
the details of libpcre implementation. Perhaps Igor or Maxim can shed
more light on that.

Of course the more specific you are the better it is in a security
sense at least, IMHO.

— appa

6

Sorry edogawan. That was funny :slight_smile:

Appa, yes I agree, it is much clearer with the blocks now making sense.

NOT working though. I will deal with Wordpress etc later. Want my root
folder stuff to work first. Here are my simple rules.

#----------------
location / {
rewrite ^/([-~a-zA-Z0-9_.]+)$ /site/get?u=$1;
}

location /site {

    location /site/get {
      fastcgi_pass    unix:/dev/shm/php5-fpm.MYDOMAIN.sock;
      include         fastcgi_params;
      fastcgi_param   SCRIPT_FILENAME

$document_root$fastcgi_script_name;
fastcgi_param PATH_TRANSLATED
$document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
}

   try_files $uri $uri/ /site/get?u=$uri  =404;

}  # END OF SITE BLOCK

#----------------

Questions:

(1) What am I missing here? The “fastcgi_params” is a file that has the
usual bits and pieces.

(2) Will a more precise regexp like above be faster, or should i leave
it as “(.*)”?

Thanks@

Posted at Nginx Forum:

7

António P. P. Almeida Wrote:

This won’t work unless you have defined the name
of the script that is
to be executed by the upstream fastcgi process.

What is the name of the script that receives the u
argument? get.php? index.php?

Oh I see. I had totally misunderstood how this works.

The name of the script is “/site/get”. Without the .php extension.

And the uri should be passed as “?u=” and in some case there will be a
second variable “&sb=”. The rewrite rules will work this way:

#--------------
location / {
rewrite ^/([-~a-zA-Z0-9_.]+)(.+)$ /site/get?u=$1&sb=$2;
rewrite ^/([-~a-zA-Z0-9_.]+)$ /site/get?u=$1;
}
#--------------

So basically I can do the upstream bit this way?

#--------------
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$/site/get;
fastcgi_param PATH_TRANSLATED $document_root$/site/get;
#--------------

Thanks for any ideas. (This is not working)

Posted at Nginx Forum:

8

It’s not working. Keeps sending me back to “/site/index”.

Let me see if I can install nginx with the debug module and post back
here. Or is it easier to install the echo module?

Posted at Nginx Forum:

9

On 1 Jun 2011 15h32 WEST, [email protected] wrote:

rewrite ^/([-~a-zA-Z0-9_.]+)(.+)$ /site/get?u=$1&sb=$2;
#--------------
Then use an exact location:

location = /site/get {
include fastcgi_params;

Where’s the fastcgi_pass directive?

fastcgi_pass unix:/path/to/socket.sock;
fastcgi_param SCRIPT_FILENAME $fastcgi_script_filename;
}

I suggest you either use the echo module by agentzh or enable the
error log with the debug flag to see what’s going on.

http://wiki.nginx.org/CoreModule#error_log

— appa

10

Thanks. I put this in my server directive:

  error_log logs/error.log debug;
  access_log logs/access.mine.log;
  log_format combined2 '$remote_addr - $remote_user [$time_local]

’
'“$request” $status $body_bytes_sent ’
‘“$http_referer” “$http_user_agent”’;

But this creates a file “access.mine.log” which is empty. And the
error_log only manages to show me messages that are about the reloading
of the nginx server (“[notice] 19142#0: signal process started”).

What can I do?

Posted at Nginx Forum:

11

On 1 Jun 2011 16h30 WEST, [email protected] wrote:

It’s not working. Keeps sending me back to “/site/index”.

Let me see if I can install nginx with the debug module and post
back here. Or is it easier to install the echo module?

There’s no debug module. The error_log directive is provided by the
Main module. Which is a required module for any nginx build.

— appa

12

Regards to all
I’m looking for a description of the Nginx Architecture in deep to work
on a
post on my blog about it.
Where I can find it?

Thanks a lot

–
Marcos Luis Ortiz Valmaseda
Software Engineer (Distributed Systems)
http://uncubanitolinuxero.blogspot.com

13

On 6/2/11, Marcos O. [email protected] wrote:

I’m looking for a description of the Nginx Architecture in deep to work on a
post on my blog about it.
Where I can find it?

I doubt it even exists. I remember there were some slides for nginx
internals and Evan M.'s guide, but that’s about it.

14

On 2 Jun 2011 11h11 WEST, [email protected] wrote:

But this creates a file “access.mine.log” which is empty. And the
error_log only manages to show me messages that are about the
reloading of the nginx server (“[notice] 19142#0: signal process
started”).

What can I do?

Did you configured the nginx build with the --with-debug option?

See http://wiki.nginx.org/NginxDebugging

— appa

15

Did you configured the nginx build with the
–with-debug option?

See http://wiki.nginx.org/NginxDebugging

Yes, this is what I was referring to earlier. Probably called it wrong,
as “module”.

Mine is a production server. My reading suggests that “with-debug” the
binary becomes quite large.

Anyway, is there absolutely no other way? The “error_log /file debug”
options is surprisingly useless.

Posted at Nginx Forum:

16

On 3 Jun 2011 00h33 WEST, [email protected] wrote:

Mine is a production server. My reading suggests that “with-debug”
the binary becomes quite large.

I think that you’re confusing --with-debug and with debugging
symbols. That’s two very different things. The later indeed generates
a binary that is much larger.

Try it: --with-debug

— appa

17

Thanks, Alexandr
That’s precisely I want to about on the blog post, to trying to explain
in deep this because many people
yet doesn’t understand why we use Nginx instead another web server.

Regards
----- Mensaje original -----
De: “Alexandr G.” [email protected]
Para: [email protected]
Enviados: Jueves, 2 de Junio 2011 16:13:08 GMT -05:00 Región oriental
EE. UU./Canadá
Asunto: Re: Nginx Architecture in deep

On 6/2/11, Marcos O. [email protected] wrote:

I’m looking for a description of the Nginx Architecture in deep to work on a
post on my blog about it.
Where I can find it?

I doubt it even exists. I remember there were some slides for nginx
internals and Evan M.'s guide, but that’s about it.

nginx mailing list
[email protected]
http://nginx.org/mailman/listinfo/nginx

–
Marcos Luís Ortíz Valmaseda
Software Engineer (Large-Scaled Distributed Systems)
Universidad de las Ciencias Informáticas
Linux User # 418229

18

On 3 Jun 2011 02h22 WEST, [email protected] wrote:

OK, I did this:

  1. Rebuilt nginx with “with-debug”

This is to be done in the configure phase of the build. It’s not a
configuration thing. The only thig needed in the config is the debug
flag in the error_log directive.

You’re confusing two very different things.

  1. Added another line to my nginx.conf:

debug_connection MY IP ADDRESS;

This means that only the requests from this IP generate the debug
log. You need to build a local version on a dev box and debug the
configuration there.

  1. Then restarted the server.

A simple reload will do.

My error_log and access_log are still empty after I visit several
pages of the website.

What else do I need to do?

http://wiki.nginx.org/NginxInstall
http://wiki.nginx.org/NginxInstallOptions

— appa

19

OK, I did this:

  1. Rebuilt nginx with “with-debug”

  2. Added another line to my nginx.conf:

    debug_connection MY IP ADDRESS;
    log_format combined2 '$remote_addr - $remote_user [$time_local] ’
    '“$request” $status $body_bytes_sent ’
    ‘“$http_referer” “$http_user_agent”’;
    error_log logs/error.log debug;
    access_log logs/access.mine.log;

  3. Then restarted the server.

My error_log and access_log are still empty after I visit several pages
of the website.

What else do I need to do?

Posted at Nginx Forum:

20

On 3 Jun 2011 03h08 WEST, [email protected] wrote:

to debug?
Are you making any request? You can do it through the browser, using
curl or wget.

If there’s no request, no debug log will be generated.

Make a request. Request the URI that should invoke the redirect
script, and so on. Go step by step.

— appa