Forum: NGINX Equivalent of Apache's SetEnv Variable

Issue:
I'd like to configure Magento to run in multi-domain mode.
I've been successful doing this via Apache in the past.
It seems that Nginx should be equally capable, but I haven't succeeded.

Server Specs:
Nginx (latest)
PHP 5.3.3
PHP-FPM enabled
Magento (latest)

What I Know:
Apache relies on the SetEnv variable in the virtual host definition or
a similar instruction in an .htaccess file to achieve this
functionality. The specifics are here:
http://www.magentocommerce.com/wiki/multi-store_set_up/multiple-website-setup.

What I Don't Know:
Does Nginx have an equivalent to SetEnv?
Can Nginx be configured to imitate this configuration through rewrites
or some other method?

Larger Community to Benefit:
There are a number of people in the Nginx forum, Magento forum, etc.
asking about this. It would be stellar if someone could provide us
with a definitive answer - even if that answer is 'it can't be done'
or 'proxy Nginx to Apache'. If anyone supplies an answer via the
mailing list, I will make sure that the answer is shared in the forums
and elsewhere for maximum benefit.

Thanks so much!

http://wiki.nginx.org/NginxHttpFcgiModule#fastcgi_param

On Thu, Jul 29, 2010 at 11:18:07AM -0500, Raina Gustafson wrote:

> 
> What I Know:
> Apache relies on the SetEnv variable in the virtual host definition or
> a similar instruction in an .htaccess file to achieve this
> functionality. The specifics are here:
> http://www.magentocommerce.com/wiki/multi-store_set_up/multiple-website-setup.
> 
> What I Don't Know:
> Does Nginx have an equivalent to SetEnv?
> Can Nginx be configured to imitate this configuration through rewrites
> or some other method?

Probably, you need

     server {

         location / {
             fastcgi_pass   ...
             fastcgi_param  MAGE_RUN_CODE  base;
             fastcgi_param  MAGE_RUN_TYPE  website;
             ...
         }
     }


--
Igor Sysoev
http://sysoev.ru/en/

Fabulous. Thank you. I will try it and report back asap.

I've tried several variations, but haven't been able to get this
working. The virtual hosts are set up fine, and both domains will load
a home page, but Magento doesn't seem to be picking up on the
MAGE_RUN_CODE or MAGE_RUN_TYPE variables being set. From the secondary
home page, all links (category/product pages) point to those pages on
the main domain. I'm confident everything within the Magento database
configuration is set up correctly.

Here's what I've got:
Primary domain: http://dev.swimkitten.com/
Secondary domain: http://dev.swimkitten.co.uk/

/etc/nginx/nginx.conf - http://pastie.org/1073715
/etc/nginx/conf.d/dev.swimkitten.com.conf - http://pastie.org/1073720
/etc/nginx/conf.d/dev.swimkitten.co.uk.conf - http://pastie.org/1073722
/etc/fastcgi_params - http://pastie.org/1073725

Any further thoughts? I've tried putting the variables in:
location /
and
location ~ \.php$

I haven't yet tried making 2 separate fastcgi_params files and
pointing to the respective ones from within each domain.conf file.

Thanks for any further help with this!

A quick note your fastcgi_param should do it. Are you sure it's in all 
the locations it needs to be?

It should expose those as $_SERVER variables.

Ok. That's really helpful information. Right now, both of these output 
nothing:
<?php echo $_SERVER['MAGE_RUN_CODE']; ?>
<?php echo $_SERVER['MAGE_RUN_TYPE']; ?>

That is what you meant, right? I'll investigate further.

yeah, var_dump($_SERVER) - should show you normal variables.

Do you see $_SERVER['HTTPS']?

I think the key location you're missing is putting it in here:

location ~ \.php$ { ## Execute PHP scripts
        expires      off; ## Do not cache dynamic content
        fastcgi_param  HTTPS $fastcgi_https;
        fastcgi_param  SCRIPT_FILENAME 
$document_root$fastcgi_script_name;
        fastcgi_param  PATH_INFO $fastcgi_script_name;
        fastcgi_pass  127.0.0.1:9000;
        fastcgi_index  index.php;
        include      fastcgi_params; ## See /etc/nginx/fastcgi_params

try adding these?

             fastcgi_param  MAGE_RUN_CODE  base;
             fastcgi_param  MAGE_RUN_TYPE  website;

    }

FYI, I was not aware of an $fastcgi_https variable. If there is one,
that is awesome and I need to add that to my fastcgi_params - it
should be default as most apps seem to rely on $_SERVER['HTTPS']

I am going to go look if it exists or not :)

Ok, great. I just noticed that the Magento config listed on the Nginx
Wiki is way different than the one I'd grabbed from the Magento
forums. I'll wade through those differences in a little while and see
if I can reconcile them intelligently.

I don't have a solid understanding of this stuff - so allow yourself
to be inspired by anything I post at your own risk. :) Trial and error
is my major MO.

Thanks for your insight, Michael.

On Tue, Aug 3, 2010 at 4:45 PM, Raina Gustafson <raina@thequeensmen.com> 
wrote:
> Ok, great. I just noticed that the Magento config listed on the Nginx
> Wiki is way different than the one I'd grabbed from the Magento
> forums. I'll wade through those differences in a little while and see
> if I can reconcile them intelligently.
>
> I don't have a solid understanding of this stuff - so allow yourself
> to be inspired by anything I post at your own risk. :) Trial and error
> is my major MO.
>
> Thanks for your insight, Michael.

I have to say that that config you were using seemed extremely
overkill. However, depending on how Magento is designed (and I haven't
heard a lot of positive feedback about it) it could require a lot of
oddball configuration.

What I've realized over the couple years I've been using nginx is that
most people overengineer their configuration. I hardly ever need more
than a few lines of special sauce for anything I've ran in nginx. Of
course, I'm a minimalist.

On Tue, Aug 3, 2010 at 4:45 PM, Raina Gustafson <raina@thequeensmen.com> 
wrote:
> Ok, great. I just noticed that the Magento config listed on the Nginx
> Wiki is way different than the one I'd grabbed from the Magento
> forums. I'll wade through those differences in a little while and see
> if I can reconcile them intelligently.

btw where do you define this value?

fastcgi_param   HTTPS $fastcgi_https;

I don't see any place in the nginx source code where $fastcgi_https is
a legitimate variable. are you defining it yourself somewhere else?

As I mentioned, I don't really know what I'm doing.

This is where I got my initial configuration from, though, in the
Magento forums. Perhaps the user who posted there can shed some light?
http://www.magentocommerce.com/boards/viewthread/7931/#t211050

Hope that helps. Once I do get this sorted out in a way that is
satisfactory to me, I will post on the wikis. I'll post the steps I've
taken to compile and install the latest versions of PHP, Nginx and
MySQL for Magento, too, as there were a few sticky bits involved. The
information floating around on Nginx+Magento isn't exactly abundant.
I'm going to try and get things working with APC cache and/or Varnish,
too, before I consider this work done. Right now, PHP-FPM is being a
total memory hog, even on my virtual server that has no traffic and
just one default Magento installation. Not quite sure how to tackle
that yet, either. I personally do like Magento a lot, though
optimizing it for performance is a chore!

Thanks again for your time!

So far, so good. I'd tried this location previously, but hadn't tested
with the variable dump, so didn't think it was working. I removed it
from location /, and it is still working. location ~ \.php$ alone
seems to be adequate, and each domain is respecting it's unique
settings in the var dump.

I personally don't use location / unless absolutely needed.

Well magento is not known to be the best piece of software.

I may be working with magento and nginx soon but not sure. Was looking 
forward to seeing how I could tame that beast.

Php-fpm itself isn't a memory hog it's a combination of the php 
built-ins, modules, and configuration and then the code itself. I have 
certain pools that average more ram per process than others - it is most 
likely due to apc usage and/or just the underlying php code. My pool 
averages less because I write pretty tight php code; another client runs 
a vbulletin forum with lots of addons and his memory usage per php 
process is much higher.

On 04/08/2010 01:21, Michael Shadle wrote:
>
> What I've realized over the couple years I've been using nginx is that
> most people overengineer their configuration. I hardly ever need more
> than a few lines of special sauce for anything I've ran in nginx. Of
> course, I'm a minimalist.
>

However, all the default configs that I have seen for PHP setups on the
wiki, etc, seem insecure to my mind.  They nearly all point *all* files
named xx.php to be processed by the your php interpreter.  Coupled with
nearly all non trivial applications having some "upload" feature this
allows a gaping potential issue to upload arbitrary files named xx.php
and you are allowing arbitrary code to be uploaded...

I setup my machines to only point files in limited directories to be
processed by the php interpreter. Coupled with specific handling of any
upload/temp/template/public directories or anywhere else that might
accidently contain something it shouldn't..


See, just checked the wiki.  Surely this example allows you to
immediately upload a new file with a .php suffix and exploit the server?
     http://wiki.nginx.org/NginxMediaWiki
Does Drupal allow uploads?  If so then good luck...
     http://drupal.org/node/110224
Surely Dokuwiki allows uploads?
     http://wiki.nginx.org/Dokuwiki

Make your config secure!  Don't just trust the upload function parsing
and allowing only certain filename patterns!

Good luck

Ed W

On Wed, Aug 4, 2010 at 2:44 PM, Ed W <lists@wildgooses.com> wrote:

> However, all the default configs that I have seen for PHP setups on the
> wiki, etc, seem insecure to my mind.  They nearly all point *all* files
> named xx.php to be processed by the your php interpreter.  Coupled with
> nearly all non trivial applications having some "upload" feature this allows
> a gaping potential issue to upload arbitrary files named xx.php and you are
> allowing arbitrary code to be uploaded...

Someone just posted this on my blog:

location ~ \.php$ {
....
try_files $uri =404;
...
}

exploit http://site.ru/images/as5df3.jpeg/.php

might be an interesting approach, haven't tried it yet. would this add
an additional stat call or two though for every PHP request, Igor?

On Wed, 2010-08-04 at 22:44 +0100, Ed W wrote:

> See, just checked the wiki.  Surely this example allows you to 
> immediately upload a new file with a .php suffix and exploit the server?
>      http://wiki.nginx.org/NginxMediaWiki

Mediawiki doesn't allow that.  It filters by an allowed list of
extensions, and .php isn't among them.

Of course, if you can also let Nginx provide another ounce of
prevention, then all the better.  Unfortunately most PHP applications
expect to be able to run arbitrary PHP scripts from almost any directory
under the sun, so you can either account for each and every script
(hopefully they put included files in a separate directory) or simply
make sure that it's not possible to upload files ending with .php.

Regards,
Cliff

--

On Wed, Aug 04, 2010 at 02:48:07PM -0700, Michael Shadle wrote:

> 
> location ~ \.php$ {
> ....
> try_files $uri =404;
> ...
> }
> 
> exploit http://site.ru/images/as5df3.jpeg/.php
> 
> might be an interesting approach, haven't tried it yet. would this add
> an additional stat call or two though for every PHP request, Igor?

Yes, it adds a stat() syscall, however, it can be eliminated with
open_file_cache. Note also, that it works only if nginx and php are
on the same host.


--
Igor Sysoev
http://sysoev.ru/en/

Yeah I expect nginx to only be aware of the filesystem it has access to. 
So open_file_cache saves stat calls? How do you invalidate the cache if 
a file is removed? Or if I put a new file that wasn't there I want it to 
instantly show up not 404 for a while? Apologies if you've covered this 
already.

On śro, sie 04, 2010 at 02:48:07 -0700, Michael Shadle wrote:
> might be an interesting approach, haven't tried it yet. would this add
> an additional stat call or two though for every PHP request, Igor?

While we're at it, I had an experimental patch some time ago that
provided location mapping based on file extensions instead of URIs which
would prevent the above exploit.

The config looked like:

types {
  # ...
  application/x-httpd-php php;
}

location / {
  root /the/document/root;
}

location @application/x-httpd-php {
  fastcgi_pass ...; # etc.
}


It never went to production but I guess I could refresh and post it if
there's some interest in it and it has a chance of being accepted
upstream (guarded with some config option, of course).

Best regards,
 Grzegorz Nosek

On Thu, Aug 05, 2010 at 12:01:38AM -0700, Michael Shadle wrote:

> Yeah I expect nginx to only be aware of the filesystem it has access to. So open_file_cache saves stat calls? How do you invalidate the cache if a file is removed? Or if I put a new file that wasn't there I want it to instantly show up not 404 for a while? Apologies if you've covered this already.

open file cache entries are valid for open_file_cache_valid time, 60s
by default. If you want to cache file errors such as "not found", you 
may
set open_file_cache_errors on.

> >>> a gaping potential issue to upload arbitrary files named xx.php and you are
> >> exploit http://site.ru/images/as5df3.jpeg/.php
> > Igor Sysoev
> http://nginx.org/mailman/listinfo/nginx
--
Igor Sysoev
http://sysoev.ru/en/

On Thu, Aug 05, 2010 at 09:17:17AM +0200, Grzegorz Nosek wrote:

> > 
>   # ...
> 
> 
> It never went to production but I guess I could refresh and post it if
> there's some interest in it and it has a chance of being accepted
> upstream (guarded with some config option, of course).

How may this prevent from the exploit if a requested file is
"/dir/1.gif/2.php" ? As I understand the file will have
"application/x-httpd-php" type ?


--
Igor Sysoev
http://sysoev.ru/en/

On Thu, Aug 5, 2010 at 12:17 AM, Grzegorz Nosek
<grzegorz.nosek@gmail.com> wrote:

> location @application/x-httpd-php {
>  fastcgi_pass ...; # etc.
> }

I think this would be more appropriate, personally:

type application/x-httpd-php {
   fastcgi_pass ...; # etc.
}

it's not really a location, and needing to use the "@" for prefixing a
named location... all that seems like a hack bolted on to the existing
framework.

it'd be nice to see

type text/plain {
   expires max;
}

type text/css {
   expires max;
}

... etc ...

currently the same thing is a location block with
js|css|ico|gif|jpg|jpeg etc... one single line to max the headers
would be interesting to fill in. and not have it be conflicting with
existing locations.

On Thu, Aug 05, 2010 at 11:29:55AM +0400, Igor Sysoev wrote:
> How may this prevent from the exploit if a requested file is
> "/dir/1.gif/2.php" ? As I understand the file will have
> "application/x-httpd-php" type ?

The patch hooks into the static module ngx_http_static_handler, takes
r->headers_out.content_type and searches for an appropriately named
location. If found, it reroutes the request there.

Like I said, the patch has never seen production use. Also, security
wasn't really the motivation so I may be badly mistaken about it.

Hmm, getting more and more uncertain about it ;) In the example above,
is 1.gif a file (and /2.php the path_info), or is it a directory (and
2.php is a normal file)? -ENOCOFFEE, I guess.

Best regards,
 Grzegorz Nosek

On Thu, Aug 05, 2010 at 10:06:08AM +0200, Grzegorz Nosek wrote:

> wasn't really the motivation so I may be badly mistaken about it.
> 
> Hmm, getting more and more uncertain about it ;) In the example above,
> is 1.gif a file (and /2.php the path_info), or is it a directory (and
> 2.php is a normal file)? -ENOCOFFEE, I guess.

What's about when "/dir/1.gif/2.php" is proxied to remote server ?
nginx has no access to a filesystem of the file.


--
Igor Sysoev
http://sysoev.ru/en/

On Thu, Aug 05, 2010 at 12:35:13AM -0700, Michael Shadle wrote:
> I think this would be more appropriate, personally:
> 
> type application/x-httpd-php {
>    fastcgi_pass ...; # etc.
> }
> 
> it's not really a location, and needing to use the "@" for prefixing a
> named location... all that seems like a hack bolted on to the existing
> framework.

Sure it is, it was the simplest thing that could possibly work. An
upstream submission would have to be much cleaner, of course.

> ... etc ...
> 
> currently the same thing is a location block with
> js|css|ico|gif|jpg|jpeg etc... one single line to max the headers
> would be interesting to fill in. and not have it be conflicting with
> existing locations.

I like your proposed syntax much better, actually.

Best regards,
 Grzegorz Nosek

On Thu, Aug 05, 2010 at 12:09:33PM +0400, Igor Sysoev wrote:
> What's about when "/dir/1.gif/2.php" is proxied to remote server ?
> nginx has no access to a filesystem of the file.

It doesn't go via the static module then and the patch won't do
anything.

Best regards,
 Grzegorz Nosek

On Thu, Aug 5, 2010 at 1:10 AM, Grzegorz Nosek 
<grzegorz.nosek@gmail.com> wrote:

> I like your proposed syntax much better, actually.

Of course :) I'm a smart guy!

It's sort of a meta/fake location, so a location block doesn't make
sense. It maps to types, so a "type" keyword makes sense :)

On Thu, Aug 05, 2010 at 10:11:29AM +0200, Grzegorz Nosek wrote:

> On Thu, Aug 05, 2010 at 12:09:33PM +0400, Igor Sysoev wrote:
> > What's about when "/dir/1.gif/2.php" is proxied to remote server ?
> > nginx has no access to a filesystem of the file.
> 
> It doesn't go via the static module then and the patch won't do
> anything.

The issue is that someone is able to upload a image file to a directory
with scripts (I do not know why he is not able to override some valid
images or even the scripts themself in this case). Then someone requests
the image file as "/dir/1.gif/2.php" making exploit. I do not see
how using types will help in a case when nginx ahs not access to remote
filesystem.


--
Igor Sysoev
http://sysoev.ru/en/

On Thu, Aug 05, 2010 at 12:19:22PM +0400, Igor Sysoev wrote:
> The issue is that someone is able to upload a image file to a directory
> with scripts (I do not know why he is not able to override some valid
> images or even the scripts themself in this case). Then someone requests

I guess it comes from apache-land, where the simplest config is "run all
.php files via the interpreter" (compared to nginx's "run all files 
under
this directory via the interpreter"). The directory where the user is
able to upload files is often a subdirectory of the application, as in:

/index.php
/foo.php
/uploads
        /1.gif
        /2.png
/images
       /foo.gif

The more naive apps simply allow uploading of everything and store that
under /uploads, while the smarter ones try to filter files for validity.
So you cannot overwrite /foo.php or /images/foo.gif (barring directory
traversal bugs...) but can upload /uploads/anythingyoulike.gif

> the image file as "/dir/1.gif/2.php" making exploit. I do not see
> how using types will help in a case when nginx ahs not access to remote
> filesystem.

It won't help at all in that case. The proposed types would be a feature
of the static module. It wouldn't be useful in single-huge-site
deployments, only for sites running on a single server (Nginx+PHP).

Best regards,
 Grzegorz Nosek

Thu, Aug 5, 2010 at 1:49 AM, Grzegorz Nosek <grzegorz.nosek@gmail.com> 
wrote:

> It won't help at all in that case. The proposed types would be a feature
> of the static module. It wouldn't be useful in single-huge-site
> deployments, only for sites running on a single server (Nginx+PHP).

or massive sites that use nginx+php/php-fpm on the same node. it's a
nice (and common) practice. also throw on a memcached instance to add
any extra memory to the cluster ;)

On Thu, Aug 05, 2010 at 10:49:52AM +0200, Grzegorz Nosek wrote:

> /index.php
> traversal bugs...) but can upload /uploads/anythingyoulike.gif
Then it can be easy fixed by

   location ^~ /uploads/ {
or
   location ~ ^/uploads/ {

> > the image file as "/dir/1.gif/2.php" making exploit. I do not see
> > how using types will help in a case when nginx ahs not access to remote
> > filesystem.
> 
> It won't help at all in that case. The proposed types would be a feature
> of the static module. It wouldn't be useful in single-huge-site
> deployments, only for sites running on a single server (Nginx+PHP).

I never liked this Apache idea of internal MIME-types such as
application/x-httpd-php, text/x-server-parsed-html, etc.
I believe it confuses and complicates things.


--
Igor Sysoev
http://sysoev.ru/en/

On Thu, Aug 05, 2010 at 01:58:04PM +0400, Igor Sysoev wrote:
> Then it can be easy fixed by
> 
>    location ^~ /uploads/ {
> or
>    location ~ ^/uploads/ {

Sure. However this requires cooperation between the application user and
the server admin, which isn't always possible (e.g. in a shared hosting
setup it's infeasible to configure every application according to its
needs; everything must be set up well enough to work with automated
configuration).

> I never liked this Apache idea of internal MIME-types such as
> application/x-httpd-php, text/x-server-parsed-html, etc.
> I believe it confuses and complicates things.

The Apache approach to e.g. FastCGI is positively insane, but (for me at
least) the major problem is tying everything to filesystem paths, not
assigning everything a MIME type.

Thankfully, those days are gone by, but I remember setting up a fake
path to some nonexistent ScriptAlias'ed directory(?) just to run a
FastCGI server. I sincerely hope Nginx never goes that way ;) but if
a URL has been resolved to a real file on disk, we do have a proper MIME
type we could use. Apart from the internal types like you (and I)
mentioned, Mike proposed a less controversial (IMHO) use case:

type text/css {
  expires max;
}

I think it's quite clean and conveys the idea better than:

location ~ \.css$ {
  expires max;
}

Best regards,
 Grzegorz Nosek

On Thu, Aug 5, 2010 at 3:16 AM, Grzegorz Nosek 
<grzegorz.nosek@gmail.com> wrote:

> type text/css {
>  expires max;
> }
>
> I think it's quite clean and conveys the idea better than:
>
> location ~ \.css$ {
>  expires max;
> }

I won't say that I am absolutely in support of this, just wanted to
add in my few cents on what the syntax should be, if it was to be
something.

My biggest pet peeve is the conflicting location blocks. I've often
mentioned a "super location" idea, something that gets executed at the
very end of all the other rules, so that something like adding expires
headers can be done at the very end.

The idea of using this "type" methodology is nice though, as nginx
could in theory factor it in at the very end before it spits out the
content, examining the content-type header to determine "hey, this is
text/css! I should apply these rules on it" and not having to setup
regexp'ed locations that could conflict with other location blocks.

However, the biggest thing here is to see if there is a magic way to
stop "any url ending in .php to be parsed as PHP" using existing nginx
configuration or a small patch. That would make anyone who says there
is any sort of security hazard go back into their caves (as nginx
rarely has anything to be concerned about!)

On czw, sie 05, 2010 at 03:22:19 -0700, Michael Shadle wrote:
> My biggest pet peeve is the conflicting location blocks. I've often
> mentioned a "super location" idea, something that gets executed at the
> very end of all the other rules, so that something like adding expires
> headers can be done at the very end.

I'd settle for a clean way to cover all of /foo (spanning several
locations, some of them being regex ones) with authentication or other
access controls. I'm currently using an ugly hack in Nginx code for
that. So I guess we're aiming for roughly the same thing.

> The idea of using this "type" methodology is nice though, as nginx
> could in theory factor it in at the very end before it spits out the
> content, examining the content-type header to determine "hey, this is
> text/css! I should apply these rules on it" and not having to setup
> regexp'ed locations that could conflict with other location blocks.

Would you expect it to be processed after static requests only or also
after *_pass?

> However, the biggest thing here is to see if there is a magic way to
> stop "any url ending in .php to be parsed as PHP" using existing nginx
> configuration or a small patch. That would make anyone who says there
> is any sort of security hazard go back into their caves (as nginx
> rarely has anything to be concerned about!)

It's effectively impossible if Nginx doesn't run on the same node as the
backend as it requires matching .../foo.php the URL with .../foo.php the
file and acting upon the results. That's why I think it belongs as an
extension in the static module.

Best regards,
 Grzegorz Nosek

On Thu, Aug 05, 2010 at 12:16:32PM +0200, Grzegorz Nosek wrote:

> needs; everything must be set up well enough to work with automated
> Thankfully, those days are gone by, but I remember setting up a fake
> I think it's quite clean and conveys the idea better than:
> 
> location ~ \.css$ {
>   expires max;
> }

Well, how can you express this

    location ~ ^/dev/.*\.css$ {
        root   /path/to/dev;
        expires off;
    }

    location ~ \.css$ {
        root   /path/to/production;
        expires max;
    }

using mime-types ?


--
Igor Sysoev
http://sysoev.ru/en/

On Thu, Aug 5, 2010 at 3:40 AM, Grzegorz Nosek 
<grzegorz.nosek@gmail.com> wrote:

> Would you expect it to be processed after static requests only or also
> after *_pass?

I don't know :) All I know is I want to apply rules like expires
headers and such at the very end - those could be CSS/JS files being
generated on the fly by PHP for instance (which is horrible, but oh
well) and I'd like those to be treated just like static .css or .js
files. A "type" fixes that, a filename regexp does not. It's just an
example. That example -may- have just unraveled my whole point of the
discussion, too. It's late :)

> It's effectively impossible if Nginx doesn't run on the same node as the
> backend as it requires matching .../foo.php the URL with .../foo.php the
> file and acting upon the results. That's why I think it belongs as an
> extension in the static module.

understood. I'd say most of us are using things like try_files and if
(-e $request_filename) type checks already, so nginx is aware of the
files themselves, hence that interesting idea of a try_files $uri
inside of the location for .php files... unless I'm missing something
which is blatantly obvious in the internals of course :)

On czw, sie 05, 2010 at 02:44:19 +0400, Igor Sysoev wrote:
>     }
> 
> using mime-types ?

Dunno, how about:

location /dev {
  root /path/to/dev;
  # ...
  type text/css {
    expires off;
  }
}

location / {
  root /path/to/production;
  # ...
  type text/css {
    expires max;
  }
}

1. Yes, this requires nesting type{} in location{}

2. I don't expect everybody to drop regex locations in favour of type{},
I just expect it'll make some types of configuration easier, just like
try_files did.

Best regards,
 Grzegorz Nosek

On czw, sie 05, 2010 at 03:45:09 -0700, Michael Shadle wrote:
> example. That example -may- have just unraveled my whole point of the
> discussion, too. It's late :)

Oh, so my patch wouldn't help you at all for dynamically generated css
files. You'd have to teach your app to send the proper headers itself.

> understood. I'd say most of us are using things like try_files and if
> (-e $request_filename) type checks already, so nginx is aware of the
> files themselves, hence that interesting idea of a try_files $uri
> inside of the location for .php files... unless I'm missing something
> which is blatantly obvious in the internals of course :)

Yes, Nginx is aware of files, of course :) but unlike Apache, where
everything must be a file, it prefers operating on URLs and uses files
as late as possible. That's my understanding of its philosophy, at
least.

Best regards,
 Grzegorz Nosek

On Thu, Aug 05, 2010 at 12:53:26PM +0200, Grzegorz Nosek wrote:

> Yes, Nginx is aware of files, of course :) but unlike Apache, where
> everything must be a file, it prefers operating on URLs and uses files
> as late as possible. That's my understanding of its philosophy, at
> least.

I believe NSCA/Apache was being developed primarily as static web 
server.
So all these <Directory>, <Files>, .htaccess, etc exist.
Then mod_proxy module has been added. In nginx static files processing
is just a part of functionality, so it uses "location"s only (analog of
Apache's <Location>) to concentrate processing in a single place.


--
Igor Sysoev
http://sysoev.ru/en/

On Thu, Aug 05, 2010 at 03:09:14PM +0400, Igor Sysoev wrote:

> On Thu, Aug 05, 2010 at 12:53:26PM +0200, Grzegorz Nosek wrote:
> 
> > Yes, Nginx is aware of files, of course :) but unlike Apache, where
> > everything must be a file, it prefers operating on URLs and uses files
> > as late as possible. That's my understanding of its philosophy, at
> > least.
> 
> I believe NSCA/Apache was being developed primarily as static web server.

I should say "as static/CGI web server".

> So all these <Directory>, <Files>, .htaccess, etc exist.
> Then mod_proxy module has been added. In nginx static files processing
> is just a part of functionality, so it uses "location"s only (analog of
> Apache's <Location>) to concentrate processing in a single place.


--
Igor Sysoev
http://sysoev.ru/en/

Slow reply - apologies

>> See, just checked the wiki.  Surely this example allows you to
>> immediately upload a new file with a .php suffix and exploit the server?
>>       http://wiki.nginx.org/NginxMediaWiki
> Mediawiki doesn't allow that.  It filters by an allowed list of
> extensions, and .php isn't among them.

Uh uh.  As demonstrated above, you appear to be able to bypass this by
adding /.php to the URL

Lets spell this out.  Anyone using these suggested default
configurations is massively vulnerable to injected php scripts.  Just
upload an "phpinfo()" script named hello.jpg then browse to
http://url/hello.jpg/.php and watch as your php gets executed

There may be a bunch of ways that this isn't a vulnerability on all
configurations, but the point is this is a *default* and a simple scan
of a bunch of nginx machines is going to reveal a high count of machines
vulnerable to injection?  Not sure why this isn't getting more 
attention?

Look I'm being a bit subtle about this, but I would have thought this
was an urgent scramble to fix kind of thing?

> Of course, if you can also let Nginx provide another ounce of
> prevention, then all the better.

This isn't an nginx "problem", it's a configuration problem.  "we" (the
wiki) are advising people to incorrectly configure their PHP apps

>    Unfortunately most PHP applications
> expect to be able to run arbitrary PHP scripts from almost any directory
> under the sun,

Hmm, I disagree on "most".  The small selection that I use tend to use
only a small number of locations.

In my config I either:
- Move the data dirs out of the htdocs patch altogether (gallery2 and
others support this)
- Configure only certain files or restricted directories to be treated
as PHP apps
- Reluctantly I might do the reverse and allow all .php files to be
treated as PHP, but then add an "if" in the location to prevent the data
dirs from being scanned.  I needed to do this for mediawiki for
example.  Note that this is a tricky config because generally you use a
regexp location for the .php and so you can't override it with a
"location /uploads" since it's parsed later - instead you need to add an
"If" into the php location...

>   or simply
> make sure that it's not possible to upload files ending with .php.

This is a partial solution at best.  As shown previously it doesn't
actually help you for a lot of configuration examples since trivially
appending /.php on the end allows you to get the file to be parsed by
the PHP interpretor under many configurations?



Webapps are often built assuming apache and will ship with a bunch of
.htaccess files that stop the php interpretor running files in that
location, ie the upload files dir will have a .htaccess in it turning
off the php interpretor.  Now even under Apache this is hit and miss
because many installations will disable htaccess files for performance
reasons (or not allow all actions from the .htaccess).  Of course when
you install your PHP app on nginx I would guess it's quite normal for
the operator to forget to scan for .htaccess files and translate these
to nginx config rules (not nginx's fault, this is an app configuration
issue)

No one seems quite as excited about this as I feel?  What am I missing?

Ed W

On Thu, Aug 26, 2010 at 4:07 AM, Ed W <lists@wildgooses.com> wrote:
>  Slow reply - apologies

There is a quick hack, that should work, but does add at least another
stat call or two. Basically it's one last additional "is this -really-
a file" check. Of course it does require nginx to have filesystem
access (not a remote fastcgi_pass)

It was something along the lines of:

location ~ \.php$ {
    try_files $uri @404; # have to make a named 404 location then.
    fastcgi_pass 127.0.0.1:11000;
}

Or this is probably a bit clearer, but more evil because it uses "if"

location ~ \.php$ {
    if (!-f $request_filename) { return 404; } # or whatever you want to 
return
    fastcgi_pass 127.0.0.1:11000;
}

On 8/26/10 7:07 AM, Ed W wrote:


> No one seems quite as excited about this as I feel? What am I missing?

What you say is true, that such a file would be parsed as PHP if
requested in that manner but it needs to be uploaded successfully first.
Most modern PHP based galleries will not upload a file ending with
".jpg" unless it actually is a JPEG. Same with a file misidentified as a
PNG. Try it with a phpinfo script. It won't upload into apps like
vBulletin or IPB. I can't speak for a lot of others since I haven't
tested them.

If an app does upload a misidentified file so easily, then the onus is
on the webmaster to configure nginx correctly or, more simply, to not
use the app or to not allow uploads from untrusted sources. The method
proposed by Mike will work fine for such insecure apps, but the real fix
is to fix the app.

The "try_files" approach will be much more efficient than any "if" will
be if you insist on using an insecure app.


--
Jim Ohlstein

On 27/08/2010 03:20, Jim Ohlstein wrote:
> into apps like vBulletin or IPB. I can't speak for a lot of others 
>
I still think you guys aren't getting the point?

The wiki *documents* an insecure method of setting up many PHP
applications.  It's almost the number one security vulnerability that
you MUST secure your uploads directory and assuming that the PHP
application can/will do this perfectly is optimistic at best.  A little
help from the webserver is your minimum second layer of defence really.
In fact many "secure" PHP applications delegate security of the upload
directory completely to the webserver and ship with .htaccess files to
disable php access...

It's not about whether you and I can setup a secure nginx setup, it's
about the wiki encourage perhaps tens of thousands of users to setup
their systems so that they can be *trivially* broken into.  Of course we
can all sit back and call the users who followed "our" instructions
idiots when this turns into a mass epidemic, but in the meantime there
is a huge encouragement to setup a bunch of PHP apps so that they can be
trivially broken into?

You are correct that all kinds of vague ways can defend against various
aspects of this.  However, bottom line is that there are few perfect php
applications which perfectly filter uploads.  There are a lot of clever
ways to sneak junk past upload filters

Just to knock one final myth on the head.  Construct a .jpg file like
this and it appears to me that it will work fine and also slip past many
file filters:

# echo -e "\xff\xd8\xff\xe0" > test.jpg
# file test.jpg
test.jpg: JPEG image data
# php test.jpg
????
hello


The *correct* solution is to:
- Enable PHP for the minimum number of file locations possible
- Treat the upload directory as a special case and disable as much as
possible there (beware ordering in conf files)
- If possible use try_files on certain locations to further limit scope
of attack (may not be possible everywhere)

Why doesn't anyone think this is a huge accident waiting to blowup?

Ed W