URL encoding and other hackery

Hi. First, let me just say that I love nginx. Thanks for creating and
maintaining it - we appreciate it.

I am using nginx as the front end to a rails cluster. When rails
generates a page I write the page to disk, where nginx can look for it
later. I want to use something like this:

if (-f $document_root/$uri)

But I anticipate a few problems:

  1. the uri might include “…” or similar hackery
  2. the uri might include query parameters

That leads to my questions:

  1. Does nginx validate incoming uris? Will it strip out “…”?
  2. Can I URL encode a variable?

Thanks!

Adam

On 2/17/08, Adam D. [email protected] wrote:

I am using nginx as the front end to a rails cluster. When rails
generates a page I write the page to disk, where nginx can look for it
later. I want to use something like this:

We’re doing the same thing.

if (-f $document_root/$uri)

This works:

if (-f $request_filename) {
  break;
}

But I anticipate a few problems:

  1. the uri might include “…” or similar hackery

It’s up to you to ensure that the saved file corresponds to the file
name Nginx generates in $request_filename. Nginx will URL-decode the
path, which means that

http://example.com/buttons/a%2F..%2Fbutton.png

will resolve to

$document_root/buttons/a/…/button.png

which is expanded to

$document_root/buttons/button.png

…which is the file name that Rails’ cache_page method will use.

In other words it’s possible to generate URLs which may end up outside
your designated document root. The risk is not Nginx that could try to
serve stuff beyond the document root, but that the Rails app might
write its cache file in unexpected places.

I don’t know if cache_page validates the file name to ensure it’s
within $RAILS_ENV/public. It certainly ought to. There’s also the risk
that cache_page could overwrite other files within the public
directory, of course, such as stuff in public/images.

  1. the uri might include query parameters

If so you will need to create an Nginx variable to appends the query
string to the end of the $request_filename. But a better option is to
rely on Rails routes. Here’s a typical route we use for rendering
buttons:

map.connect ‘cache/button/:id’, :controller => “theme”, :action =>
‘button’,
:requirements => {:id => /.*/}

This will map a URL such as this:

http://example.com/cache/button/style=green;text=Click+me.png

to a controller action as well as a nicely readable file name within
our cache directory. In the controller we parse the file name and do
the rendering:

def button
options = Button.parse_options(params[:id])
button = Button.new(options)

data = button.render.to_blob
send_data(data, :type => button.content_type, :disposition =>
“inline”)
cache_page(data)
end

Alexander.

On Sat, Feb 16, 2008 at 08:44:00PM -0800, Adam D. wrote:

Hi. First, let me just say that I love nginx. Thanks for creating and
maintaining it - we appreciate it.

I am using nginx as the front end to a rails cluster. When rails
generates a page I write the page to disk, where nginx can look for it
later. I want to use something like this:

if (-f $document_root/$uri)

You should use $request_filename instead: - it’s “$document_root$uri”.
However, $request_filename correctly handle “root” as “alias”.

But I anticipate a few problems:

  1. the uri might include “…” or similar hackery
  2. the uri might include query parameters

$uri and $request_filename does not contains query parameters.
The query parameters are available via $args or $query_string (the later
is for compatibilty with Apache).

That leads to my questions:

  1. Does nginx validate incoming uris? Will it strip out “…”?

Yes, nginx processes various /./, /…/ in clear and escaped form,
and does not allow to to below URI’s root.

  1. Can I URL encode a variable?

I do not understand the question.

On Sun, Feb 17, 2008 at 06:47:20PM +0100, Manlio P. wrote:

Igor S. ha scritto:

[…]
$uri and $request_filename does not contains query parameters.
The query parameters are available via $args or $query_string (the later
is for compatibilty with Apache).

By the way, I have noted that the uri fragment is removed? Why?

Do you mean “/uri#fragment” ? It is removed by browser, but not nginx.

Igor S. ha scritto:

[…]
$uri and $request_filename does not contains query parameters.
The query parameters are available via $args or $query_string (the later
is for compatibilty with Apache).

By the way, I have noted that the uri fragment is removed? Why?

[…]

Thanks Manlio P.

Igor S. wrote:

later. I want to use something like this:

if (-f $document_root/$uri)

You should use $request_filename instead: - it’s “$document_root$uri”.
However, $request_filename correctly handle “root” as “alias”.

I’m looking for a flavor of $request_filename that includes query
arguments, so that I can cache a version of the page that includes the
arguments. At the very least, I need to differentiate between a version
WITH arguments and a version WITHOUT arguments. I don’t want those two
requests to serve up the same page. This is very important for SEO
purposes, to avoid duplicate content being shown at two different urls.

I could use this:

if (-f $document_root/$request_uri)

But there are escaping issues. Perhaps nginx needs something like this:

$request_filename_with_args

  1. Can I URL encode a variable?

I do not understand the question.

Another solution to my problem would be something like:

set $path url_escape($request_uri)
if (-f $document_root/$path) {

Adam

What escaping issues? $request_uri will already be uri-encoded, so
you can use it directly on the file system. You just need to make sure
whatever’s behind nginx making those files have ordered the parameters
correctly.

for example, given a request $request_uri = /path/to/some.xml?a=%22&b=
%23

$document_root$request_uri ==> /path/to/root/path/to/some.xml?a=%22&b=
%23

this is a valid path in most file systems.

(note, there’s no slash between document_root and request_uri)

Perhaps nginx needs something like this:
$request_filename_with_args

Of course, you can just use: $request_filename$is_args$args
without if () {} statement. :slight_smile: