Strange rewrite_by_lua outcome

luislavena · October 16, 2011, 10:12pm

This works produces a 403 Forbidden error as expected:
#GET /submit_links/
if ($request_uri ~*
:|[|]|--|@|^|{|}|~|<|>|..|++|//|%0|%A|%B|%C|%D|%E|%F|%22|%27|%28|%29|%3C|%3E|%5C|%7B|%7C|%7D|select(\s*)(|convert(\s*)(|/query/|function.|remoteFile|servername|&rptmode=|/(null)/|(maincore|authorize|macromates|head_auth|submit_links|change_action|admin_db_utilities|admin.webring.docs|Table/Latest/index).|w00t|MNG/LIVE|/x[0-9][0-9]|\x[0-9][0-9]|/(cgi|https?)/|.css(|)+|/,/|{0}|eval(|_vti_|(null)|echo.*kae|function(.array-rand|())
) {
return 403;
}

This produces a 500 Internal server error instead:
#GET /submit_links/
rewrite_by_lua ’
local request_uri = ngx.re.match(ngx.var.request_uri,
“:|[|]|--|@|^|{|}|~|<|>|..|++|//|%0|%A|%B|%C|%D|%E|%F|%22|%27|%28|%29|%3C|%3E|%5C|%7B|%7C|%7D|select(\s*)(|convert(\s*)(|/query/|function.|remoteFile|servername|&rptmode=|/(null)/|(maincore|authorize|macromates|head_auth|submit_links|change_action|admin_db_utilities|admin.webring.docs|Table/Latest/index).|w00t|MNG/LIVE|/x[0-9][0-9]|\x[0-9][0-9]|/(cgi|https?)/|.css(|)+|/,/|{0}|eval(|_vti_|(null)|echo.*kae|function(.array-rand|())”,
“io”)
if request_uri then
ngx.exit(ngx.HTTP_FORBIDDEN)
end
';

Line 62: “23388#0: *258 Failed to load Lua inlined code: [string
“rewrite_by_lua”]:24: ‘)’ expected near ‘|’, client: xx.xxx.xx.xx,
server: testsite.com, request: “GET /submit_links HTTP/1.1”, host:
“testsite.com””
http://pastebin.com/XYNwEciX

Nginx_U · October 17, 2011, 2:56am

On Mon, Oct 17, 2011 at 4:11 AM, Nginx U. [email protected] wrote:

This produces a 500 Internal server error instead:
#GET /submit_links/
rewrite_by_lua ’
local request_uri = ngx.re.match(ngx.var.request_uri,

“:|[|]|--|@|^|{|}|~|<|>|..|++|//|%0|%A|%B|%C|%D|%E|%F|%22|%27|%28|%29|%3C|%3E|%5C|%7B|%7C|%7D|select(\s*)(|convert(\s*)(|/query/|function.|remoteFile|servername|&rptmode=|/(null)/|(maincore|authorize|macromates|head_auth|submit_links|change_action|admin_db_utilities|admin.webring.docs|Table/Latest/index).|w00t|MNG/LIVE|/x[0-9][0-9]|\x[0-9][0-9]|/(cgi|https?)/|.css(|)+|/,/|{0}|eval(|_vti_|(null)|echo.*kae|function(.array-rand|())”,

“io”)

I think you should escape "" here according to nginx’s string literal
syntax (you do not need to escape "" in your previous “if” example
because Nginx uses special regex syntax in that regex context. You’ll
also have to escape the "" according to Lua string escaping rules
too. That is, escaping "" twice.

Line 62: “23388#0: *258 Failed to load Lua inlined code: [string
“rewrite_by_lua”]:24: ‘)’ expected near ‘|’, client: xx.xxx.xx.xx,
server: testsite.com, request: “GET /submit_links HTTP/1.1”, host:
“testsite.com””
http://pastebin.com/XYNwEciX

The error message indicates a Lua syntax error in your Lua source
code. A quick solution to this is to put your code into a separate
.lua file and use rewrite_by_lua_file instead such that you do not
have to escape "" twice (once for Nginx string escaping rules and
another for Lua string escaping rules).

Here’s an example:
# nginx.conf
location /foo {
rewrite_by_lua ‘ngx.re.match(“\\d+”, ngx.var.uri)’; # we’re
using \d+ here but requires twice escaping
}

And to use external .lua file:

# nginx.conf
location /foo {
    rewrite_by_lua_file conf/my.lua;
}

# conf/my.lua
ngx.re.match("\\d+", ngx.var.uri)

We can see that in the first example, we need to escape "" for Nginx
string literals (so it becomes “\”) and then escape each of these two
back-slashes again according for Lua string literals and thus got
“\\” at last.

But in the second example, we only need to esacpe "" once according
to Lua string literal syntax, which is a bit better looking

This gotcha is documented in ngx_lua’s wiki page here:

http://wiki.nginx.org/HttpLuaModule#ngx.re.match

Best,
-agentzh

Nginx_U · October 17, 2011, 6:48am

On 17 October 2011 03:55, agentzh [email protected] wrote:

syntax (you do not need to escape "" in your previous “if” example

rewrite_by_lua ‘ngx.re.match(“\\d+”, ngx.var.uri)’; # we’re

conf/my.lua

This gotcha is documented in ngx_lua’s wiki page here:

Lua | NGINX

I’ll use the rewrite module for this regex since it works as is and
try to understand the issue later.

Thanks.

Nginx_U · October 17, 2011, 7:04am

On 10/16/2011 10:50 PM, Nginx U. wrote:

BTW this works fine even though it has “”:

local query_string = ngx.re.match(ngx.var.request_uri,

You’re just doing “.” in that line. If Nginx strips that “”, then it
ends up in Lua as “.”, which changes the meaning but will happen to work
in most cases (though it would match sqlpatch_php and other similar
strings, and not just sqlpatch.php, since the “.” will be the wildcard).

Tim

Nginx_U · October 17, 2011, 6:34pm

On 17 October 2011 08:03, Tim M. [email protected] wrote:

cases (though it would match sqlpatch_php and other similar strings, and not
just sqlpatch.php, since the “.” will be the wildcard).

Tim

Let me get this right. Do I need to always escape the "" inserted to
escape “.” in “.” in lua and then add another couple of ""s? I.E.,
do I need to do “sqlpatch\\.php” to get “sqlpatch.php” passed to
the regex finally? IOW, do I basically replace each "" with “\\”?

If I understand agentzh, the rewrite module handles this behind the
scenes and allows us to use familiar syntax. Is that right?

Thanks

Nginx_U · October 17, 2011, 7:45pm

On 10/17/2011 10:33 AM, Nginx U. wrote:

Let me get this right. Do I need to always escape the “” inserted
to escape “.” in “.” in lua and then add another couple of ""s?
I.E., do I need to do “sqlpatch\\.php” to get “sqlpatch.php”
passed to the regex finally? IOW, do I basically replace each “”
with “\\”?

Yes.

If I understand agentzh, the rewrite module handles this behind the
scenes and allows us to use familiar syntax. Is that right?

The rewrite module apparently changes how the string is parsed in the
Nginx file, yes.

Tim

Nginx_U · October 17, 2011, 7:55pm

On 17 October 2011 20:45, Tim M. [email protected] wrote:

On 10/17/2011 10:33 AM, Nginx U. wrote:

Let me get this right. Do I need to always escape the "" inserted
to escape “.” in “.” in lua and then add another couple of ""s?
I.E., do I need to do “sqlpatch\\.php” to get “sqlpatch.php”
passed to the regex finally? IOW, do I basically replace each ""
with “\\”?

Yes.
Thanks. I had done that anyway and got expected results. Not strictly
changing "" to “\\”. It was “\” to "\\" & "" to “\\”

If I understand agentzh, the rewrite module handles this behind the
scenes and allows us to use familiar syntax. Is that right?

The rewrite module apparently changes how the string is parsed in the Nginx
file, yes.
Very thoughtful.

Cheers.

Nginx_U · October 17, 2011, 6:51am

On 17 October 2011 07:47, Nginx U. [email protected] wrote:

I think you should escape "" here according to nginx’s string literal

http://pastebin.com/XYNwEciX
location /foo {

This gotcha is documented in ngx_lua’s wiki page here:

Lua | NGINX

I’ll use the rewrite module for this regex since it works as is and
try to understand the issue later.

Thanks.

BTW this works fine even though it has "":

Haven’t read the gotcha link yet to understand the intricacies yet
though.

Cheers.