# Ngx\_lua location capture issue

**URL:** <https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942>\
**Category:** NGINX\
**Created:** [October 18, 2011, 6:55pm UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942 "2011-10-18T18:55:24Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![luislavena](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/0ea827/32.png) [@luislavena](https://www.ruby-forum.com/u/luislavena)\
**Post date:** [October 18, 2011, 6:55pm UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/1 "2011-10-18T18:55:24Z")

</div>

I tried to create the following scenario:

1. Request test\_page.php
2. lua exec to @checkpoint
3. @checkpoint does capture location to test\_loc (future phpids)
4. test\_loc/index.php returns either 200 or 403 status
5. @checkpoint continues or halts request accordingly

# GET /test\_page.php

server {  
listen 80;  
server\_name [testsite.com](http://testsite.com);  
root /home/user/testsite.com/public\_html;  
location @checkpoint {  
access\_by\_lua ’  
local res = ngx.location.capture(“/test\_loc”)

```
        if res.status == ngx.HTTP_OK then
            return
        end

        if res.status == ngx.HTTP_FORBIDDEN then
            ngx.exit(res.status)
        end

        ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
    ';
}

```

location @proxy {  
include /etc/nginx/firewall.default;

```
# Block IPs in Spamhaus drop list
if ($block = 1) {
  return 444;
}

proxy_pass http://127.0.0.1:8080;
...

```

}  
location /error\_docs {  
internal;  
alias /home/user/$host/error\_docs;  
}  
location /test\_loc {  
internal;  
alias /usr/share/test\_loc/;  
rewrite\_by\_lua ‘ngx.exec(“@proxy”);’;  
}  
location / {  
try\_files $uri $uri/ @proxy;  
}  
location ~ .+.php$ {  
location ~ ^/test\_page.php$ {  
rewrite\_by\_lua ‘ngx.exec(“@checkpoint”);’;  
}

```
rewrite_by_lua 'ngx.exec("@proxy");';

```

}  
}

Result is “the http output chain is empty while connecting to  
upstream” blah blah blah and output is equivalent to issuing “return  
444”. [http://pastebin.com/7WisVBDU](http://pastebin.com/7WisVBDU)

Logs seem to show “GET /test\_page.php” being run a second time.

Any tips on fixing?

Cheers

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 18, 2011, 7:29pm UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/2 "2011-10-18T19:29:27Z")

</div>

On 18 October 2011 19:55, Nginx U. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> listen 80;  
> if res.status == ngx.HTTP\_FORBIDDEN then  
> if ($block = 1) {  
> location /test\_loc {  
> }
> 
> Any tips on fixing?
> 
> Cheers

A more considered read of the docs shows I had been mixingsubrequests  
and internal redirections all over the place and that this works as  
expected

# GET /test\_page.php

server {  
listen 80;  
server\_name [testsite.com](http://testsite.com);  
root /home/user/testsite.com/public\_html;  
location @proxy {  
include /etc/nginx/firewall.default;

```
# Block IPs in Spamhaus drop list
if ($block = 1) {
  return 444;
}

proxy_pass http://127.0.0.1:8080;
...

```

}  
location /error\_docs {  
internal;  
alias /home/user/$host/error\_docs;  
}  
location /test\_loc {  
internal;  
alias /usr/share/test\_loc/;  
rewrite\_by\_lua ‘ngx.exec(“@proxy”);’;  
}  
location / {  
try\_files $uri $uri/ @proxy;  
}  
location ~ .+.php$ {  
location ~ ^/test\_page.php$ {  
access\_by\_lua ’  
local res = ngx.location.capture(“/phpids”)  
if res.status == ngx.HTTP\_OK then  
ngx.exec(“@proxy\_no\_cache”)  
end  
if res.status == ngx.HTTP\_FORBIDDEN then  
ngx.exit(res.status)  
end  
ngx.exit(ngx.HTTP\_INTERNAL\_SERVER\_ERROR)  
';  
}

```
rewrite_by_lua 'ngx.exec("@proxy");';

```

}  
}

No need for @checkpoint. Just put the access check directly in place.

Cool beans!

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 19, 2011, 4:34am UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/3 "2011-10-19T04:34:02Z")

</div>

On Wed, Oct 19, 2011 at 12:55 AM, Nginx U. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
wrote:

> location /test\_loc {  
> internal;  
> alias /usr/share/test\_loc/;  
> rewrite\_by\_lua ‘ngx.exec(“@proxy”);’;

Could you please try using content\_by\_lua here instead of  
rewrite\_by\_lua? I want to confirm that it is indeed a bug in  
rewrite\_by\_lua rather than content\_by\_lua 🙂

> }  
> location / {  
> try\_files $uri $uri/ @proxy;  
> }  
> location ~ .+.php$ {  
> location ~ ^/test\_page.php$ {  
> rewrite\_by\_lua ‘ngx.exec(“@checkpoint”);’;

Same here 🙂

> Result is “the http output chain is empty while connecting to  
> upstream” blah blah blah and output is equivalent to issuing “return  
> 444”. [http://pastebin.com/7WisVBDU](http://pastebin.com/7WisVBDU)

Thanks for the debug log. It helped a lot 🙂

> Any tips on fixing?

Please try content\_by\_lua instead of rewrite\_by\_lua to call  
ngx.exec(). It can help me fix the issue here 🙂

Thanks for the report!  
-agenzh

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 19, 2011, 4:50am UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/4 "2011-10-19T04:50:02Z")

</div>

On Wed, Oct 19, 2011 at 1:28 AM, Nginx U. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> A more considered read of the docs shows I had been mixingsubrequests  
> and internal redirections all over the place and that this works as  
> expected

Mixing subrequests and internal redirections should be fine for recent  
releases of ngx\_lua. (I’ve found a way to survive my ngx\_lua’s context  
though an internal redirection! :D)

Which part of the documentation is giving you this invalid  
implication? I’d fix the doc 🙂

Thanks!  
-agentzh

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 19, 2011, 11:25am UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/5 "2011-10-19T11:25:23Z")

</div>

On Wed, Oct 19, 2011 at 10:33 AM, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> On Wed, Oct 19, 2011 at 12:55 AM, Nginx U. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> 
> > location /test\_loc {  
> > internal;  
> > alias /usr/share/test\_loc/;  
> > rewrite\_by\_lua ‘ngx.exec(“@proxy”);’;
> 
> Could you please try using content\_by\_lua here instead of  
> rewrite\_by\_lua? I want to confirm that it is indeed a bug in  
> rewrite\_by\_lua rather than content\_by\_lua 🙂

I think I’ve already fixed this issue in the ngx\_lua v0.3.1rc17 release:

```
https://github.com/chaoslawful/lua-nginx-module/tags

```

There was a bug in ngx.exec()'s handling when being used within  
rewrite\_by\_lua\* and access\_by\_lua\* directives, which could cause  
hanging in certain extreme conditions.

Could you please try it out on your side?

Thanks for the report!  
-agentzh

P.S. This new version of ngx\_lua has also been included in the latest  
ngx\_openresty 1.0.8.15 devel release here:

```
http://openresty.org/#Download

```

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 19, 2011, 1:10pm UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/6 "2011-10-19T13:10:33Z")

</div>

On Wed, Oct 19, 2011 at 1:59 AM, Nginx U. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> ```
> if res.status == ngx.HTTP_FORBIDDEN then
> 
> ```
> 
> access by lua within the sub location. Changing the access by lua in  
> the sub location to rewrite by lua executes the script.

The inner location (i.e., location “^/test\_page.php$”) automatically  
inherits the rewrite\_by\_lua directive defined in the outer location  
(i.e., location “.+.php$”). So your rewrite\_by\_lua code runs before  
your access\_by\_lua code in the inner location. And further, your  
rewrite\_by\_lua code does an internal redirection and thus your  
access\_by\_lua never has a chance to run.

Apparently you do not want the inner location to inherit the outer  
rewrite\_by\_lua setting, so just add the following line to your inner  
location:

```
rewrite_by_lua return;

```

This will override the rewrite\_by\_lua directive in the outer scope.

> In any case, my original config is the preferred to get workingas I  
> need the request params to be passed along the redirection chain.

See my last reply 🙂

> Better wait until agentzh wakes up in his timezone.

10:41 AM here already 😉

Best,  
-agentzh

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 19, 2011, 6:46pm UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/7 "2011-10-19T18:46:09Z")

</div>

On 19 October 2011 05:49, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> Which part of the documentation is giving you this invalid  
> implication? I’d fix the doc 🙂  
> Hi. I didn’t mean to imply that the docs said there was anything wrong  
> with this just that it identified that some items were internal  
> redirection and others were subrequests.

I started wondering, all by myself, whether there was an implication  
… with post requests for instance.

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 19, 2011, 1:01pm UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/8 "2011-10-19T13:01:54Z")

</div>

Jumped the gun there. There’s a sting in the tail.

# GET /test\_page.php

```
   location ~ .+\.php$ {
           location ~ ^/test_page\.php$ {
                   access_by_lua '
                           local res = 

```

ngx.location.capture("/phpids")  
if res.status == ngx.HTTP\_OK then  
ngx.exec("@proxy")  
end  
if res.status == ngx.HTTP\_FORBIDDEN then  
ngx.exit(res.status)  
end  
ngx.exit(ngx.HTTP\_INTERNAL\_SERVER\_ERROR)  
';  
}

```
           rewrite_by_lua 'ngx.exec("@proxy");';
   }

```

Will result in the rewrite by lua line being executed instead of the  
access by lua within the sub location. Changing the access by lua in  
the sub location to rewrite by lua executes the script.

In any case, my original config is the preferred to get workingas I  
need the request params to be passed along the redirection chain.

Better wait until agentzh wakes up in his timezone.

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 19, 2011, 8:11pm UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/9 "2011-10-19T20:11:40Z")

</div>

On 19 October 2011 12:24, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > 
> 
> I think I’ve already fixed this issue in the ngx\_lua v0.3.1rc17 release:
> 
> [Tags · openresty/lua-nginx-module · GitHub](https://github.com/chaoslawful/lua-nginx-module/tags)
> 
> There was a bug in ngx.exec()'s handling when being used within  
> rewrite\_by\_lua\* and access\_by\_lua\* directives, which could cause  
> hanging in certain extreme conditions.
> 
> Could you please try it out on your side?

I’ll have a go in due course. I have dumped the config originally  
posted. Will look into reconfiguring along those lines later.

In the meantime, can you help out with my fallback?

I have a set of about 70 regexes that are run using access\_by\_lua\_file  
after finding out that access\_by\_lua has a string length limit  
(undefined).

I have applied the escaping sequences and they generally work but  
these ones generate errors:

– 42  
– original  
“(?:”\s_or\s_"?\d)|(?:\x(?:23|27|3d))|(?:^.?“$)|(?:(?:\[1\]_(?:[\d"]+|[^“]+”))+\s_(?:n?and|x?or|not||||&&)\s\*[\w”[+&!@(),.-])|(?:[^\w\s]\w+\s\*[|-]\s\*“\s\*\w)|(?:@\w+\s+(and|or)\s\*[”\d]+)|(?:@[\w-]+\s(and|or)\s\*[^\w\s])|(?:[^\w\s:]\s\*\d\W+[^\w\s]\s\*“.)|(?:\Winformation\_schema|table\_name\W)”  
local query\_string =  
ngx.re.match(ngx.var.request\_uri,“(?:”\\s_or\\s_"?\\d)|(?:\\\x(?:23|27|3d))|(?:^.?“$)|(?:(?:\[2\]_(?:[\\d"]+|[^“]+”))+\\s_(?:n?and|x?or|not|\|\||\&\&)\\s\*[\\w”[+&!@(),.-])|(?:[^\\w\\s]\\w+\\s\*[|-]\\s\*“\\s\*\\w)|(?:@\\w+\\s+(and|or)\\s\*[”\\d]+)|(?:@[\\w-]+\\s(and|or)\\s\*[^\\w\\s])|(?:[^\\w\\s:]\\s\*\\d\\W+[^\\w\\s]\\s\*“.)|(?:\\Winformation\_schema|table\_name\\W)”,  
“io”)  
– 45  
– original  
“(?:union\s\*(?:all|distinct|[(!@]_)?\s_[(_\s_select)|(?:\w+\s+like\s+")|(?:like\s\*”%)|(?:“\s_like\W_[”\d])|(?:“\s\*(?:n?and|x?or|not  
||||&&)\s+[\s\w]+=\s\*\w+\s_having)|(?:"\s_\*\s\*\w+\W+”)|(?:“\s\*[^?\w\s=.,;)(]+\s\*[(@”]_\s_\w+\W+\w)|(?:select\s\*[[]()\s\w.,“-]+from)|(?:find\_in\_set\s\*()”  
local query\_string = ngx.re.match(ngx.var.request\_uri,  
“(?:union\\s\*(?:all|distinct|[(!@]_)?\\s_[(_\\s_select)|(?:\\w+\\s+like\\s+\")|(?:like\\s\*”\%)|(?:“\\s_like\\W_[”\\d])|(?:“\\s\*(?:n?and|x?or|not  
|\|\||\&\&)\\s+[\\s\\w]+=\\s\*\\w+\\s_having)|(?:"\\s_\*\\s\*\\w+\\W+”)|(?:“\\s\*[^?\\w\\s=.,;)(]+\\s\*[(@”]_\\s_\\w+\\W+\\w)|(?:select\\s\*[\[\]()\\s\\w\.,“-]+from)|(?:find\_in\_set\\s\*\()”,  
“io”)

Thanks

* * *

1. "\ 

2. "\\\

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 20, 2011, 2:44am UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/10 "2011-10-20T02:44:14Z")

</div>

On Thu, Oct 20, 2011 at 2:10 AM, Nginx U. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> I’ll have a go in due course. I have dumped the config originally  
> posted. Will look into reconfiguring along those lines later.

Cool 🙂

> In the meantime, can you help out with my fallback?
> 
> I have a set of about 70 regexes that are run using access\_by\_lua\_file  
> after finding out that access\_by\_lua has a string length limit  
> (undefined).

It is the nginx config file parser has a length limit on individual  
config directives. Use access\_by\_lua\_file instead to put your Lua code  
into an external .lua file and thus eliminating the length constraint  
altogether.

> I have applied the escaping sequences and they generally work but  
> these ones generate errors:
> 
> – 42  
> – original

“(?:”\s_or\s_"?\d)|(?:\x(?:23|27|3d))|(?:^.?“$)|(?:(?:\[1\]_(?:[\d"]+|[^“]+”))+\s_(?:n?and|x?or|not||||&&)\s\*[\w”[+&!@(),.-])|(?:[^\w\s]\w+\s\*[|-]\s\*“\s\*\w)|(?:@\w+\s+(and|or)\s\*[”\d]+)|(?:@[\w-]+\s(and|or)\s\*[^\w\s])|(?:[^\w\s:]\s\*\d\W+[^\w\s]\s\*“.)|(?:\Winformation\_schema|table\_name\W)”

> local query\_string =

ngx.re.match(ngx.var.request\_uri,“(?:”\\s_or\\s_"?\\d)|(?:\\\x(?:23|27|3d))|

A rule of thumb is that "" should become “\\” (four back-slashes!)  
while using access\_by\_lua and “\” (two back-slashes!) while using  
access\_by\_lua\_file. You’re writing things like “\\s” (three  
back-slashes!), which is weird.

Why not put your Lua code into an external .lua file and use  
access\_by\_lua\_file or rewrite\_by\_lua\_file instead? That way you only  
need to escape each "" only once.

Regards,  
-agentzh

* * *

1. "\

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 20, 2011, 10:27am UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/11 "2011-10-20T10:27:58Z")

</div>

On 20 October 2011 03:43, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> On Thu, Oct 20, 2011 at 2:10 AM, Nginx U. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> 
> > I have a set of about 70 regexes that are run using access\_by\_lua\_file  
> > A rule of thumb is that "" should become “\\” (four back-slashes!)  
> > while using access\_by\_lua and “\” (two back-slashes!) while using  
> > access\_by\_lua\_file. You’re writing things like “\\s” (three  
> > back-slashes!), which is weird.
> 
> Why not put your Lua code into an external .lua file and use  
> access\_by\_lua\_file or rewrite\_by\_lua\_file instead? That way you only  
> need to escape each "" only once.  
> I am using an external .lua file and I only get consistent results  
> when I use “\\s” etc. “\s” etc resulted is several “)” expected near  
> “|” type messages.

Basically, it’s what seems to work for me.

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 20, 2011, 10:47am UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/12 "2011-10-20T10:47:12Z")

</div>

On Thu, Oct 20, 2011 at 4:17 PM, Nginx U. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > Why not put your Lua code into an external .lua file and use  
> > access\_by\_lua\_file or rewrite\_by\_lua\_file instead? That way you only  
> > need to escape each "" only once.  
> > I am using an external .lua file and I only get consistent results  
> > when I use “\\s” etc. “\s” etc resulted is several “)” expected near  
> > “|” type messages.

“\\s” is essentially equivalent to “\s” in Lua string literals  
because “\s” evaluates to “s”.

You can try small examples on your shell (ensure you have the “lua”  
interpreter visible in your PATH environment):

```
$ lua -e 'print("\s" == "s")'
true

$ lua -e 'print("\\\s" == "\\s")'
true

$ lua -e 'print("\\s")'
\s

$ lua -e 'print("\\\s")'
\s

```

> Basically, it’s what seems to work for me.

I suggest you design trivial samples like above to get clear what is  
going on here 🙂

Regards,  
-agentzh

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 20, 2011, 1:48pm UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/13 "2011-10-20T13:48:59Z")

</div>

On Thu, Oct 20, 2011 at 6:48 PM, Nginx U. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> Would the Nginx string literal you mentioned before not then turn  
> “\s” into “\s” … which is were I want to be in the end? I suspect I  
> am missing something in the process.

The Nginx config file parser first parses Nginx string literal

```
 'ngx.re.match("\\\\s")'

```

into Lua code

```
ngx.re.match("\\s")

```

and then Lua code parser parse the Lua string literal “\s” into the  
character string

```
\s

```

which is the regex pattern that you want.

> Wish I could just use the familiar "" and have it figured out in the  
> background without me having to worry about it as rewrite apparently  
> does though.

Nginx does have builtin support for regex syntax, so does Perl and  
JavaScript, and that’s why you do not have to esacpe "" in that  
context. Unfortunately Lua does not have such built-in regex syntax  
support and the ngx.re API was implemented by ngx\_lua and by no means  
to be a real a language extension to Lua.

As I said, try using external .lua file and  
content/rewrite/access/set\_by\_lua\_file to avoid nginx string escaping  
issues.

Regards,  
-agentzh

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 20, 2011, 6:03pm UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/14 "2011-10-20T18:03:00Z")

</div>

On 20 October 2011 14:48, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> As I said, try using external .lua file and  
> content/rewrite/access/set\_by\_lua\_file to avoid nginx string escaping  
> issues.

Understood. However, when I follow your instructions on this, things  
fail. They seem to work my way.

Take this regex for example: (?:^\>[\w\s]\*\</?\w{2,}\>)

When I use my “incorrect” escaping in access\_by\_lua file …

```
   local query_string = ngx.re.match(ngx.var.request_uri,

```

“(?:^\>[\\w\\s]\*\<\/?\\w{2,}\>)”, “io”)  
– finds unquoted attribute breaking injections – xss – csrf  
– 2  
if query\_string then  
ngx.exit(ngx.HTTP\_BAD\_REQUEST)  
end

… the debug log entry is …

[debug] 24803#0: _154 lua regex cache miss for match regex  
"(?:^\>[\w\s]_\</?\w{2,}\>)" with options “io”  
[debug] 24803#0: _154 lua compiling match regex  
"(?:^\>[\w\s]_\</?\w{2,}\>)" with options “io” (compile once: 1)  
[debug] 24803#0: \*154 lua saving compiled regex (0 captures) into the  
cache (entries 6)  
[debug] 24803#0: _154 regex "(?:^\>[\w\s]_\</?\w{2,}\>)" not matched on  
string “/trackip/?searchip=213.162.113.89” starting from 0

I.E. the match regex, “(?:^\>[\w\s]\*\</?\w{2,}\>)” is the same as the  
original.

I don’t know why, but it works and the “correct” escaping does not.

So I’m sticking with this until I start to see problems.

Cheers.

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 20, 2011, 12:48pm UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/15 "2011-10-20T12:48:39Z")

</div>

On 20 October 2011 11:46, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> because “\s” evaluates to “s”.  
> Would the Nginx string literal you mentioned before not then turn  
> “\s” into “\s” … which is were I want to be in the end? I suspect I  
> am missing something in the process.

Wish I could just use the familiar "" and have it figured out in the  
background without me having to worry about it as rewrite apparently  
does though.

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 21, 2011, 6:09am UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/16 "2011-10-21T06:09:02Z")

</div>

On Fri, Oct 21, 2011 at 12:02 AM, Nginx U. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
wrote:

> On 20 October 2011 14:48, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:  
> Take this regex for example: (?:^\>[\w\s]\*\</?\w{2,}\>)

Good lord!

Why are you using “^” here? Are you meant to match from the very start  
of your $request\_uri string?

And why are you escaping “/” ? It is _not_ a special thing in the  
regex syntax that requires escaping.

As years of Perl programmer, I must say your regex here is by no means  
correct.

> When I use my “incorrect” escaping in access\_by\_lua file …
> 
> local query\_string = ngx.re.match(ngx.var.request\_uri,  
> “(?:^\>[\\w\\s]\*\<\/?\\w{2,}\>)”, “io”)  
> – finds unquoted attribute breaking injections – xss – csrf  
> – 2  
> if query\_string then  
> ngx.exit(ngx.HTTP\_BAD\_REQUEST)  
> end

I’m not meant to help with Perl compatible regex usage, but here’s my  
working version:

```
-- html/foo.lua
local uri = "<impact>2</impact>"
local regex = '(?:>[\\w\\s]*</?\\w{2,}>)';
ngx.say("regex: ", regex)
m = ngx.re.match(uri, regex, "oi")
if m then
    ngx.say("[", m[0], "]")
else
    ngx.say("not matched!")
end

# nginx.conf
location /re {
    access_by_lua_file html/foo.lua;
    content_by_lua return;
}

```

GET /re yields

```
regex: (?:>[\w\s]*</?\w{2,}>)
[>2</impact>]

```

Regards,  
-agentzh

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 21, 2011, 9:35am UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/17 "2011-10-21T09:35:59Z")

</div>

On 21 October 2011 07:13, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> For example, Forefox will escape “3” into “a=%3Ca%3E3%3C/a%3E”,  
> which will surely never be matched by the regexes used here.
> 
> You can try ngx.unescape\_uri to preprocess the $request\_uri thing first, see:
> 
> [http://wiki.nginx.org/HttpLuaModule#ngx.unescape\_uri](http://wiki.nginx.org/HttpLuaModule#ngx.unescape_uri)
> 
> Good luck!

Thanks for the emails above. I’ll look into tackling the issues  
raised in due course.  
“/” is escaped because the original regex is from a php application  
which uses “/” as a delimiter. I left it in place because the snippet  
posted in just a part of the result of a series of “find and replace”  
sequences on an xml file  
([https://dev.itratos.de/svn/php-ids/trunk/lib/IDS/default\_filter.xml](https://dev.itratos.de/svn/php-ids/trunk/lib/IDS/default_filter.xml))  
that changes it to the lua format. “-- 2” is not the  
target but just a hang over from the original xml I couldn’t find an  
easy find and replace for to cater for all possible “” tags  
and is there as a lua comment.

As said, things are working as expected for me at present in that the  
resultant regexes are consistent with the target regexes from the xml  
file so I am keeping them as they are. I will change them if/when I  
see issues.

In any case, this is just a fallback I put in place when trying to  
call the actual application and having just recompiled with rc17, I’ll  
look into having a go at that again.

Thanks!

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 21, 2011, 6:13am UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/18 "2011-10-21T06:13:57Z")

</div>

On Fri, Oct 21, 2011 at 12:08 PM, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > local query\_string = ngx.re.match(ngx.var.request\_uri,  
> > “(?:^\>[\\w\\s]\*\<\/?\\w{2,}\>)”, “io”)  
> > – finds unquoted attribute breaking injections – xss – csrf  
> > – 2

BTW, it’s bad practice to match against $request\_uri directly because  
query strings may be escaped according to URI escaping rules. (Yes!  
there’s escaping everywhere!)

For example, Forefox will escape “3” into “a=%3Ca%3E3%3C/a%3E”,  
which will surely never be matched by the regexes used here.

You can try ngx.unescape\_uri to preprocess the $request\_uri thing first,  
see:

```
http://wiki.nginx.org/HttpLuaModule#ngx.unescape_uri

```

Good luck!  
-agentzh

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 21, 2011, 9:55am UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/19 "2011-10-21T09:55:47Z")

</div>

On 21 October 2011 10:35, Nginx U. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> In any case, this is just a fallback I put in place when trying to  
> call the actual application and having just recompiled with rc17, I’ll  
> look into having a go at that again.

“… when trying to call the actual application failed …” .

---

<div class="post-metadata">

**Author:** ![Nginx\_U](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/dec6dc/32.png) [@Nginx\_U](https://www.ruby-forum.com/u/Nginx_U)\
**Post date:** [October 21, 2011, 4:01pm UTC](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942/20 "2011-10-21T16:01:43Z")

</div>

On 21 October 2011 07:13, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> BTW, it’s bad practice to match against $request\_uri directly because  
> query strings may be escaped according to URI escaping rules. (Yes!  
> there’s escaping everywhere!)  
> That’s a great point. ngx.unescape.uri helps get over this.

Better still yet, it is the actual arguments I need to be matching  
against and that was just an initial setup thing.

I have since moved on to …

local args = ngx.req.get\_uri\_args()  
for key, val in pairs(args) do  
if type(val) == “table” then  
my\_arg = table.concat(val, ", ")  
else  
my\_arg = val  
end  
if my\_arg then  
local query\_string = ngx.re.match(my\_arg,  
“regex\_1”, “io”)  
…  
local query\_string = ngx.re.match(my\_arg,  
“regex\_n”, “io”)  
…  
end  
end

The val entities are url unescaped by ngx\_lua so no issues with that.

Now that this is all working fine (after using "\" in place of the  
documented “\” to get it to actually work - and it does work while  
the “correct” version does not), I can get back trying out the  
location capture again.

Will let you know how that goes in a new thread since I have derailed  
this one with the regex issue.

Cheers!

[Next page](https://www.ruby-forum.com/t/ngx-lua-location-capture-issue/212942.md?page=2)