# Nginx as reverse proxy with "mod secdownload" feature - possible?

**URL:** <https://www.ruby-forum.com/t/nginx-as-reverse-proxy-with-mod-secdownload-feature-possible/197731>\
**Category:** NGINX\
**Created:** [October 19, 2010, 4:37pm UTC](https://www.ruby-forum.com/t/nginx-as-reverse-proxy-with-mod-secdownload-feature-possible/197731 "2010-10-19T16:37:39Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![okkezSS](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/o/85f322/32.png) [@okkezSS](https://www.ruby-forum.com/u/okkezSS)\
**Post date:** [October 19, 2010, 4:37pm UTC](https://www.ruby-forum.com/t/nginx-as-reverse-proxy-with-mod-secdownload-feature-possible/197731/1 "2010-10-19T16:37:39Z")

</div>

Hi,

currently i’m searching for some easy way to setup some kind of content  
cache.  
nginx is one of my favorites for this, because it seems to be the most  
powerfull  
and performant proxys out there.

The idea:

cache01 located in location abc  
…  
cache10 located in location xyz

main01 located in base DC  
…  
main04 located in base DC

The cacheXX servers should fetch the static content (pictures, videos)  
from one  
of the main servers, store it locally and deliver it to the users. This  
seems to  
be not a big deal so far. But we use mod\_secdownload from lighttpd to  
protect the  
content from beeing hotlinked or “guessed”. If we now to an 1:1 caching,  
this  
would work fine so far. But it also would require the cache to fetch one  
object  
multiple times since the url will change. And this is something i would  
like to  
avoid.

So my idea was to setup some “internal” url on the mainserver without  
mod\_secdownload (access limited to the cache servers!), to have always  
the same  
URL for one object. But then the cache has to do the protection of the  
content  
instead of the main servers.

nginx has no feature like mod\_secdownload so far, right? Is there  
anything i  
could do to get nginx working as a proxy with something like  
mod\_secdownload?  
maybe some perl module or so? the important thing would just be to have  
every  
picture/video only downloaded once, or at least once in xx days…

Thanks and regards,  
Sven

---

<div class="post-metadata">

**Author:** ![Sven\_SDarkmanS\_Mich](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/e99b99/32.png) [@Sven\_SDarkmanS\_Mich](https://www.ruby-forum.com/u/Sven_SDarkmanS_Mich)\
**Post date:** [October 19, 2010, 5:11pm UTC](https://www.ruby-forum.com/t/nginx-as-reverse-proxy-with-mod-secdownload-feature-possible/197731/2 "2010-10-19T17:11:45Z")

</div>

> nginx has no feature like mod\_secdownload so far, right? Is there anything  
> i  
> could do to get nginx working as a proxy with something like  
> mod\_secdownload?  
> maybe some perl module or so? the important thing would just be to have  
> every  
> picture/video only downloaded once, or at least once in xx days…

I havent followed what is the current status with the TTL (time to live)  
secure downloads ( there is some blog post  
[Nginx secure link module with TTL - Masterzen’s Blog](http://www.masterzen.fr/2009/07/18/nginx-secure-link-module-with-ttl/) )  
but  
we use [GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com · GitHub](http://wiki.nginx.org/HttpSecureLinkModule)

The secure link configuration is put on the caching servers (while of  
course  
you can duplicate it also on the backend in case you want to support  
also  
direct requests) which then fetch the object (if it doesnt exist in  
cache  
(local tree)) from backend via normal url (and do proxy\_store locally) -  
the  
url normalising (striping out the hash parts) allows to store the same  
object just once).

But to implement the download-once feature you could use php (or any  
other  
dynamic backend) and do it via X-Accel-Redirect (  
[GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com · GitHub](http://wiki.nginx.org/XSendfile) ) - like transparently parse all the  
file  
downloading requests (through either try\_files or rewrite) and let php  
decide if the file is available or not and nginx just does the transfer.

rr

---

<div class="post-metadata">

**Author:** ![Sven\_SDarkmanS\_Mich](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/e99b99/32.png) [@Sven\_SDarkmanS\_Mich](https://www.ruby-forum.com/u/Sven_SDarkmanS_Mich)\
**Post date:** [October 21, 2010, 11:42am UTC](https://www.ruby-forum.com/t/nginx-as-reverse-proxy-with-mod-secdownload-feature-possible/197731/3 "2010-10-21T11:42:47Z")

</div>

Hi Reinis,

thanks for your reply.

Am 19.10.2010 17:10, schrieb Reinis R.:

> I havent followed what is the current status with the TTL (time to live)  
> secure downloads ( there is some blog post  
> [Nginx secure link module with TTL - Masterzen’s Blog](http://www.masterzen.fr/2009/07/18/nginx-secure-link-module-with-ttl/) )  
> but we use [GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com · GitHub](http://wiki.nginx.org/HttpSecureLinkModule)

we definitivly need the expiring urls, so the ttl module points to the  
right direction.

> The secure link configuration is put on the caching servers (while of  
> course you can duplicate it also on the backend in case you want to  
> support also direct requests) which then fetch the object (if it doesnt  
> exist in cache (local tree)) from backend via normal url (and do  
> proxy\_store locally) - the url normalising (striping out the hash parts)  
> allows to store the same object just once).

We currently use lighttpd on the servers, which work ok so far. We just  
want to add caches we can put infront of the lighty setup to serve the  
content from different places.

> But to implement the download-once feature you could use php (or any  
> other dynamic backend) and do it via X-Accel-Redirect (  
> [GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com · GitHub](http://wiki.nginx.org/XSendfile) ) - like transparently parse all the  
> file downloading requests (through either try\_files or rewrite) and let  
> php decide if the file is available or not and nginx just does the  
> transfer.

This would not help (at least i think so) because we do not want to sync  
any content to the caches. We just want them to fetch the stuff from the  
main servers if they haven’t stored it locally in their proxy cache, and  
deliver it from the cache if they already have it.

So from all my readings, i think(! and might be wrong), the easiest and  
maybe best way for now would be having a small perl module doing the  
secdownload stuff (just a few lines of code, so no big deal) and rewrite  
the request to a normalized url which can be found on the backend.

So, is such a setup poosible? like → request to nginx → perl module  
does  
some check and rewrite of the request → request passed to proxy module  
 → fetches content from local cache if avail or from backend server.

If yes, some pointers/samples would be nice.

Thanks and regards,  
Sven

---

<div class="post-metadata">

**Author:** ![Sven\_SDarkmanS\_Mich](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/e99b99/32.png) [@Sven\_SDarkmanS\_Mich](https://www.ruby-forum.com/u/Sven_SDarkmanS_Mich)\
**Post date:** [October 22, 2010, 4:12am UTC](https://www.ruby-forum.com/t/nginx-as-reverse-proxy-with-mod-secdownload-feature-possible/197731/4 "2010-10-22T04:12:01Z")

</div>

On Thu, Oct 21, 2010 at 7:32 PM, Reinis R. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> 

And don’t miss our ngx\_srcache and ngx\_lua modules! 😉

```
http://github.com/agentzh/srcache-nginx-module
http://github.com/chaoslawful/lua-nginx-module

```

Cheers,  
-agentzh

---

<div class="post-metadata">

**Author:** ![Sven\_SDarkmanS\_Mich](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/e99b99/32.png) [@Sven\_SDarkmanS\_Mich](https://www.ruby-forum.com/u/Sven_SDarkmanS_Mich)\
**Post date:** [October 21, 2010, 1:33pm UTC](https://www.ruby-forum.com/t/nginx-as-reverse-proxy-with-mod-secdownload-feature-possible/197731/5 "2010-10-21T13:33:47Z")

</div>

> This would not help (at least i think so) because we do not want to sync  
> any content to the caches. We just want them to fetch the stuff from the  
> main servers if they haven’t stored it locally in their proxy cache, and  
> deliver it from the cache if they already have it.

You don’t need to “store” directly (like push any files beforehand) to  
the  
cache servers - nginx can store the files on demand in the same tree  
structure as on backend (quite easy to examine that way what is getting  
fetched and purge the cache with simple filesystem tools like ‘find /  
rm’)  
by using the “proxy\_store on” (  
[GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com · GitHub](http://wiki.nginx.org/NginxHttpProxyModule#proxy_store) ) directive or  
either  
in its own cache tree but then you need to adjust proxy\_cache\_key so  
that it  
doesnt include the default $request\_uri (which would contain the dynamic  
hash that way storing a single file multiple times (someone correct me  
if  
I’m wronge here)) but just the real path (do something like $secure\_link  
rewrite) and the file technically should be fetched from cache each time  
rather than backend (the advantage of this is you can have a dynamic  
garbage  
collector (cache cleaner) by adjusting the overal size and time to live  
rather than have to do it yourself) ..

To give some example - some pseido config for the first approach:

```
upstream backend {
    server backendip:8080;
}

 server {
    root /webroot;
    error_page 404 = @store;

    location /dlpath/ {
    secure_link_secret randomkey;
            if ($secure_link = "") {
                    return 403;
            }
            rewrite ^ /dlpath/$secure_link break;
    }

    location @store {
            internal;
            proxy_pass http://backend;
            proxy_store on;
    }
}

```

> So from all my readings, i think(! and might be wrong), the easiest and  
> maybe best way for now would be having a small perl module doing the  
> secdownload stuff (just a few lines of code, so no big deal) and rewrite  
> the request to a normalized url which can be found on the backend.

Since I am not aware of any third party modules which can keep track of  
the  
download status thats one of the solutions (imo the easy way).

On the other hand if you plan to _exploit_ all of nginx possibilities /  
features you could use the memcache + echo module

> **[GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki...](https://github.com/nginxinc/nginx-wiki)**
>
> ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com

> **[GitHub - openresty/echo-nginx-module: An Nginx module for bringing the power of "echo",...](https://github.com/openresty/echo-nginx-module)**
>
> An Nginx module for bringing the power of "echo", "sleep", "time" and more to Nginx's config file

In a way the Memcached would hold an unique key (inserted by third party  
app  
or some nginx subrequest) which consists of the file path or true/false  
and  
after making a request the key would be deleted - thats just theory  
though  
and requires some voodoo 🙂

rr