Migrating from Lighttpd : mod_secdownload show-stopper?

Hi, Privet :slight_smile:

I would like to switch all my media servers from Lighttpd 1.5.0 to
latest Nginx version but I have a strong requirement : mod_secdownload.
I saw that a similar module exists for Nginx but it’s not working
exactly the same way as Lighty’s one and “httpsecurelink” uses a too
different mechanism. Moreover, the “securedownload” seems to have been
not updated for a long time (since 3rd of august last year).

Do you have any advices/ideas to help me mimick Lighty’s
“mod_secdownload”, please ? Actually, my application produce links like
this one :

/$prefix/$path/to/file.mp4/$hash/$timestamp

Any help will be greatly appreciated as this is the only point I need to
solve before proceeding to migration.

Nginx : 0.8.54
Secure Download : downloaded from github
OS : Centos 5.5 x86_64

Sincerly,

Thomas S

Posted at Nginx Forum:

Well, to low down priority of issue, you may live first with url rewrite
function, so nginx will return your file, it will not obey
hash and timestamp.

Anyway, you claim that http://wiki.nginx.org/NginxHttpSecureDownload not
working for you?

On 2/10/2011 11:22 AM, Vitaly T. wrote:

Well, to low down priority of issue, you may live first with url
rewrite function, so nginx will return your file, it will not obey
hash and timestamp.

Anyway, you claim that http://wiki.nginx.org/NginxHttpSecureDownload
not working for you?

We have found that the secure download module only works reliably with
0.7.67. Mauro Settler is busy traveling for work and is not able to
look into why it doesn’t work “reliably” for another few weeks. He
thinks that perhaps the content-length is not being set under certain
circumstances.

–


Robert La Ferla
VP Engineering
OMS SafeHarbor

This message (and any attachments) contains confidential information and
is protected by law. If you are not the intended recipient, you should
delete this message and are hereby notified that any disclosure,
copying, distribution, or the taking of any action based on this
message, is strictly prohibited.

So we just need to wait, aren’t we?

And of course, I made a mistake with the example link provided… I
should have written :

$prefix/$hash/$timestamp/$path/to/file.mp4

Sorry for the mistake.
Sincerly,

Thomas S

Posted at Nginx Forum:

Well, we can’t force him to do something, right?

Or you can just to make your nginx work correctly without checking
timestamp and hash, but not braking the links - if you in rush.

Thanks for answers.

i confirm that I don’t succeed in the configuration of the module. The
best result I had with it was “error 500” refering to a misconfiguration
(as the doc says).

Thus the only option we got at this moment is to wait for Mauro Settler
?
Regards,

Thomas S

Posted at Nginx Forum:

On 2/10/2011 11:53 AM, Vitaly T. wrote:

Well, we can’t force him to do something, right?

Or you can just to make your nginx work correctly without checking
timestamp and hash, but not braking the links - if you in rush.

Mauro is very response and will fix these issues. He is just busy. His
module is a very useful one and I think he would want it to work
reliably on all new versions of nginx.

–


Robert La Ferla
VP Engineering
OMS SafeHarbor

This message (and any attachments) contains confidential information and
is protected by law. If you are not the intended recipient, you should
delete this message and are hereby notified that any disclosure,
copying, distribution, or the taking of any action based on this
message, is strictly prohibited.

Do you have any advices/ideas to help me mimick Lighty’s
“mod_secdownload”, please ? Actually, my application produce links like
this one :
/$prefix/$path/to/file.mp4/$hash/$timestamp

One way to work arround this would be to use X-Accel-Redirect (
XSendfile | NGINX) and rewrite (or with try_file) requests
to
some script (like php, perl or ruby) and make the decision on
application
level (would require just few lines of code to verify the hash/timestamp
but
you could even check against DB etc) while still pushing the file
directly
through nginx.

Few articles:
http://kovyrin.net/2006/11/01/nginx-x-accel-redirect-php-rails/
http://kovyrin.net/2010/07/24/nginx-fu-x-accel-redirect-remote/

rr

On Thu, Feb 10, 2011 at 11:25:23AM -0500, thoseg wrote:

And of course, I made a mistake with the example link provided… I
should have written :

$prefix/$hash/$timestamp/$path/to/file.mp4

In 0.8.51+ you can try new ngx_http_secure_link_module functionality:

/dir/MD5_HASH/TIMESTAMP/path/to/file.mp4

    location /dir/ {
        location ~ ^/dir/(?<HASH>[\w\-=]+)/(?<TIME>\d+)(?<FILE>/.+)$ 

{

            secure_link       $HASH,$TIME;
            secure_link_md5   $FILE.PaSsWoRd;

            if ($secure_link = "") {    # no valid
                return 403;
            }

            if ($secure_link == 0) {   # expired
                return 410;
            }

            alias  /path/to$file;
        }

        return 404;

        error_page  403       /forbidden.html;
        error_page  404       /not_found.html;
        error_page  410  =403 /link_expired.html;
    }

–
Igor S.
http://sysoev.ru/en/

On Feb 10, 2011, at 23:06 , Igor S. wrote:

   location /dir/ {
       location ~ ^/dir/(?<HASH>[\w\-=]+)/(?<TIME>\d+)(?<FILE>/.+)$ {

           secure_link       $HASH,$TIME;
           secure_link_md5   $FILE.PaSsWoRd;

secure_link_md5 $FILE.$TIME.PaSsWoRd;

MD5_HASH = md5(“/path/to/file.mp4.TIMESTAMP.PaSsWoRd”)

       return 404;

       error_page  403       /forbidden.html;
       error_page  404       /not_found.html;
       error_page  410  =403 /link_expired.html;
   }

–
Igor S.
http://sysoev.ru/en/

I believe he is, I just meant that we can’t force him to change
priorities right now. It’s open source, and this contributing is only
good will.

On 10 Fev 2011 16h25 WET, [email protected] wrote:

Like Igor suggested use the Module ngx_http_secure_link_module

The example in the wiki is tested.

— appa

On Thu, Feb 10, 2011 at 6:22 PM, Reinis R. [email protected] wrote:

through nginx.

Few articles:
http://kovyrin.net/2006/11/01/nginx-x-accel-redirect-php-rails/
http://kovyrin.net/2010/07/24/nginx-fu-x-accel-redirect-remote/

FYI, this was the route I choose when I switched from Lighttpd to
nginx. I also used mod_secdownload a lot.

Recommended.

Bye,

Pedro Melo
http://www.simplicidade.org/
xmpp:[email protected]
mailto:[email protected]

Thanks for all your replies. Spasibo :slight_smile:

I have some (strong) requirements like to not modify the application
code or to do as few changes in the platform’s
configuration/architecture as possible. Thus I’m using Igor’s example
that seemed to me the best choice for migration from Lighttpd to Nginx.
I’m trying to make the following configuration works :

Secured link :
http://secure.domain.com/get/24b9cb61c9c2c9070038aceaaf7bae5a/4d63842e/2/H264-384x288/04/85/3120485.h264


Configuration :

server {
server_name secure.domain.com;
root /var/www;

location /get/ {
    location ~ ^/get/([\w\-=]+)/(\w+)(/.+)$ {

            root /mnt/medias;

            secure_link $1,$2;
            secure_link_md5 $3.$2.$mysalt

            if ($secure_link = "") { # no valid
                    return 403;
            }

            if ($secure_link = "0") { # expired
                    return 410;
            }
    }

    return 404;

    error_page 403 /forbidden.html;
    error_page 404 /not_found.html;
    error_page 410 =403 /link_expired.html;
}

}


After trying to access the link I always got this into my error.log :

2011/02/22 10:50:32 [error] 26143#0: *1 open() “/var/www/forbidden.html”
failed (2: No such file or directory), client: 192.168.156.11, server:
secure.domain.com, request: “GET
/get/24b9cb61c9c2c9070038aceaaf7bae5a/4d63842e/2/H264-384x288/04/85/3120485.h264
HTTP/1.1”, host: “secure.domain.com”

As far as I understand I got a “403” everytime I’m trying to validate
the URL. That means $secure_link is empty and I don’t know why… Of
course, I’m not using an “epoch Unix time” in the link so I modified it
to be able to send a date in this format but I keep having the same
error into my log.

It will be (really) better if can keep my actual link format with the
expiration time given in hex not in epoch but If there is no other
solution, I will change the application’s code.

Thanks for your answers.
Regards,

Thomas S

Posted at Nginx Forum:

On Tue, Feb 22, 2011 at 05:03:00AM -0500, thoseg wrote:

Hi there,

I have some (strong) requirements like to not modify the application
code or to do as few changes in the platform’s
configuration/architecture as possible. Thus I’m using Igor’s example
that seemed to me the best choice for migration from Lighttpd to Nginx.

Secured link :

http://secure.domain.com/get/24b9cb61c9c2c9070038aceaaf7bae5a/4d63842e/2/H264-384x288/04/85/3120485.h264

That link includes a (lighttpd) 32-byte “hex-ascii” md5sum
representation.

nginx’s http_secure_link_module uses a 22-byte base64 encoded md5sum
representation.

You can’t convert the one into the other using just nginx configuration.

If you want to use http_secure_link_module, you will have to modify the
application code, or modify the http_secure_link_module code, or, maybe,
try using an embedded interpreter within the nginx config to massage
the url and issue a rewrite or redirect.

That last “maybe” option also counts as coding.

Not thought through properly, of course, but could it work if your
“/get” location calculated the nginx equivalent of the 32-byte string,
and then rewrote or redirected to another location, which was handled
normally by http_secure_link_module?

You’ld also have to worry about the “time” string – you could
recalculate
it in the same way, but since it is involved in the md5sum calculation,
it wouldn’t work trivially. So that looks like a “no”.

As far as I understand I got a “403” everytime I’m trying to validate
the URL. That means $secure_link is empty and I don’t know why… Of

It’s because the (representation of the) hash is incorrect. 32 vs
22 bytes.

It will be (really) better if can keep my actual link format with the
expiration time given in hex not in epoch but If there is no other
solution, I will change the application’s code.

To use the vanilla nginx http_secure_link_module, your old links will
be broken, and you’ll have to change the application’s code.

Unless there’s something major I’ve missed.

Good luck with it,

f

Francis D. [email protected]

On Tue, Feb 22, 2011 at 12:20:20PM +0000, Francis D. wrote:

On Tue, Feb 22, 2011 at 05:03:00AM -0500, thoseg wrote:

Hi there,

I have some (strong) requirements like to not modify the application
code

Secured link :

http://secure.domain.com/get/24b9cb61c9c2c9070038aceaaf7bae5a/4d63842e/2/H264-384x288/04/85/3120485.h264

To use the vanilla nginx http_secure_link_module, your old links will
be broken, and you’ll have to change the application’s code.

As was suggested earlier in the thread, you could preserve the
lighttpd
interface by scripting, while waiting for / working on a mod_secdownload
equivalent in nginx.

One (rushed) version in php is included below. Do read it carefully and
check for unexpected results (do you know what happens when the string
given to hexdec() is rather long?) before testing whether the system
performance is adequate in your proposed new environment.

The nginx.conf section is something like

===
location /get/ {
location ~ /get/[0-9a-f]+/[0-9a-f]+/ {
fastcgi_pass unix:php.sock;
include fastcgi.conf;
fastcgi_param SCRIPT_FILENAME /mnt/medias/secure_get;
}
location /get/secured/ {
internal;
alias /mnt/medias/;
}
return 404;
}

(spot the bits you need to change)

and the secure_get script is similar to

===

<?php # migrate from lighttpd/mod_secdownload # I care about parts 2, 3, and remains of REQUEST_URI. # $_SERVER['REQUEST_URI'] = '/get/md5sum_hex/expires_hex/dir/and/file/name.ext # This is the secret used to generate the link in the first place $secret = "secret"; list( $x, $x, $hash, $time, $file ) = explode( '/', $_SERVER['REQUEST_URI'], 5 ); $file = '/' . $file; $check = md5("${file}.${time}.${secret}"); $expires = hexdec($time); if ($check != $hash) { # hash is wrong header("HTTP/1.0 403 Forbidden"); # add your preferred html body here } elseif ($expires < $_SERVER['REQUEST_TIME']) { # hash is right, but time is past header("HTTP/1.0 410 Gone"); # add your preferred html body here } else { # hash is right and time is ok header("X-Accel-Redirect: /get/secured${file}"); } ?>

===

Good luck with it,

f

Francis D. [email protected]

Hi,
I want to migrate from Lighty too. Works so far but i need to use that
“secdownload” functionality of Lighty.
I tried ntr0pies solution, but can’t get it to work because of that
time-value … i think. I get a 404 everytime.

My Script pulls the url like this:

public static function secdownload_url($video_id, $ext, $key)
  {
    $file  = ($ext == 'mobile') ? '/mobile/'.$video_id.'.mp4' :
'/'.$ext.'/'.$video_id.'.'.$ext;
                $hex    = sprintf("%08x", time());

        return md5($key.$file.$hex).'/'.$hex.$file;
  }

Maybe someone can help me with that. Thank you very much.

Posted at Nginx Forum:

Hello there,

since the only option is to modify the C source code of
nginx/src/http/modules/ngx_http_secure_link_module.c to support
hexadecimal md5 encoding in addition to base64 encoding besides
specifying an optional ttl value to add to expiration time (e.g. 3600
sec in example below) I made a patch to provide this functions in
addition to the documented behavior.
It can be used with a variant of the the configuration Igor suggested:

    location /get/ {
            #location ~

^/get/(?[\w-=]+)/(?\d+)(?/.+)$ { #
/get/<md5_base64>/<time_dec>
location ~
^/get/(?[[:xdigit:]]+)/(?\d+)(?/.+)$ { #
/get/<md5_hex>/<time_dec>
set $PASS sn983pjcnhupclavsnda; # secret string

                    secure_link $HASH,$TIME+3600;
                    secure_link_md5 $PASS$PATH$TIME;

                    if ($secure_link = "") { # invalid
                            return 403;
                    }

                    if ($secure_link = 0) { # expired
                            return 410;
                    }

                    if ($secure_link = 1) { # correct
                            rewrite ^ /media$PATH last;
                    }
            }

            return 404;

            error_page 403 /forbidden.html;
            error_page 404 /not_found.html;
            error_page 410 =403 /link_expired.html;
    }

I would recommend to specify /media/ as internal location to make it
accessible only for internal requests.
It should be noted that the time value here is in decimal encoding in
opposition to lighttpd’s mod_secure_dowload.c hexadecimal encoding, with
the advantege that it can easily be created via
nginx/src/http/modules/ngx_http_ssi_filter_module.c:

With this example it should be possible to support software like
flowplayer (maybe with additional mod_h264_streaming) without any
serverside scripting requirements.

Since i suspect the diff to get corrupted by posting, here is the
original: http://pastebin.com/Ln7bS48n

Maybe someone with more experience in C programming / nginx source can
look over the following patch to
nginx/src/http/modules/ngx_http_secure_link_module.c:

— nginx.orig/src/http/modules/ngx_http_secure_link_module.c
2010-09-13
14:44:43.000000000 +0200
+++ nginx/src/http/modules/ngx_http_secure_link_module.c 2011-04-10
17:12:55.000000000 +0200
@@ -100,13 +100,15 @@
ngx_http_secure_link_variable(ngx_http_request_t *r,
ngx_http_variable_value_t *v, uintptr_t data)
{

  • u_char *p, *last;
  • u_char *p, *q, *last;
    ngx_str_t val, hash;
  • time_t expires;
  • time_t expires, ttl;
    ngx_md5_t md5;
    ngx_http_secure_link_ctx_t *ctx;
    ngx_http_secure_link_conf_t *conf;
    u_char hash_buf[16], md5_buf[16];

  • ngx_int_t n;

  • ngx_uint_t i;

    conf = ngx_http_get_module_loc_conf(r,
    ngx_http_secure_link_module);

@@ -128,16 +130,33 @@
last = val.data + val.len;

 p = ngx_strlchr(val.data, last, ',');
  • expires = 0;
  • ttl = 0;
  • if (p) {
  •    val.len = p++ - val.data;
    
  • if (p) { // expires
  •    val.len = p - val.data;
    
  •    expires = ngx_atotm(p, last - p);
    
  •    if (expires <= 0) {
    
  •    if (last - ++p < 0) {
           goto not_found;
       }
    
  •    q = ngx_strlchr(p, last, '+');
    
  •    if (q) { // ttl
    
  •        if (last - ++q < 0 || (ttl = ngx_atotm(q, last - q)) < 0 ||
    

–q - p <= 0) {

  •            goto not_found;
    
  •        }
    
  •    } else {
    
  •            q = last;
    
  •    }
    
  •    if ((expires = ngx_atotm(p, q - p)) <= 0) {
    
  •        goto not_found;
    
  •    }
    
  •    expires += ttl;
    
  •    ctx = ngx_pcalloc(r->pool,
    

sizeof(ngx_http_secure_link_ctx_t));
if (ctx == NULL) {
return NGX_ERROR;
@@ -145,22 +164,26 @@

     ngx_http_set_ctx(r, ctx, ngx_http_secure_link_module);
  •    ctx->expires.len = last - p;
    
  •    ctx->expires.len = q - p;
       ctx->expires.data = p;
    
    }
  • if (val.len > 24) {

  •    goto not_found;
    
  • }

  • hash.len = 16;
    hash.data = hash_buf;

  • if (ngx_decode_base64url(&hash, &val) != NGX_OK) {

  •    goto not_found;
    
  • }

  • if (hash.len != 16) {

  • if (val.len == 32) { // hexadecimal md5
  •   for (i = 0; i < 16; i++) {
    
  •        n = ngx_hextoi(&val.data[2 * i], 2);
    
  •        if (n == NGX_ERROR) {
    
  •            goto not_found;
    
  •        }
    
  •        hash.data[i] = n;
    
  •    }
    
  • } else if (val.len <= 24) { // base64 md5
  •    if (ngx_decode_base64url(&hash, &val) != NGX_OK || hash.len !=
    
  1. {
  •        goto not_found;
    
  •    }
    
  • } else {
    goto not_found;
    }

Posted at Nginx Forum:

On Thu, Jun 09, 2011 at 11:54:46AM -0400, Epstein wrote:

Hi there,

I want to migrate from Lighty too. Works so far but i need to use that
“secdownload” functionality of Lighty.

To be utterly clear: do you need to use secdownload from Lighty; or
do you need to use some expiring download feature, such as Lighty’s
secdownload or nginx’s secure_link_module?

Because if “something like” is good enough, then you can aim to use
nginx’s one in future, and therefore only need a short-term workaround
for currently-advertised urls. When the last of the old urls expires,
you can disable the workaround.

In the meantime, you create new urls in the nginx style, and they are
handled natively.

I tried ntr0pies solution, but can’t get it to work because of that
time-value … i think. I get a 404 everytime.

Look one reply above that in the forum, from February 23.

The line

$check = md5("${file}.${time}.${secret}");

is utterly wrong – it should be something more like

$check = md5($secret.$file.$time);

or

$check = md5("${secret}${file}${time}");

but the rest should probably Just Work; or at least it should be obvious
to find out where it is failing.

Good luck,

f

Francis D. [email protected]