Forum: NGINX flv streaming and proxy_pass

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
B2f5aac76ec8ac6d834848422361b5ee?d=identicon&s=25 Karolis (Guest)
on 2009-02-18 18:51
(Received via mailing list)
hi list,

I am using nginx / apache combination to stream flv files.
Nginx serves static files and forwards dynamic requests to
apache/mod_php.

Request flow would be as follows:

 - client requests some file
 - nginx rewrites the filename to a local cache dir
 - if file is found in the cache dir, nginx sends the file
 - if file is not found, then request is passed to apache, which will
create the file and send the contents.

For flv files we want flv streaming.

The rules I am using to make this work are as follows:

    # Map request url to cache dir
    location /dbfile {
            rewrite  "^/dbfile/([0-9]{3})([0-9]{3})([0-9]{3})/(.*)$"
/cache/$1/$2/$1$2$3__$4  last;
    }

    # Forward PHP requests to apache, in case we don't have cache yet
    location ~ ^/cache/ {
        if (!-f $request_filename) {
            proxy_pass  http://127.0.0.1:8080;
        break;
        }
    }

    # Handle FLV streaming
    location ~ ^/cache/(.*)\.flv$ { flv; }

In this configuration, proxy_pass works, but flv streaming doesn't.
If I move FLV "location" directive above proxy_pass, then flv will work,
and proxy_pass won't.

How do I get them both to work together? I don't want streaming of
proxied requests, I only want to
flv-stream the files that are on local filesystem.

Any help?

thanks
Karolis
B2f5aac76ec8ac6d834848422361b5ee?d=identicon&s=25 Karolis D (Guest)
on 2009-02-19 00:32
(Received via mailing list)
On Wed, Feb 18, 2009 at 7:39 PM, Karolis <karolis@pg.lt> wrote:

> - if file is not found, then request is passed to apache, which will create
>   }
>   location ~ ^/cache/(.*)\.flv$ { flv; }
>
> thanks
> Karolis
>
>
I've also tried to work around the problem by rewriting flv file to php
script, and then proxy_pass'ing the php to apache, but the results are
no
better:

    # Map request url to cache dir
    location ~ ^/dbfile/ {
            rewrite  "^/dbfile/([0-9]{3})([0-9]{3})([0-9]{3})/(.*)$"
/cache/$1/$2/$1$2$3__$4  last;
    }

        # If required cache file does not exist...
    location ~ ^/cache/ {
        if (!-f $request_filename) {
            rewrite "/cache/([0-9]{3})/([0-9]{3})/([0-9]{9})__(.*)$"
/file.php?id=$3 last;
        }
    }

        location ~ ^/cache/(.*)\.flv$ { flv; }

    # Forward PHP requests to Apache
    location ~ ^/file.php {
        proxy_pass  http://127.0.0.1:8080; # pass on request to apache
    }

If I comment out the proxy_pass section, then the FLV streaming starts
to
work fine. So it kind of means nginx doesn't like the proxy part of the
setup.

I do understand that you can't stream a file that is retrieved from a
proxy
server, but in this setup  I am trying to do only one of those things:
Either fetch the file from proxy if it doesn't exist locally, or
flv-stream
it if it does exist.

Problem is that lighttpd does it without a problem with mod_magnet :)

Karolis
E84e93646376def3fbed0907fe9335e1?d=identicon&s=25 Max (Guest)
on 2009-02-19 01:13
(Received via mailing list)
Hi,

Do you use this module to stream? http://h264.code-shop.com/trac

This might not useful for you, but just for references.

I use this to stream mp4 movies (h264 video code) with lighttpd(they
have a
module for ngnix and apache), it's working great and without any
problem.

Max
B2f5aac76ec8ac6d834848422361b5ee?d=identicon&s=25 Karolis D (Guest)
on 2009-02-19 01:40
(Received via mailing list)
On Thu, Feb 19, 2009 at 2:01 AM, Max <maxbear@gmail.com> wrote:

>
I am using standard flv streaming module that comes with nginx.

Server is compiled with:
 --with-http_flv_module

nginx version: nginx/0.7.35 (same results with latest stable version)

Karolis
B2f5aac76ec8ac6d834848422361b5ee?d=identicon&s=25 Karolis D (Guest)
on 2009-02-19 22:26
(Received via mailing list)
Anyone?

I kind of expected that this would be a known configuration...

I've read all docs I could find regarding this subject (not that there
are a
lot of flv streaming), and still can't get it to work. Really wouldn't
like
to move back to lighttpd because of such a trivial problem.

Karolis
714df4094c6cc4a71b835b5ed3668f6b?d=identicon&s=25 Todd Fisher (taf2)
on 2009-02-19 23:03
(Received via mailing list)
Just a thought, but have you tried:
  location ~ ^/cache/ {
      if (!-f $request_filename) {
          proxy_pass  http://127.0.0.1:8080;
      break;
      }
      flv;
  }
B2f5aac76ec8ac6d834848422361b5ee?d=identicon&s=25 Karolis (Guest)
on 2009-02-19 23:22
(Received via mailing list)
Todd Fisher wrote:
> Just a thought, but have you tried:
>
>   location ~ ^/cache/ {
>       if (!-f $request_filename) {
>           proxy_pass  http://127.0.0.1:8080;
>       break;
>       }
>       flv;
>   }
That's not really correct configuration, as I want only FLV files in
/cache/ location to be treated as flv, not all of them.
I have tried nesting another location inside the /cache/ location, but
that didn't work any better.

Karolis
B2f5aac76ec8ac6d834848422361b5ee?d=identicon&s=25 Karolis D (Guest)
on 2009-02-23 18:56
(Received via mailing list)
I would appreciate if someone confirmed that this feature is not
supported
so I could move back to lighttpd instead of trying to make nginx do what
it
isn't capable of.

thanks.
4e1ae4b836a9cfe3945d8c661b37246b?d=identicon&s=25 Manlio Perillo (Guest)
on 2009-02-23 19:48
(Received via mailing list)
Karolis D ha scritto:
> I would appreciate if someone confirmed that this feature is not
> supported so I could move back to lighttpd instead of trying to make
> nginx do what it isn't capable of.
>

Do you have checked proxy_store directive?
http://wiki.codemongers.com/NginxHttpProxyModule#proxy_store

> thanks.


Manlio Perillo
2e321cc0efe9422d37165e922298494e?d=identicon&s=25 Cliff Wells (Guest)
on 2009-02-23 19:50
(Received via mailing list)
On Wed, 2009-02-18 at 19:39 +0200, Karolis wrote:
>  - if file is not found, then request is passed to apache, which will
>     }
>     location ~ ^/cache/(.*)\.flv$ { flv; }
You have two locations that match.  It appears that Nginx takes the
first one.   I'd try something like:

location /cache {
    location ~ \.flv$ { flv; }
    error_page 404 = @apache;
}

location @apache {
    proxy_pass http://127.0.0.1:8080;
}


Untested and don't really have time to experiment for you, so YMMV.

Regards,
Cliff
B2f5aac76ec8ac6d834848422361b5ee?d=identicon&s=25 Karolis (Guest)
on 2009-02-23 20:54
(Received via mailing list)
> location /cache {
>     location ~ \.flv$ { flv; }
>     error_page 404 = @apache;
> }
>
> location @apache {
>     proxy_pass http://127.0.0.1:8080;
> }
>
>
This one seems to have worked like charm!
does the "@" have any special meaning in the config?

thanks!!!

Karolis
2e321cc0efe9422d37165e922298494e?d=identicon&s=25 Cliff Wells (Guest)
on 2009-02-23 23:33
(Received via mailing list)
On Mon, 2009-02-23 at 21:39 +0200, Karolis wrote:
> This one seems to have worked like charm!
> does the "@" have any special meaning in the config?

It's an "internal" location, that is, one that is not directly available
via URL.   I'm not sure if it's handled more efficiently than a normal
location, Igor would have to answer that.

> thanks!!!

Glad to help.  Can't have you going back to Lighty ;-)

Regards,
Cliff
5640e332954fc0006aea97a155ce0afd?d=identicon&s=25 Igor Sysoev (Guest)
on 2009-02-24 11:23
(Received via mailing list)
On Wed, Feb 18, 2009 at 07:39:41PM +0200, Karolis wrote:

> - if file is not found, then request is passed to apache, which will
>    }
>    location ~ ^/cache/(.*)\.flv$ { flv; }
>
> In this configuration, proxy_pass works, but flv streaming doesn't.
> If I move FLV "location" directive above proxy_pass, then flv will work,
> and proxy_pass won't.
>
> How do I get them both to work together? I don't want streaming of
> proxied requests, I only want to
> flv-stream the files that are on local filesystem.

    location /dbfile {
        rewrite  "^/dbfile/([0-9]{3})([0-9]{3})([0-9]{3})/(.*)$"
                 /cache/$1/$2/$1$2$3__$4  last;
    }

    location /cache/ {
        flv;
        error_page  404  = @apache;
    }

    location @apache {
        proxy_pass  http://127.0.0.1:8080;
    }

However, Apache should pass flv headers by itself.
5cec4fdb1193157e1f900cdfa6c973db?d=identicon&s=25 Kamil Gorlo (Guest)
on 2009-02-24 12:43
(Received via mailing list)
Wiadomo¶æ napisana w dniu 2009-02-24, o godz. 11:02, przez Igor Sysoev:

>> - client requests some file
>>   location /dbfile {
>>   }
>> proxied requests, I only want to
>    }
>
>    location @apache {
>        proxy_pass  http://127.0.0.1:8080;
>    }
>
> However, Apache should pass flv headers by itself.

Igor - is this possible to store flv file (not only from certain
seeked point)  from "Apache" in cache (maybe using proxy_store) and
also have flv seekeing working both with cached and proxied files?

Kamil
D222f31797275f0ff3632fed8f750589?d=identicon&s=25 Michal Frackowiak (michalf)
on 2009-02-24 20:45
Honestly I think making mp4 of flv modules capable of streaming videos
from a proxy source would be great. One way around we are just trying
(we are streaming from S3) is to mount the S3 bucket through a FUSE
filesystem and make mp4 stream it from there. There are (unfortunately)
a few other problems with that, e.g. s3fs cannot fetch parts of the file
(seek), it downloads the whole file, but we are on our way for a custom,
read-only filesystem that does the trick.

Caching files locally is also a good option, but when you have thousands
of large movies it makes it quite difficult. Direct streaming would be
much nicer.

@Karolis: how exactly can you do this with lighttpd? Any reference?

Michal
B2f5aac76ec8ac6d834848422361b5ee?d=identicon&s=25 Karolis (Guest)
on 2009-02-24 21:29
(Received via mailing list)
> @Karolis: how exactly can you do this with lighttpd? Any reference?
>
> Michal
>
Just to make it clear: it works now on nginx too, using code from  Cliff
Wells;

Regarding lighty, you can do the same (similar) with mod_magnet. The
difference
is that with lighty I used to pass the request to php-fcgi instead of
apache. But I actually
like using apache better, as our cache miss rate is very low and I find
it much easier
to maintain apache/mod_php installation than php-fcgi.

anyway, in lighttpd.conf:
  magnet.attract-physical-path-to = (server.document-root +
"/content.lua")

Then content.lua would be similar to this:

--------------------------------------------------------------------
-- Get full request URL
full_request = lighty.env["uri.path"]

-- Check if the URL has required syntax
id_request = string.match(full_request, "^/dbfile/%d+/")

-- Create 9-char zero-padded file ID
file_id = string.match(id_request, "%d+")
pad_id = string.format("%09d", file_id)

part1 = string.sub(pad_id, 1, 3)
part2 = string.sub(pad_id, 4, 6)
part3 = string.sub(pad_id, 7, 9)


file_path = "cache/" .. part1 .. "/" .. part2 .. "/" .. file_id .. "__"
.. filename
stat = lighty.stat(lighty.env["physical.doc-root"] .. file_path)

if stat and stat["is_file"] then
  -- Rewrite to show image/movie
  lighty.env["uri.path"] = file_path
  lighty.env["physical.rel-path"] = lighty.env["uri.path"]
  lighty.env["physical.path"] = lighty.env["physical.doc-root"] ..
lighty.env["physical.rel-path"]

  lighty.status["cache.hit"] = lighty.status["cache.hit"] + 1
else
  -- Rewrite to generator script
  lighty.env["uri.path"] = "/file.php"
  lighty.env["uri.query"] = "id=" .. pad_id
  lighty.env["physical.rel-path"] = lighty.env["uri.path"]
  lighty.env["physical.path"] = lighty.env["physical.doc-root"] ..
lighty.env["physical.rel-path"]

  lighty.status["cache.miss"] = lighty.status["cache.miss"] + 1
end
--------------------------------------------------------------------

This is probably by far not the most efficient way to do it, but this
was my first lua script and it worked, so...

PS. I miss cache.hit and cache.miss features a lot . Does nginx have any
similar counters?

Karolis
0931e119fd0d29f5522ee40cc1f63b93?d=identicon&s=25 Tom Harris (tomfunkle)
on 2009-06-09 23:52
Michal Frackowiak wrote:
> Honestly I think making mp4 of flv modules capable of streaming videos
> from a proxy source would be great. One way around we are just trying
> (we are streaming from S3) is to mount the S3 bucket through a FUSE
> filesystem and make mp4 stream it from there. There are (unfortunately)
> a few other problems with that, e.g. s3fs cannot fetch parts of the file
> (seek), it downloads the whole file, but we are on our way for a custom,
> read-only filesystem that does the trick.
>
> Caching files locally is also a good option, but when you have thousands
> of large movies it makes it quite difficult. Direct streaming would be
> much nicer.
>
> @Karolis: how exactly can you do this with lighttpd? Any reference?
>
> Michal

Hi Michal,
How did you get on with this? I've been stumbling around with this idea
for a while but not quite cracked it yet. It seems that a block device
approach with the right filesystem could provide the answer, maybe some
sort of cluster fs. Anyway any chance of a collaboration?
This topic is locked and can not be replied to.