Flv streaming and proxy_pass

Server NGINX
1

hi list,

I am using nginx / apache combination to stream flv files.
Nginx serves static files and forwards dynamic requests to
apache/mod_php.

Request flow would be as follows:

  • client requests some file
  • nginx rewrites the filename to a local cache dir
  • if file is found in the cache dir, nginx sends the file
  • if file is not found, then request is passed to apache, which will
    create the file and send the contents.

For flv files we want flv streaming.

The rules I am using to make this work are as follows:

# Map request url to cache dir
location /dbfile {
        rewrite  "^/dbfile/([0-9]{3})([0-9]{3})([0-9]{3})/(.*)$"

/cache/$1/$2/$1$2$3__$4 last;
}

# Forward PHP requests to apache, in case we don't have cache yet
location ~ ^/cache/ {
    if (!-f $request_filename) {
        proxy_pass  http://127.0.0.1:8080;
    break;
    }
}

# Handle FLV streaming
location ~ ^/cache/(.*)\.flv$ { flv; }

In this configuration, proxy_pass works, but flv streaming doesn’t.
If I move FLV “location” directive above proxy_pass, then flv will work,
and proxy_pass won’t.

How do I get them both to work together? I don’t want streaming of
proxied requests, I only want to
flv-stream the files that are on local filesystem.

Any help?

thanks
Karolis

2

On Wed, Feb 18, 2009 at 7:39 PM, Karolis [email protected] wrote:

  • if file is not found, then request is passed to apache, which will create
    }
    location ~ ^/cache/(.*).flv$ { flv; }

thanks
Karolis

I’ve also tried to work around the problem by rewriting flv file to php
script, and then proxy_pass’ing the php to apache, but the results are
no
better:

# Map request url to cache dir
location ~ ^/dbfile/ {
        rewrite  "^/dbfile/([0-9]{3})([0-9]{3})([0-9]{3})/(.*)$"

/cache/$1/$2/$1$2$3__$4 last;
}

    # If required cache file does not exist...
location ~ ^/cache/ {
    if (!-f $request_filename) {
        rewrite "/cache/([0-9]{3})/([0-9]{3})/([0-9]{9})__(.*)$"

/file.php?id=$3 last;
}
}

    location ~ ^/cache/(.*)\.flv$ { flv; }

# Forward PHP requests to Apache
location ~ ^/file.php {
    proxy_pass  http://127.0.0.1:8080; # pass on request to apache
}

If I comment out the proxy_pass section, then the FLV streaming starts
to
work fine. So it kind of means nginx doesn’t like the proxy part of the
setup.

I do understand that you can’t stream a file that is retrieved from a
proxy
server, but in this setup I am trying to do only one of those things:
Either fetch the file from proxy if it doesn’t exist locally, or
flv-stream
it if it does exist.

Problem is that lighttpd does it without a problem with mod_magnet :slight_smile:

Karolis

3

Hi,

Do you use this module to stream? http://h264.code-shop.com/trac

This might not useful for you, but just for references.

I use this to stream mp4 movies (h264 video code) with lighttpd(they
have a
module for ngnix and apache), it’s working great and without any
problem.

Max

4

On Thu, Feb 19, 2009 at 2:01 AM, Max [email protected] wrote:

I am using standard flv streaming module that comes with nginx.

Server is compiled with:
–with-http_flv_module

nginx version: nginx/0.7.35 (same results with latest stable version)

Karolis

5

Anyone?

I kind of expected that this would be a known configuration…

I’ve read all docs I could find regarding this subject (not that there
are a
lot of flv streaming), and still can’t get it to work. Really wouldn’t
like
to move back to lighttpd because of such a trivial problem.

Karolis

6

Todd F. wrote:

Just a thought, but have you tried:

location ~ ^/cache/ {
if (!-f $request_filename) {
proxy_pass http://127.0.0.1:8080;
break;
}
flv;
}
That’s not really correct configuration, as I want only FLV files in
/cache/ location to be treated as flv, not all of them.
I have tried nesting another location inside the /cache/ location, but
that didn’t work any better.

Karolis

7

I would appreciate if someone confirmed that this feature is not
supported
so I could move back to lighttpd instead of trying to make nginx do what
it
isn’t capable of.

thanks.

8

Just a thought, but have you tried:
location ~ ^/cache/ {
if (!-f $request_filename) {
proxy_pass http://127.0.0.1:8080;
break;
}
flv;
}

9

Karolis D ha scritto:

I would appreciate if someone confirmed that this feature is not
supported so I could move back to lighttpd instead of trying to make
nginx do what it isn’t capable of.

Do you have checked proxy_store directive?
http://wiki.codemongers.com/NginxHttpProxyModule#proxy_store

thanks.

Manlio P.

10

On Wed, 2009-02-18 at 19:39 +0200, Karolis wrote:

  • if file is not found, then request is passed to apache, which will
    }
    location ~ ^/cache/(.*).flv$ { flv; }
    You have two locations that match. It appears that Nginx takes the
    first one. I’d try something like:

location /cache {
location ~ .flv$ { flv; }
error_page 404 = @apache;
}

location @apache {
proxy_pass http://127.0.0.1:8080;
}

Untested and don’t really have time to experiment for you, so YMMV.

Regards,
Cliff

11

On Mon, 2009-02-23 at 21:39 +0200, Karolis wrote:

This one seems to have worked like charm!
does the “@” have any special meaning in the config?

It’s an “internal” location, that is, one that is not directly available
via URL. I’m not sure if it’s handled more efficiently than a normal
location, Igor would have to answer that.

thanks!!!

Glad to help. Can’t have you going back to Lighty :wink:

Regards,
Cliff

12

On Wed, Feb 18, 2009 at 07:39:41PM +0200, Karolis wrote:

  • if file is not found, then request is passed to apache, which will
    }
    location ~ ^/cache/(.*).flv$ { flv; }

In this configuration, proxy_pass works, but flv streaming doesn’t.
If I move FLV “location” directive above proxy_pass, then flv will work,
and proxy_pass won’t.

How do I get them both to work together? I don’t want streaming of
proxied requests, I only want to
flv-stream the files that are on local filesystem.

location /dbfile {
    rewrite  "^/dbfile/([0-9]{3})([0-9]{3})([0-9]{3})/(.*)$"
             /cache/$1/$2/$1$2$3__$4  last;
}

location /cache/ {
    flv;
    error_page  404  = @apache;
}

location @apache {
    proxy_pass  http://127.0.0.1:8080;
}

However, Apache should pass flv headers by itself.

13

location /cache {
location ~ .flv$ { flv; }
error_page 404 = @apache;
}

location @apache {
proxy_pass http://127.0.0.1:8080;
}

This one seems to have worked like charm!
does the “@” have any special meaning in the config?

thanks!!!

Karolis

14

Honestly I think making mp4 of flv modules capable of streaming videos
from a proxy source would be great. One way around we are just trying
(we are streaming from S3) is to mount the S3 bucket through a FUSE
filesystem and make mp4 stream it from there. There are (unfortunately)
a few other problems with that, e.g. s3fs cannot fetch parts of the file
(seek), it downloads the whole file, but we are on our way for a custom,
read-only filesystem that does the trick.

Caching files locally is also a good option, but when you have thousands
of large movies it makes it quite difficult. Direct streaming would be
much nicer.

@Karolis: how exactly can you do this with lighttpd? Any reference?

Michal

15

Wiadomo¶æ napisana w dniu 2009-02-24, o godz. 11:02, przez Igor S.:

  • client requests some file
    location /dbfile {
    }
    proxied requests, I only want to
    }

location @apache {
proxy_pass http://127.0.0.1:8080;
}

However, Apache should pass flv headers by itself.

Igor - is this possible to store flv file (not only from certain
seeked point) from “Apache” in cache (maybe using proxy_store) and
also have flv seekeing working both with cached and proxied files?

Kamil

16

@Karolis: how exactly can you do this with lighttpd? Any reference?

Michal

Just to make it clear: it works now on nginx too, using code from Cliff
Wells;

Regarding lighty, you can do the same (similar) with mod_magnet. The
difference
is that with lighty I used to pass the request to php-fcgi instead of
apache. But I actually
like using apache better, as our cache miss rate is very low and I find
it much easier
to maintain apache/mod_php installation than php-fcgi.

anyway, in lighttpd.conf:
magnet.attract-physical-path-to = (server.document-root +
“/content.lua”)

Then content.lua would be similar to this:

– Get full request URL
full_request = lighty.env[“uri.path”]

– Check if the URL has required syntax
id_request = string.match(full_request, “^/dbfile/%d+/”)

– Create 9-char zero-padded file ID
file_id = string.match(id_request, “%d+”)
pad_id = string.format("%09d", file_id)

part1 = string.sub(pad_id, 1, 3)
part2 = string.sub(pad_id, 4, 6)
part3 = string.sub(pad_id, 7, 9)

file_path = “cache/” … part1 … “/” … part2 … “/” … file_id … “__”
… filename
stat = lighty.stat(lighty.env[“physical.doc-root”] … file_path)

if stat and stat[“is_file”] then
– Rewrite to show image/movie
lighty.env[“uri.path”] = file_path
lighty.env[“physical.rel-path”] = lighty.env[“uri.path”]
lighty.env[“physical.path”] = lighty.env[“physical.doc-root”] …
lighty.env[“physical.rel-path”]

lighty.status[“cache.hit”] = lighty.status[“cache.hit”] + 1
else
– Rewrite to generator script
lighty.env[“uri.path”] = “/file.php”
lighty.env[“uri.query”] = “id=” … pad_id
lighty.env[“physical.rel-path”] = lighty.env[“uri.path”]
lighty.env[“physical.path”] = lighty.env[“physical.doc-root”] …
lighty.env[“physical.rel-path”]

lighty.status[“cache.miss”] = lighty.status[“cache.miss”] + 1
end

This is probably by far not the most efficient way to do it, but this
was my first lua script and it worked, so…

PS. I miss cache.hit and cache.miss features a lot . Does nginx have any
similar counters?

Karolis

17

Michal Frackowiak wrote:

Honestly I think making mp4 of flv modules capable of streaming videos
from a proxy source would be great. One way around we are just trying
(we are streaming from S3) is to mount the S3 bucket through a FUSE
filesystem and make mp4 stream it from there. There are (unfortunately)
a few other problems with that, e.g. s3fs cannot fetch parts of the file
(seek), it downloads the whole file, but we are on our way for a custom,
read-only filesystem that does the trick.

Caching files locally is also a good option, but when you have thousands
of large movies it makes it quite difficult. Direct streaming would be
much nicer.

@Karolis: how exactly can you do this with lighttpd? Any reference?

Michal

Hi Michal,
How did you get on with this? I’ve been stumbling around with this idea
for a while but not quite cracked it yet. It seems that a block device
approach with the right filesystem could provide the answer, maybe some
sort of cluster fs. Anyway any chance of a collaboration?