Nginx + proxy_store + flv/mp4

I’m trying to set up sort of an origin/edge config for nginx to stream
mp4s and flvs.

I’m using proxy_store to cache flvs/mp4s from the origin on the
frontends. I also want to enable seeking in those files using the
flv and mp4 (mp4 is a module from h264.code-shop.com) directives.

The problem I have is that I can’t get the seeking to work if the file
hasn’t
previously been cached on the frontend and I don’t want to enable
flv/mp4
modules on the backend since that would mean the cache on the frontend
is polluted with lots of “seeked” files. I only want the whole files
cached.

Is it possible to somehow make nginx do this:

  1. If file not in proxy store, get from backend wait until completely
    downloaded else go to 2
  2. Start streaming from local cache

What I mean is - is it possible to get nginx to ALWAYS read from local
store (proxy_store) and
not just pass along the data received from backend?

I want seeking to always work and I only want one file cached locally.

Thanks for an AWESOME webbserver!

/John

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,11284,11284#msg-11284

Hello!

On Mon, Oct 05, 2009 at 12:04:41PM -0400, benjinx wrote:

is polluted with lots of “seeked” files. I only want the whole files cached.

Thanks for an AWESOME webbserver!

You may try to use

http://mdounin.ru/hg/ngx_http_bytes_filter_module/

instead of flv on frontend. It works as filter, and should be
able to handle flv seeking while still downloading full file from
backend to use in cache/proxy_store.

Maxim D.

To be clear, I’m actually specifically using the h264/mp4 module now. As
far as I’ve understood
you don’t seek through bytes but rather time when you seek flv/mp4 files
since byte!=time… perhaps
I’m mistaken and those flv/mp4 mods are unnecessary?

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,11284,11309#msg-11309

Hello!

On Mon, Oct 05, 2009 at 01:19:10PM -0400, benjinx wrote:

To be clear, I’m actually specifically using the h264/mp4 module now. As far as I’ve understood
you don’t seek through bytes but rather time when you seek flv/mp4 files since byte!=time… perhaps
I’m mistaken and those flv/mp4 mods are unnecessary?

For flv, player just reports byte position to start downloading
from, as calculated (by the player) from embedded metadata.

Basically player may do the same thing without any server modules -
just by using normal http partial downloads via Range header. No
idea why they don’t do this - probably the reason is limited http
client support inside Flash.

For mp4, player reports times, and this won’t work without module
which is able to parse mp4 file content and convert this to byte
positions (and a bunch of server-side work otherwise unneeded).
No idea why people use this for streaming.

Maxim D.

Well. I’ll tell you the reason :-).

I’m working on the infrastructure for streaming more than 20 radio
channels. Content
from the last 30 days should be available in several bitrates for a
variety of players,
not only Flash. This content totals about 4 TB or more at any one time.
The reason for
choosing AAC/Mp4 is because of compatibility and ease of compression. We
simply don’t
want to add yet another format to store (flv). Since flash can play
AAC/Mp4 now we want to
use that with the Flash player.

I’ve tested the h264 module and it seems to be pretty fast and working
well. My only problem
is the mirroring on demand through nginx by using proxy_store. I guess
there are other ways
to do this but this solution was SO simple and elegant if not for this
little problem which may very
well force me to find another solution.

Any ideas?

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,11284,11343#msg-11343

Hello!

On Mon, Oct 05, 2009 at 02:56:05PM -0400, benjinx wrote:

is the mirroring on demand through nginx by using proxy_store. I guess there are other ways
to do this but this solution was SO simple and elegant if not for this little problem which may very
well force me to find another solution.

Any ideas?

If you stick with mp4, you may do something like:

  1. If there is no file in store on frontend in question, fallback
    to script.

  2. From script request full file via frontend in question (and
    proxy_store it there).

  3. Return X-Accel-Redirect to the file in store.

Maxim D.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs