Hmm well i have figured out it is my mp4 buffers that need fixing but i
recon my largest video file size on the server is maybe 700mb as of
figuring
out what to set this to i am currently just playing around with it to
see
what works best.
Posted at Nginx Forum:
I think i found the soloution rather than buffer or envolve pseudo
streaming
mp4 already html5 compatible videos.
I just leave it to the browsers rather than my server.
So to solve my I/O usage issue i dropped “mp4;” from my server config
“#mp4;” and now my I/O usage is basically back at 0.
Perhaps nginx should look at the I/O usage to do with that function and
see
if they can make it better.
Posted at Nginx Forum:
Which shows disk IO is much better which to me indicates there were/are
too
many small writes to disk, when some parts are slow tuning is a big time
issue with nginx no matter which OS your running.
Posted at Nginx Forum:
My new soloution did not last very long everything shot up again so the
mp4
function is needed to drop I/O usage but as of what the optimal setting
for
the buffers are realy does baffle me
Posted at Nginx Forum:
Perhaps nginx should look at the I/O usage to do with that function
and see if they can make it better.
Its a disk subsystem issue which is under control by the OS not nginx, a
good 15k sas does wonders.
Posted at Nginx Forum:
So a disk spinning at 15000 rpm compared to my current hard drive
spinning
at 7000 rpm does better than a SSD still ?
This is my current hard drive i posted earlyer i do believe
Posted at Nginx Forum:
I don’t know if what you’re experiencing is related to a problem I’m
still tracking down, specifically that multiple redundant read-streams
and corresponding temp_files are being opened to read the same file from
a backend server for what appears to be a single initial get request by
a client for a large mp4 file which was not yet been locally reverse
proxy cashed by nginx as an substantially static file. This appears to
end up creating 6-10x more traffic and disk activity than is actually
required to cache the single file (depending on how many redundant
read-stream/temp_files are created. If a server is attempting to
reverse proxy many such relatively large files, it could easily saturate
nginx with network/disk traffic until most such files requested were
eventually locally cached.
Latest picture
Everything utilizing the read’s and writes is nginx and when i set the
following buffers i get massive spikes like that.
location ~ .mp4$ {
mp4;
mp4_buffer_size 9000m;
mp4_max_buffer_size 9000m;
}
Posted at Nginx Forum:
easily saturate nginx with network/disk traffic until most such files
Posted at Nginx Forum:
nginx Info Page
Well i don’t proxy anything everything is hosted locally and php is run
by
fastcgi but it is all on the same machine.
Posted at Nginx Forum:
After benchmarking this was my output. I have no idea if it is good or
bad
or what, i am rather hoping someone with more understanding of I/O usage
and
if i have hit my max or not can tell me.
MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 168.230 MB/s
Sequential Write : 166.230 MB/s
Random Read 512KB : 61.540 MB/s
Random Write 512KB : 84.223 MB/s
Random Read 4KB (QD=1) : 0.787 MB/s [ 192.1 IOPS]
Random Write 4KB (QD=1) : 1.701 MB/s [ 415.3 IOPS]
Random Read 4KB (QD=32) : 2.032 MB/s [ 496.2 IOPS]
Random Write 4KB (QD=32) : 1.389 MB/s [ 339.0 IOPS]
Test : 1000 MB [C: 21.9% (427.9/1953.1 GB)] (x9)
Date : 2014/06/29 15:58:08
OS : Windows Server 2008 R2 Web Server SP1 [6.1 Build 7601] (x64)
Posted at Nginx Forum:
Hello!
On Sat, Jun 28, 2014 at 12:38:12AM -0400, c0nw0nk wrote:
}
Note that when using mp4 module, you should not set
mp4_buffer_size too high. It’s initial size of a buffer used to
read mp4 files, and if a value specified is bigger than size of
the moov atom in an mp4 file, reading extra bytes will be
meaningless. It is generally recommended to set it to something
like 512k (the default) or slightly larger (1m, 2m). Larger
values make sense only if most of your mp4 files have huge moov
atoms.
Note well that if you use mp4 pseudo streaming, you should make
sure that mp4 files are correctly prepared - i.e., with moov atom
placed before mdat. While nginx will be able to work with files
with moov atom at the end, it won’t be able to do this
efficiently.
–
Maxim D.
http://nginx.org/
MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 167.799 MB/s
Sequential Write : 166.560 MB/s
Random Read 512KB : 59.075 MB/s
Random Write 512KB : 83.099 MB/s
Random Read 4KB (QD=1) : 0.760 MB/s [ 185.6 IOPS]
Random Write 4KB (QD=1) : 1.755 MB/s [ 428.5 IOPS]
Random Read 4KB (QD=32) : 2.003 MB/s [ 489.0 IOPS]
Random Write 4KB (QD=32) : 1.346 MB/s [ 328.6 IOPS]
Test : 2000 MB [C: 21.9% (427.9/1953.1 GB)] (x9)
Date : 2014/06/29 16:57:08
OS : Windows Server 2008 R2 Web Server SP1 [6.1 Build 7601] (x64)
MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]
Sequential Read : 168.459 MB/s
Sequential Write : 166.944 MB/s
Random Read 512KB : 58.245 MB/s
Random Write 512KB : 81.734 MB/s
Random Read 4KB (QD=1) : 0.741 MB/s [ 180.8 IOPS]
Random Write 4KB (QD=1) : 1.677 MB/s [ 409.5 IOPS]
Random Read 4KB (QD=32) : 1.943 MB/s [ 474.4 IOPS]
Random Write 4KB (QD=32) : 1.696 MB/s [ 413.9 IOPS]
Test : 4000 MB [C: 21.9% (427.9/1953.1 GB)] (x9)
Date : 2014/06/29 17:20:31
OS : Windows Server 2008 R2 Web Server SP1 [6.1 Build 7601] (x64)
Posted at Nginx Forum:
I actualy came accross a setting in my device manager called write cache
buffer flushing. When you disable Write Cache Buffer Flushing, this
allows
application software to blaze ahead after writing data to disk without
waiting for the physical write to complete.
I have enabled it rebooted my machine and will post in a few hours how
things are going with it.
Now this i won’t be a soloution but will deffinetly help write allot
more
data simultaneously. What should help until i find a SSD/SAS to dedicate
my
money to.
Posted at Nginx Forum:
meaningless. It is generally recommended to set it to something
Maxim D.
http://nginx.org/
nginx mailing list
[email protected]
nginx Info Page
All videos are already set out like that i also use the following python
script to ensure that they have meta data at the start of the file
instead
of the end. Use qt-faststart to move the moov atom on all mp4 files in a given folder · GitHub
I did a simple test to confirm it is I/O usage due to large files i run
nginx same traffic conditions but i block “deny all;” to mp4, flv files
and
my i/o usage by nginx is about 1mb max and a few kb so i can confirm it
is
large files that are doing it i am currently looking into what would be
better for me a SSD or a SAS 15K RPM.
Posted at Nginx Forum:
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.
Sponsor our Newsletter | Privacy Policy | Terms of Service | Remote Ruby Jobs