Caching servers in Local ISPs!

Hello,

  We're running a high traffic website similar to youtube.com. Due 

to
high bandwidth utilization over the network, we’re in contact with the
local ISP in order to put caching server to reduce bandwidth utilization
for file streaming. Our main front end content servers (nginx) are
located
in U.S and we want to put caching servers in ASIA as most of the traffic
is
originating from asia.

We’ve no idea how this caching would work. Would the caching servers
will
be configured and deployed by Local ISP ? or we’re required to do some
work
with our application coding ?

I know it is bit off topic on nginx forum. But this is the only forum
that
is intensively active and helpful.

Please guide me a bit, i am new to this caching environment.

Shahzaib

Am 09.05.2014 um 16:58 schrieb shahzaib shahzaib
[email protected]:

Hello,

  We're running a high traffic website similar to youtube.com. Due to high 

bandwidth utilization over the network, we’re in contact with the local ISP in
order to put caching server to reduce bandwidth utilization for file streaming.
Our main front end content servers (nginx) are located in U.S and we want to put
caching servers in ASIA as most of the traffic is originating from asia.

We’ve no idea how this caching would work. Would the caching servers will be
configured and deployed by Local ISP ? or we’re required to do some work with our
application coding ?

I know it is bit off topic on nginx forum. But this is the only forum that is
intensively active and helpful.

Please guide me a bit, i am new to this caching environment.

I think you would need to move your DNS to somebody like easydns.com or
dyn.com (just of the top of my head, there are probably lots more) and
use their geo-location feature.

That way, somebody with an IP from Asia will receive the IP of your
Asian server when it asks DNS about the IP of your domain-name.

Maybe somebody who has done this before can comment - Ive looked into
it, but never seriously.

:wink:

@Rainer, we’re already in contact with one of our Country’s ISP(80% of
the
country users are using that ISP services) .So, they can do much better
work than the DNS sites you provided because we only required caching
for
our country.

Its quite simple, think of it this way, a DNS entry does not have to
point
to the same IP everywhere.

Place your cache machines at a ISP, have them assign its IP to your
preferred dns name, thats about it.

The rest like distribution works like a reverse riverbed with a master
mirror, rsync or the likes.

And of course this can all be done with nginx at all locations.

Posted at Nginx Forum:

@itpp thanks for replying.

So on easy note, i would have to assign those machines the preferred dns
and use rsync on regular basis in order to make identical data between
local caching machines and main front end content servers ?

What if a client request a video which is not in local caching server ?
Does nginx has the configuration for it to check the files locally and
then
forward the request to main content servers if requested file is not
cached
locally ?

I need a bit of guidance in order to configure nginx this way.

Shahzaib

So on easy note, i would have to assign those machines the preferred
dns
and use rsync on regular basis in order to make identical data
between
local caching machines and main front end content servers ?

Yep.

What if a client request a video which is not in local caching server
?

You need to maintain a cache index on each cache machine in order to
determine what is available to the users, for most content you need to
do
this anyway since not all content can legally be everywhere and you also
might want to customize what you present for each region.

Does nginx has the configuration for it to check the files locally and
then
forward the request to main content servers if requested file is not
cached
locally ?

There are many ways to do this with nginx and Lua but a independent
cache
index would be much better, with it you can do much more like redirect a
content source from elsewhere depending on load and demand. You simply
feed
nginx the cache index. A very simplistic cache index system is abusing a
local (local to nginx) dns server, assign local IP’s to resources and
change
them according to load and demand. Again for a local DNS you can assign
whatever you want to a dns name, with a local ttl of 15 seconds and
nginx
loadbalancing between 4 regional resources it will be peanuts to change
the
load based on demand (provided you have monitoring in place which can
act on
such data). Basically a DIY BGP :slight_smile:

Posted at Nginx Forum:

Thanks for replying guyz.

Can i use nginx (origin and edge) ? As the question in following link.

If i use the origin and edge method, i think i’ll change my application
codes to redirect local country traffic to edge webservers (ISP caching
server for video files) and that edge server will check if the requested
file is not in cache and it’ll fetch the requested video file from
origin
web-server located in U.S and cache it to local.

For this procedure,

I’ll have to configure DNS A entries against local ISP caching servers
and
put those DNS to my application code to stream videos from those LOCAL
CACHING SERVERS for specific country.

Please correct me if i am wrong.

You might want to look at lsyncd - a GZSOC project - to ease the
synchronisation. I have had good results with it.

Steve
On Sat, 2014-05-10 at 00:22 +0500, shahzaib shahzaib wrote:

locally and then forward the request to main content servers if

    The rest like distribution works like a reverse riverbed with
    nginx mailing list
    [email protected]
    http://mailman.nginx.org/mailman/listinfo/nginx

nginx mailing list
[email protected]
nginx Info Page


Steve H. BSc(Hons) MIITP

Linkedin: http://www.linkedin.com/in/steveholdoway
Skype: sholdowa

Hello ,

Came after a long time to this thread. I have finished the caching

method. I’ve used bind-geoip to resolve all requests from asia against
the
local ISP caching server ips. Now, here’s the most critical part.
Suppose a
request came from asia for the video file which is not yet downloaded on
local caching server, how will the local nginx will deal with it ? I
want
it to forward that request to the main Server and keep forwarding
request
until the next rsync runs and new files propagate to the local caching
server.

Please point me to some nginx guide if this feature works well with
nginx.

Regards.

I’d call that provisioning, a simple try_files directive can deal with
that
while the provisioning service syncs.
http://nginx.org/en/docs/http/ngx_http_core_module.html#try_files

Then jump left for cached contents or jump right for non-cached
contents.

Posted at Nginx Forum:

See Content delivery network - Wikipedia
and File:Akamaiprocess.png - Wikipedia

Make yourself a HLD (high level design) before getting to technology.

Posted at Nginx Forum:

I’m not seeing where you determine if a file is present on an edge
system,
the try_files directive allows you to redirect or not (locally present
or
not). I don’t really see a point for using (proxy)cache, the file is
present
or it is not.

location / {
try_files $uri @getfrom_origin;
}

location @getfrom_origin {
proxy_pass …;
}

Posted at Nginx Forum:

@itpp, thanks for reply. I am working on proxy_cache . My all webservers
in
local ISP will be proxy servers and all requests regarding flv,mp4,jpeg
from asia will be route through the proxy servers and data will be cache
in
proxy servers. I think, this will help me with caching. I am testing it
on
local and its working as it should. Following is the edge server config
:

proxy_cache_path /data/nginx/cache levels=1:2 keys_zone=static:1000m;
proxy_temp_path /tmp;

server {
listen 80;
server_name edge.com;

proxy_cache one;

    root /var/www/html/test;

access_log /var/log/nginx/access.log ;
error_log /var/log/nginx/error.log;

location / {
proxy_pass http://origin.com:80;
proxy_cache static;
proxy_cache_min_uses 1;
proxy_cache_valid 200 301 302 120m;

proxy_cache_valid 404 1m;

}
}

Please let me know if the method is correct.

Hello itpp,

       the Local caching server is nothing but a proxy server which

will route all traffic from asia to origin server and will cache the
requested file in /data/nginx/cache folder. So when the request made
again,
it’ll be served from cache.

I’m not seeing where you determine if a file is present on an edge system,
Edge system will route request for new file to origin server and will
cache
the resulted file to edge server,so when the new request for same file
will arrive, nginx edge server will serve it from /data/nginx/cache
folder.

I did experiment on locally and by checking the logs, i think its
working .
I got hint from following link :

Thanks itpp. So according to you the more preferable way is non-caching
replication of main content servers to local ISP servers and than run
rsync
periodically for replication. So if file not found in local ISP nginx,
it
should forward request to main content server using try_files method ?

@itpp22, proxy_cache is eating too much memory and cpu on local testing.
1
nginx process was using 1G of Ram and load-avg reached to 22, eventually
local server got killed.

Local server specs are : 8cores and 4G ram.

Looks like proxy_cache will not be suitable method for 3000+ concerrent
connections on local ISP caching server.

Right. I’ll proceed with my research and get back to you with better
approach . :slight_smile:

shahzaib1232 Wrote:

Thanks itpp. So according to you the more preferable way is
non-caching
replication of main content servers to local ISP servers and than run
rsync
periodically for replication. So if file not found in local ISP nginx,
it
should forward request to main content server using try_files method ?

@itpp22, proxy_cache is eating too much memory and cpu on local testing.
1
nginx process was using 1G of Ram and load-avg reached to 22, eventually
local server got killed.

Local server specs are : 8cores and 4G ram.

Looks like proxy_cache will not be suitable method for 3000+ concerrent
connections on local ISP caching server.

A HLD should have pointed that out :slight_smile:
You should change your thinking what caching actually is, just because
some
functionality is called ‘caching’ it doesn’t mean anything else is not.

If exist do {
present local via local interfaces
} else {
present remote via remote interfaces
}

Another (none-nginx) process (provisioning) controls which remote
components
flows to local (or is removed) via provisioning interfaces.

Look at something simple like a filesystem with replication, in it’s
basics
it’s not a cache but more a glorified 2way rsync solution. However
global
files are always local to be determined to which resource you are
connecting
to, yet the filesystem doesn’t need to cache much no matter how many
files
it serves, which doesn’t defer much from how nxing would work.

Posted at Nginx Forum:

Of course this will work but when nginx restarts or the cache gets
damaged/corrupted the whole shebang starts again… And you can’t
control
the cached files versus try_files and a normal filesystem.

Posted at Nginx Forum:

@itpp, i just used your method try_files and it worked flawlessly :).
Following is the testing config :

server {
listen 80;
server_name domain.com;
root /var/www/html/files;

location / {
root /var/www/html/files;
try_files $uri @getfrom_origin;
}
location @getfrom_origin {
proxy_pass http://127.0.0.1:8080;
}
}

if proxy_pass worked for localhost, i hope it will also work for remote
host to forward request if the file doesn’t exist on local caching
server.
:slight_smile:

Would you suggest me to add some more configs for tweaking on nginx ?

Btw, proxy_pass should only be for mp4 and jpeg, cause the caching is
only
for video files.

Should i use rsync or lsync for mirroring the files between Origin and
caching server ?

Suggestion will be highly appreciated.

Regards.
Shahzaib

On Wed, May 28, 2014 at 4:19 PM, shahzaib shahzaib
[email protected]