So I was thinking of creating a ramdisk and then pointing proxy_cache at the ramdisk. do you think that will be a good combo? If so, to the people that use proxy_cache, how much space is it using on average so I can make it the right size?
on 2010-01-29 17:45
on 2010-01-31 12:47
Is there any examples of a proxy_cache config? I see the wiki but I would like to see some working example that people are using successfully. Right now I'm just using php-fpm, nginx, and xCache. Not sure how to get the most out of and utilize proxy_cahce.
on 2010-02-01 04:05
On Friday, January 29, 2010, AMP Admin <admin@ampprod.com> wrote: > So I was thinking of creating a ramdisk and then pointing > proxy_cache at the ramdisk… do you think that will be a good combo? Works fine with the cache dir in /tmp on Ubuntu Linux. Tmpfs is a ram disk solution. > If so, to the people that use proxy_cache, how much space is > it using on average so I can make it the right size? Sizing totally depends on the sites and applications you are proxying. As a start, look at your sites log files to see which files are hit frequently , then add up their size. -- RPM
on 2010-02-01 13:59
On 1/29/10 11:44 AM, "AMP Admin" <admin@ampprod.com> wrote: > So I was thinking of creating a ramdisk and then pointing proxy_cache at the > ramdiskŠ do you think that will be a good combo? We use some very fast SSD's and that works very well. -- Brian Akins
on 2010-02-01 15:14
On Sun, Jan 31, 2010 at 9:12 PM, quan nexthop <quan.nexthop@gmail.com>
wrote:
> Could you please share us your configuration ?
The basics of tmpfs are here: http://en.wikipedia.org/wiki/Tmpfs
Here's my config with most comments and white-space trimmed out:
________________________________________
worker_processes 4;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
access_log /var/log/nginx/access.log;
sendfile on;
keepalive_timeout 65;
tcp_nodelay on;
gzip on;
gzip_disable "MSIE [1-6]\.(?!.*SV1)";
gzip_proxied any;
gzip_http_version 1.0;
gzip_types text/plain text/xml text/css text/javascript
application/x-javascript;
gzip_buffers 8 32k;
gzip_comp_level 2;
#put nginx cache files into tmpfs ram disk that is backed by swap
proxy_cache_path /tmp/nginx_cache
levels=1:2
keys_zone=zone1:10m
inactive=7d
max_size=128m;
upstream backend {
ip_hash;
server 10.204.122.137;
server 10.224.122.138;
}
server {
listen 80;
server_name_in_redirect off;
#normalize all accept-encoding headers to just gzip or empty string
set $myae "";
if ($http_accept_encoding ~* gzip) {set $myae "gzip";}
#get rid of accept-encoding header for known-compressed file types
#we only want to store one version of these files in cache
if ($uri ~* \.(gif|jpg|jpeg|png|pdf|swf|flv|mp3|mp4|wmv)$) {set
$myae "";}
access_log /var/log/nginx/localhost.access.log;
location / {
proxy_pass http://backend;
proxy_set_header Host $host;
proxy_set_header Accept-Encoding $myae;
proxy_read_timeout 900;
proxy_redirect default;
proxy_cache zone1;
proxy_cache_valid 200 301 302 1m;
proxy_cache_key "$host$request_uri$myae";
}
}
on 2010-02-01 16:29
I'm able to cache php pages with the following but I can't seem to cache
static images with proxy_cache.
This works:
location ~ \.php$ {
fastcgi_index
index.php;
fastcgi_pass
127.0.0.1:9000;
fastcgi_cache
cachephp;
fastcgi_cache_key
127.0.0.1:9000$request_uri;
fastcgi_cache_valid
200 1h;
include
fastcgi_params;
fastcgi_intercept_errors
On;
fastcgi_ignore_client_abort
On;
fastcgi_buffer_size
128k;
fastcgi_buffers
4 128k;
}
This does not work:
location ~*
\.(jpg|jpeg|gif|css|png|js|ico|tif)$ {
access_log
off;
expires
30d;
proxy_pass
http://127.0.0.1;
proxy_cache_key
$scheme$host$request_uri
proxy_cache
cachestatic;
proxy_cache_valid
200 1h;
proxy_cache_valid
404 5m;
break;
}
Using:
fastcgi_temp_path
/etc/nginx/temp_cache;
fastcgi_cache_path
/etc/nginx/cache
levels=1:2
keys_zone=cachephp:10m
inactive=7d
max_size=128m;
proxy_temp_path
/etc/nginx/temp_cache;
proxy_cache_path
/etc/nginx/cache
levels=1:2
keys_zone=cachestatic:10m
inactive=7d
max_size=128m;
on 2010-02-01 17:01
-----Original Message-----
From: Ryan Malayter [mailto:malayter@gmail.com]
Sent: Monday, February 01, 2010 8:14 AM
To: nginx@nginx.org
Subject: Re: proxy_cache ramdisk
On Sun, Jan 31, 2010 at 9:12 PM, quan nexthop <quan.nexthop@gmail.com>
wrote:
> Could you please share us your configuration ?
The basics of tmpfs are here: http://en.wikipedia.org/wiki/Tmpfs
Here's my config with most comments and white-space trimmed out:
________________________________________
worker_processes 4;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
access_log /var/log/nginx/access.log;
sendfile on;
keepalive_timeout 65;
tcp_nodelay on;
gzip on;
gzip_disable "MSIE [1-6]\.(?!.*SV1)";
gzip_proxied any;
gzip_http_version 1.0;
gzip_types text/plain text/xml text/css text/javascript
application/x-javascript;
gzip_buffers 8 32k;
gzip_comp_level 2;
#put nginx cache files into tmpfs ram disk that is backed by swap
proxy_cache_path /tmp/nginx_cache
levels=1:2
keys_zone=zone1:10m
inactive=7d
max_size=128m;
upstream backend {
ip_hash;
server 10.204.122.137;
server 10.224.122.138;
}
server {
listen 80;
server_name_in_redirect off;
#normalize all accept-encoding headers to just gzip or empty string
set $myae "";
if ($http_accept_encoding ~* gzip) {set $myae "gzip";}
#get rid of accept-encoding header for known-compressed file types
#we only want to store one version of these files in cache
if ($uri ~* \.(gif|jpg|jpeg|png|pdf|swf|flv|mp3|mp4|wmv)$) {set
$myae "";}
access_log /var/log/nginx/localhost.access.log;
location / {
proxy_pass http://backend;
proxy_set_header Host $host;
proxy_set_header Accept-Encoding $myae;
proxy_read_timeout 900;
proxy_redirect default;
proxy_cache zone1;
proxy_cache_valid 200 301 302 1m;
proxy_cache_key "$host$request_uri$myae";
}
}
_______________________________________________
Thanks!!!!
So this is only caching gif|jpg|jpeg|png|pdf|swf|flv|mp3|mp4|wmv?
That's almost exactly what I want.
Upstream backend is the ip address of what nginx is running on? This
box has 4 IPs for 4 different sites, should I list them all in upstream
backend?
on 2010-02-01 17:07
Oh and I added the following to the nginx startup script so it will
always create the ram drives if they don't already exsist when nginx
starts. Not sure if that helps anyone else or not.
Obviously change the path and name to your config. This is a centos
box.
start() {
if /bin/mount | grep temp_cache ; then
echo "is mounted"
else
echo "Mounting nginx Temp Cache"
mount -t tmpfs none /etc/nginx/temp_cache -o size=128m
fi
if /bin/mount | grep nginx_cache ; then
echo "is mounted"
else
echo "Mounting Nginx Cache"
mount -t tmpfs none /etc/nginx/nginx_cache -o size=128m
fi
on 2010-02-01 17:34
On Fri, Jan 29, 2010 at 10:44:46AM -0600, AMP Admin wrote: > So I was thinking of creating a ramdisk and then pointing proxy_cache at the > ramdisk. do you think that will be a good combo? > > If so, to the people that use proxy_cache, how much space is it using on > average so I can make it the right size? Caching in ramdisk is usually not good idea, since there is OS VM cache. You just waste memory and CPU time. -- Igor Sysoev http://sysoev.ru/en/
on 2010-02-01 17:38
On Sun, Jan 31, 2010 at 04:26:05PM -0600, AMP Admin wrote: > fastcgi_cache cachephp; > fastcgi_cache_key 127.0.0.1:9000$request_uri; > fastcgi_cache_valid 200 1h; > include fastcgi_params; > fastcgi_intercept_errors On; > fastcgi_ignore_client_abort On; "On" are invalid parameters. > expires 30d; > proxy_pass http://127.0.0.1; > proxy_cache_key $scheme$host$request_uri > proxy_cache cachestatic; > proxy_cache_valid 200 1h; > proxy_cache_valid 404 5m; Why do you want to cache image from localhost ? You should handle them as static files. > break; > } "break" is useless here. It's just waste of CPU cycles. -- Igor Sysoev http://sysoev.ru/en/
on 2010-02-01 17:51
-----Original Message----- From: Igor Sysoev [mailto:igor@sysoev.ru] Sent: Monday, February 01, 2010 10:38 AM To: nginx@nginx.org Subject: Re: proxy_cache ramdisk On Sun, Jan 31, 2010 at 04:26:05PM -0600, AMP Admin wrote: > fastcgi_cache cachephp; > fastcgi_cache_key 127.0.0.1:9000$request_uri; > fastcgi_cache_valid 200 1h; > include fastcgi_params; > fastcgi_intercept_errors On; > fastcgi_ignore_client_abort On; "On" are invalid parameters. > expires 30d; > proxy_pass http://127.0.0.1; > proxy_cache_key $scheme$host$request_uri > proxy_cache cachestatic; > proxy_cache_valid 200 1h; > proxy_cache_valid 404 5m; Why do you want to cache image from localhost ? You should handle them as static files. > break; > } "break" is useless here. It's just waste of CPU cycles. -------- So I should just really just use cache like this: location ~ \.php$ { fastcgi_index index.php; fastcgi_pass 127.0.0.1:9000; fastcgi_cache cachephp; fastcgi_cache_key 127.0.0.1:9000$request_uri; fastcgi_cache_valid 200 1h; include fastcgi_params; } And images like this: location ~* \.(jpg|jpeg|gif|css|png|js|ico|tif)$ { access_log off; expires 30d; } That would be best for performance? Also, I noticed when I use the fastcgi_cache above that it caches logged in pages and users see the wrong user profile... should I change the fastcgi_cache_key to fix that?
on 2010-02-01 18:06
On Mon, Feb 01, 2010 at 10:50:37AM -0600, AMP Admin wrote: > > > > include fastcgi_params; > > > > > > And images like this: > location ~* \.(jpg|jpeg|gif|css|png|js|ico|tif)$ { > access_log off; > expires 30d; > } > > That would be best for performance? Yes. > Also, I noticed when I use the fastcgi_cache above that it caches logged in > pages and users see the wrong user profile... should I change the > fastcgi_cache_key to fix that? Yes. Probably, you may add an user cookie. Actually, you should mark logged pages as non-cachable, otherwise transient proxy servers may cache them as well. -- Igor Sysoev http://sysoev.ru/en/
on 2010-02-01 18:19
HI Igor Sysoev:
Could you please give us an explaination for these ?
> Caching in ramdisk is usually not good idea, since there is OS VM cache.
You just waste memory and CPU time.
[nexthop] If we want to get a high performace, can we cache using
RAMdisk ?
According to your comments, it seems that cache in ramdisk have a lower
performace.
I do a google search with ramdisk cache, it seems that Cache in RamDisk
will
get a high performance.
http://lowendmac.com/tech/diskcache.shtml
I am confusing it.
Could you please give us more information about it ?
thanks
Nexthop
on 2010-02-01 18:26
-----Original Message----- From: Igor Sysoev [mailto:igor@sysoev.ru] Sent: Monday, February 01, 2010 11:06 AM To: nginx@nginx.org Subject: Re: proxy_cache ramdisk On Mon, Feb 01, 2010 at 10:50:37AM -0600, AMP Admin wrote: > > > > include fastcgi_params; > > > > > > And images like this: > location ~* \.(jpg|jpeg|gif|css|png|js|ico|tif)$ { > access_log off; > expires 30d; > } > > That would be best for performance? Yes. > Also, I noticed when I use the fastcgi_cache above that it caches logged in > pages and users see the wrong user profile... should I change the > fastcgi_cache_key to fix that? Yes. Probably, you may add an user cookie. Actually, you should mark logged pages as non-cachable, otherwise transient proxy servers may cache them as well. ------------------------ All pages on this site have a login option so I guess caching our site isn't a good idea.
on 2010-02-01 18:32
On Tue, Feb 02, 2010 at 01:19:16AM +0800, quan nexthop wrote: > I do a google search with ramdisk cache, it seems that Cache in RamDisk will > get a high performance. > http://lowendmac.com/tech/diskcache.shtml > > I am confusing it. > Could you please give us more information about it ? RAMdisk has no lower performace. It just wastes memory: OS stores two copies of an object: one in OS cache and second in RAMdisk. Some tmpfs implementations may eliminate this double copy. > > > > nginx@nginx.org > > http://nginx.org/mailman/listinfo/nginx > > > _______________________________________________ > nginx mailing list > nginx@nginx.org > http://nginx.org/mailman/listinfo/nginx -- Igor Sysoev http://sysoev.ru/en/
on 2010-02-01 18:36
On Mon, Feb 01, 2010 at 11:26:23AM -0600, AMP Admin wrote: > > Sent: Monday, February 01, 2010 10:38 AM > > > > > > > > location ~* \.(jpg|jpeg|gif|css|png|js|ico|tif)$ { > > > > > > access_log off; > > fastcgi_cache_key to fix that? > > Yes. Probably, you may add an user cookie. > Actually, you should mark logged pages as non-cachable, otherwise transient > proxy servers may cache them as well. > > ------------------------ > > All pages on this site have a login option so I guess caching our site isn't > a good idea. Yes, even if you cache them properly in nginx, they will have poor hit ratio. However, you may try to cache non-personalized pages. -- Igor Sysoev http://sysoev.ru/en/
on 2010-02-01 23:46
On Mon, Feb 1, 2010 at 10:00 AM, AMP Admin <admin@ampprod.com> wrote: > > So this is only caching gif|jpg|jpeg|png|pdf|swf|flv|mp3|mp4|wmv? Â That's almost exactly what I want. No it is caching anything that is marked as publicly cachable with Cache-Control or Expires headers on the upstream. I am just removing the "Accept-Encoding: gzip" for image requests because I do not want to cache both compressed and uncompressed versions (since they are already compressed). > Upstream backend is the ip address of what nginx is running on? Â This box has 4 IPs for 4 different sites, should I list them all in upstream backend? No the upstream block has the IP addresses of the web servers nginx is proxying and load-balacing to (IIS in this case running ASP.net applications). -- RPM
on 2010-02-02 15:43
On Mon, Feb 1, 2010 at 11:35 AM, Igor Sysoev <igor@sysoev.ru> wrote: >> All pages on this site have a login option so I guess caching our site isn't >> a good idea. > > Yes, even if you cache them properly in nginx, they will have poor hit ratio. > However, you may try to cache non-personalized pages. You can cache pages for "public" users that aren't logged in, and then serve the pages directly without caching to users who are logged in. This can be done simply by simply changing the URI in your application a bit. For example, anonymous users would hit "http://example.com/index.php", and the application could return a "Cache-Control: public,max-age=3600". After a user logs in, you can simply redirect them to "http://example.com/loggedin/index.php". This virtual diectory can run the same code, but instead it sets a "Cache-Control: private, max-age=0" header. So logged in users would not received cached pages. Another option in nginx would be to use $http_cookie (or a variable based on it) as part of the proxy_cache_key. Users with an empty cookie would get the cached page for public users, while users that have logged in and have a session cookie would be proxied directly to the back-end. Again, you should set appropriate "Cache-Control" headers in the application for anonymous versus logged-in users. Finally, all of your static images, .js files, and CSS files should have appropriate cache-control headers, so they can be cached for both logged-in and anonymous users. As Igor points out, this only makes sense if those files are not already "local" to the nginx server, but are instead being served from back-end via proxy. You should set Cache-Control headers on these files even if you aren't using nginx proxy_cache at all, since you still want to take advantage of any caches "downstream" from you (such as at the user's ISP, or in their corporate datacenter). These shared caches are actually more common than people think, and many are transparent to end-users. When we first implemented cache-control headers on static files many years ago, our bandwidth bill dropped by 75% in one month. -- RPM
on 2010-02-02 18:51
-----Original Message----- From: Ryan Malayter [mailto:malayter@gmail.com] Sent: Tuesday, February 02, 2010 8:43 AM To: nginx@nginx.org Subject: Re: proxy_cache ramdisk On Mon, Feb 1, 2010 at 11:35 AM, Igor Sysoev <igor@sysoev.ru> wrote: >> All pages on this site have a login option so I guess caching our site isn't >> a good idea. > > Yes, even if you cache them properly in nginx, they will have poor hit ratio. > However, you may try to cache non-personalized pages. You can cache pages for "public" users that aren't logged in, and then serve the pages directly without caching to users who are logged in. This can be done simply by simply changing the URI in your application a bit. For example, anonymous users would hit "http://example.com/index.php", and the application could return a "Cache-Control: public,max-age=3600". After a user logs in, you can simply redirect them to "http://example.com/loggedin/index.php". This virtual diectory can run the same code, but instead it sets a "Cache-Control: private, max-age=0" header. So logged in users would not received cached pages. Another option in nginx would be to use $http_cookie (or a variable based on it) as part of the proxy_cache_key. Users with an empty cookie would get the cached page for public users, while users that have logged in and have a session cookie would be proxied directly to the back-end. Again, you should set appropriate "Cache-Control" headers in the application for anonymous versus logged-in users. Finally, all of your static images, .js files, and CSS files should have appropriate cache-control headers, so they can be cached for both logged-in and anonymous users. As Igor points out, this only makes sense if those files are not already "local" to the nginx server, but are instead being served from back-end via proxy. You should set Cache-Control headers on these files even if you aren't using nginx proxy_cache at all, since you still want to take advantage of any caches "downstream" from you (such as at the user's ISP, or in their corporate datacenter). These shared caches are actually more common than people think, and many are transparent to end-users. When we first implemented cache-control headers on static files many years ago, our bandwidth bill dropped by 75% in one month. -- RPM _______________________________________________ Good to know... thanks!
Enable email notification
Enable multi-page viewPlease log in before posting. Registration is free and takes only a minute.
Existing account
(Switch to SSL-encrypted connection)
NEW: Do you have a Google/GoogleMail or Yahoo account? No registration required!
Log in with Google account | Log in with Yahoo account
Log in with Google account | Log in with Yahoo account
No account? Register here.