Re: SlowFS Cache or Proxy_Cache for GridFS

Server NGINX
1

Hi,

AFAIK it still uses the Mongo-provided C library, which is blocking, so
you’ll lose many of the benefits of Nginx’s performance.

Marcus.

Thanks for the hint. Blocking sounds not good, i found some threads
about issues with gridfs.

GridFS does not use the upstream, if it did then it’d be
easy to hack a cache. You could use memcache and srcache.

Thanks for this option, is memcache really a good option for caching
images?

What do you think about such a configuration?

proxy_cache_path /data/http levels=1 keys_zone=httpcache:2048m;

server {
listen 80;
location / {
proxy_cache httpcache;
proxy_cache_valid 1d;
proxy_cache_key $request_uri;
proxy_pass http://127.0.0.1:81;
}
}

server {
listen 81;
location / {
gridfs mediadb field=filename type=string;
}
}

Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de

2

Hi,

What do you think about such a configuration?
(…)

Please avoid using gridfs module in production,
it’s probably the worst nginx module I ever saw.

Best regards,
Piotr S. < [email protected] >

3

Please avoid using gridfs module in production,
it’s probably the worst nginx module I ever saw.

Thanks so much for this information Piotr. And by the way thanks
for sharing such useful addons like ngx_cache_purge.

I will rethink my idea, but GridFS/MongoDB replication maintains
so nice instead of rsync copy jobs :slight_smile:

Have a nice day.

Elena

Schon gehört? GMX hat einen genialen Phishing-Filter in die
Toolbar eingebaut! http://www.gmx.net/de/go/toolbar

4

If you wanted to use gridfs, you could have a really simple script that
nginx proxied to (or via fastcgi) that did the gridfs stuff. You’d get
caching from the proxy module and avoid blocking. Not ideal, but it
should
work fine. A simple thing in perl/ruby/C, etc.

–
Brian A.

5

Hi,

On 22/02/2011 10:19, Elena Z. wrote:

I will rethink my idea, but GridFS/MongoDB replication maintains
so nice instead of rsync copy jobs :slight_smile:
Given the popularity of MongoDB, I’m sure it won’t be too long before an
Nginx non-blocking module appears for it. This would probably happen
quicker if the Mongo guys provided a async client C library. I would
suggest contacting them or finding an appropriate forum asking them
about an async client. The more people that mention it to them, the
quicker they’re likely to get one done.

If you’re looking for a clustered filesystem, you might also want to
consider MogileFS - there’s an Nginx module for that (which I’m pretty
sure is non-blocking).

Marcus.

6

If you wanted to use gridfs, you could have a really simple script that
nginx proxied to (or via fastcgi) that did the gridfs stuff. You’d get
caching from the proxy module and avoid blocking. Not ideal, but it
should
work fine. A simple thing in perl/ruby/C, etc.

Hello Brian,

thanks for your answer. Yes, that sounds interesting, i will try such a
setup with node.js which should non-blocking too, an use the nginx
proxy_cache function.

Thanks for your ideas.

–
GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit
gratis Handy-Flat! Aktuelle Nachrichten aus Politik, Wirtschaft & Panorama | GMX

7

Elena,
would you please share your experiences with us? it’s been 3 ears and
now we
(a small startup) want to create a production solution that can scale.

Mongo still didn’t release asynchronous driver and we want to go down
nginx+nodejs script for gridfs (nginx-gridfs proejct also no longer
supported).

I’d love to hear from you if possible,

Best,
Alex on behalf of Reviyou team

Posted at Nginx Forum:

8

Thanks Roberto,
since we are not using python in our stack I just realized that we can
keep
async script\route as part of our play application and only add
proxy_cache
nginx configuration for this route. I think we can even leave it without
the
same app server and application, would be simpler to maintain and we can
always scale it by adding a few more app servers just for image
processing
if needed.

thanks,
Alex

Posted at Nginx Forum:

9

Elena,
would you please share your experiences with us? it’s been 3 ears and now
we
(a small startup) want to create a production solution that can scale.

Mongo still didn’t release asynchronous driver and we want to go down
nginx+nodejs script for gridfs (nginx-gridfs proejct also no longer
supported).

I’d love to hear from you if possible,

You may be interested in this:

http://uwsgi-docs.readthedocs.org/en/latest/GridFS.html

–
Roberto De Ioris
http://unbit.it