Rails on Mongrel

Rails
1

Hi,

After following advice from you good folks, I gave mongrel a try in a
cluster design based on the example on the Mongrel website - and it
worked right out of the box which is great - and it’s very fast :slight_smile:

I have got a bit stuck though. I have two MS Word files in
/public/files/. I can pull back these files no problem but it isn’t
sending a mimetype.

I think I have two options, either to tell lighttpd to serve files in
/public/ itself and not pass them on to mongrel ( I have various paths
though i.e /store, /csv and /products to proxy ) or add to mongrels list
of mime types.

I’d prefer to do the former - but if that can’t be done, does anyone
have an example mime file in YAML format? I’ve googled around a bit but
I haven’t been able to find any examples of this.

Can anyone help or point me in the right direction?

Thanks in advance!

2

I would definitly choose the 1st solution, proxy Mongrel to serve
Rails request and use Apache/Lighty to do the static serving, here’s a
Apache 2.2 config snippet for it:

################# WebRick/Mongrel ######################
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so

<Directory “@@DocumentRoot@@”>
Options Indexes FollowSymLinks
AllowOverride none
Order allow,deny
Allow from all

ProxyRequests Off

<Proxy *>
Order deny,allow
Allow from all

ProxyPass /images !
ProxyPass /stylesheets !
ProxyPass /javascripts !
#ProxyPass /whatever_path_you_dont_want_to_send_to_mongrel !

ProxyPass / http://127.0.0.1:9001/
ProxyPassReverse / http://127.0.0.1:9001/

########################################################

3

I’d prefer to do the former - but if that can’t be done, does anyone
have an example mime file in YAML format? I’ve googled around a bit but
I haven’t been able to find any examples of this.

Can anyone help or point me in the right direction?

Here is my mime.yaml file:

.ai: application/postscript
.asc: text/plain
.asf: video/x-ms-asf
.asx: video/x-ms-asf
.avi: video/x-msvideo
.bin: application/octet-stream
.bz2: application/x-bzip
.c: text/plain
.conf: text/plain
.css: text/css
.doc: application/msword
.dot: application/msword
.dtd: text/xml
.eps: application/postscript
.exe: application/octet-stream
.gif: image/gif
.gtar: application/x-gtar
.gz: application/x-gzip
.htm: text/html
.html: text/html
.jpeg: image/jpeg
.jpg: image/jpeg
.js: text/javascript
.m3u: audio/x-mpegurl
.mov: video/quicktime
.mp3: audio/mpeg
.mpeg: video/mpeg
.mpg: video/mpeg
.ogg: audio/x-wav
.pac: application/x-ns-proxy-autoconfig
.pdf: application/pdf
.png: image/png
.pps: application/powerpoint
.ppt: application/powerpoint
.ps: application/postscript
.qt: video/quicktime
.rtf: application/rft
.swf: application/x-shockwave-flash
.tar.bz2: application/x-bzip-compressed-tar
.tar.gz: application/x-tgz
.tar: application/x-tar
.tbz: application/x-bzip-compressed-tar
.text: text/plain
.tgz: application/x-tgz
.txt: text/plain
.txt: text/plain
.wav: audio/x-wav
.wax: audio/x-ms-wax
.wma: audio/x-ms-wma
.wmv: video/x-ms-wmv
.xbm: image/x-xbitmap
.xls: application/excel
.xml: text/xml
.xpm: image/x-xpixmap
.xwd: image/x-xwindowdump
.zip: application/zip

Jonathan

4

Also, as Jonathan mentioned previously, the mime setting is broken on
win32.
It’ll be fixed tonight. I would also agree with Gael that you should
really
get lighttpd to do the serving. Take a look at the CML script in the
lighttpd instructions for some quite nice voodoo to pull this off
simply.

Zed

5

I guess because you’re proxying Mongrel behind Lighty so Mongrel sees
a request coming from Lighty and not your actual users…
Not sure you can do anything about this…

Gael

6

Hi,

I’ve just been testing the setup and from machines in the main office,
everything is fine. However when people connect from the stores,
request.remote_ip shows “127.0.0.1”. When checking lighty’s logs, it’s
showing the correct IP.

This poses a problem because the application has to go live tomorrow
morning (trust me, had I had my way there’d have been time for proper
testing) and the system uses the remote IPs to determine which store is
connecting to the application :S

There are no proxies in the way and as I’ve said, lighty shows the
correct IP in its logs - just Mongrel can’t see it for some reason.

Any suggestions? :slight_smile:

Thanks in advance!

Pete Palmer

7

Gael P. wrote:

I guess because you’re proxying Mongrel behind Lighty so Mongrel sees
a request coming from Lighty and not your actual users…
Not sure you can do anything about this…

Gael

I would tend to agree - yet it works perfectly for machines in the
office. I really can’t see how this can happen. The office network is on
90.1.* and the store network is 10.* - they both have direct
connections to the server. Lighty can see both real IPs - but Mongrel
can’t see the store IPs.

8

Gael P. wrote:

I guess because you’re proxying Mongrel behind Lighty so Mongrel sees
a request coming from Lighty and not your actual users…
Not sure you can do anything about this…

Gael

Works okay with just mongrel i.e if I remove lighty from the picture.
Still, that kills my cluster of nice mongrel backend servers :slight_smile:

9

Pete,

I actually don’t track any of the REMOTE_USER, REMOTE_IDENT, or
REMOTE_HOST
parameters from CGI because the first two are security holes and all
three
are huge performance hits on Mongrel. Typically the last one is used by
web
servers to log requests, but since Mongrel is behind a faster real web
server logging or tracking this again is a double waste. Because this
(and
the fact that lighttpd hides the remote host) you get no remote host.

Also, I hope you realize that this is not a way to do a security check.
Remote IP addresses are easily faked.

Now, you have a real problem but there is a possible win-win solution.
Lighttpd supports mod_proxy (which you’re already using), but it also
lets
you pick backends according to the $HTTP[“remoteip”]. Rather than have
mongrel pick this, what you can do is setup one mongrel instance per
remote
site, and then have lighttpd proxy based on each using this variable.

So, if you had this before:

$HTTP[“host”] == “www.example.org” {
proxy.balance = “hash”
proxy.server = ( “” => ( ( “host” => “10.0.0.10” ),
( “host” => “10.0.0.11” ),
( “host” => “10.0.0.16” ),
( “host” => “10.0.0.17” ) ) )
}

You’d change it to:

$HTTP[“remoteip”] == “10.0.0.0/8” {
proxy.balance = “hash”
proxy.server = ( “” ( ( “host” => “127.0.0.1”, “port” => 9000 ) ) )
}

$HTTP[“remoteip”] == “10.0.1.0/8” {
proxy.balance = “hash”
proxy.server = ( “” ( ( “host” => “127.0.0.1”, “port” => 9001 ) ) )
}

And so on. The /8 is to pick a network rather than a single remoteip.

The docs for this are:

http://www.lighttpd.net/documentation/proxy.html
http://www.lighttpd.net/documentation/configuration.html

Two other options are:

  1. Give each remote site their own special DNS entry and then do what
    37signals does with basecamp to setup their access. There’s a wiki
    entry on
    this.
  2. Use the same $HTTP[“remoteip”] but use it to rewrite the request to
    pre-pend a “remote site id” to the request. Then change your routes.rb
    so
    that you got something like: /:remotesite/:controller/:action/:id.
    This
    then lets you avoid one backend per site but still gives you site
    specific
    stuff.
  3. Totally experimental, but look at the mod_setenv ability to
    setenv.request_header and see if you can just set some header to the
    remote
    IP. Ask in the #lighttpd IRC channel on irc.freenode.org.

Hope that helps, and good luck.

Zed A. Shaw

http://mongrel.rubyforge.org/

10

Hi,

I really appreciate the long reply. In this case it is reasonably secure
to use the remote IPs as each store (there are over a thousand) has a
specific IP and changing that IP would break a lot of stuff - plus they
don’t have acces to make those changes etc etc - but yes, I would never
use this to do a security check for an app on the Internet.

I will go through your examples and give them all a try and see which
works best.

Thanks again for all your help and kudos for mongrel - it is definately
my server of choice.

Cheers,

Pete

Zed S. wrote:

Pete,

I actually don’t track any of the REMOTE_USER, REMOTE_IDENT, or
REMOTE_HOST
parameters from CGI because the first two are security holes and all
three
are huge performance hits on Mongrel. Typically the last one is used by
web
servers to log requests, but since Mongrel is behind a faster real web
server logging or tracking this again is a double waste. Because this
(and
the fact that lighttpd hides the remote host) you get no remote host.

Also, I hope you realize that this is not a way to do a security check.
Remote IP addresses are easily faked.

Now, you have a real problem but there is a possible win-win solution.
Lighttpd supports mod_proxy (which you’re already using), but it also
lets
you pick backends according to the $HTTP[“remoteip”]. Rather than have
mongrel pick this, what you can do is setup one mongrel instance per
remote
site, and then have lighttpd proxy based on each using this variable.

So, if you had this before:

$HTTP[“host”] == “www.example.org” {
proxy.balance = “hash”
proxy.server = ( “” => ( ( “host” => “10.0.0.10” ),
( “host” => “10.0.0.11” ),
( “host” => “10.0.0.16” ),
( “host” => “10.0.0.17” ) ) )
}

You’d change it to:

$HTTP[“remoteip”] == “10.0.0.0/8” {
proxy.balance = “hash”
proxy.server = ( “” ( ( “host” => “127.0.0.1”, “port” => 9000 ) ) )
}

$HTTP[“remoteip”] == “10.0.1.0/8” {
proxy.balance = “hash”
proxy.server = ( “” ( ( “host” => “127.0.0.1”, “port” => 9001 ) ) )
}

And so on. The /8 is to pick a network rather than a single remoteip.

The docs for this are:

http://www.lighttpd.net/documentation/proxy.html
http://www.lighttpd.net/documentation/configuration.html

Two other options are:

  1. Give each remote site their own special DNS entry and then do what
    37signals does with basecamp to setup their access. There’s a wiki
    entry on
    this.
  2. Use the same $HTTP[“remoteip”] but use it to rewrite the request to
    pre-pend a “remote site id” to the request. Then change your routes.rb
    so
    that you got something like: /:remotesite/:controller/:action/:id.
    This
    then lets you avoid one backend per site but still gives you site
    specific
    stuff.
  3. Totally experimental, but look at the mod_setenv ability to
    setenv.request_header and see if you can just set some header to the
    remote
    IP. Ask in the #lighttpd IRC channel on irc.freenode.org.

Hope that helps, and good luck.

Zed A. Shaw
http://www.zedshaw.com/
http://mongrel.rubyforge.org/

11

“Pete” == Pete [email protected] writes:

Works okay with just mongrel i.e if I remove lighty from the picture.
Still, that kills my cluster of nice mongrel backend servers :slight_smile:

Have Lighty rewrite the IP into the URL and Rails extract it into a
parameter with a route?

	     Calle D. <[email protected]>
	 http://www.livejournal.com/users/cdybedahl/
 "I wish more lesbians were normal and valued firm boobs over
	intellectual development." -- babycola
12

Pete.

As I mentioned before I took REMOTE_ADDR since it was redundant and a
huge
performance hit. I’ll look at adding a flag so people can turn it on if
they need it.

But, in your case it wouldn’t help since you have lighttpdin front of
your
servers. What you’ll see is always 127.0.0.1. Take a look at this
patch
for lighttpd:

http://trac.lighttpd.net/trac/attachment/wiki/Release-1.4.10-patches/lighttp
d-1.4.10-mod_extforward.c

It seems that they are working on a way to pass this on, but that Apache
has
support for doing this right now. I’ll investigate this further and get
back to you on it.

Zed A. Shaw

http://mongrel.rubyforge.org/

13

Oddly everything worked great on Mongrel until this morning when I
restarted it with the new version - now it can’t see any IPs :slight_smile:

So I’m back with webrick atm. I’m hoping it will handle the load, but
it’s a shame I can’t use mongrel without updating the application and
unfortunately I don’t have time right now.

I appreciate that most people won’t even need this feature, but perhaps
it could be added to Mongrel as an option so that people who do need it,
can get to it?

Thanks all for your help :slight_smile:

Cheers,

Pete

Calle D. wrote:

“Pete” == Pete [email protected] writes:

Works okay with just mongrel i.e if I remove lighty from the picture.
Still, that kills my cluster of nice mongrel backend servers :slight_smile:

Have Lighty rewrite the IP into the URL and Rails extract it into a
parameter with a route?

       Calle D. <[email protected]>
   http://www.livejournal.com/users/cdybedahl/
 "I wish more lesbians were normal and valued firm boobs over
  intellectual development." -- babycola
14

Pete,

Simple solution to your problem. Change any code where you need to get
the
remote host to:

request.params[“HTTP_X_FORWARDED_FOR”]

And that should be the IP of the remote client.

Zed A. Shaw

http://mongrel.rubyforge.org/