#17446 [PATCH] Add option to mongrel_rails to force mongrel not to serve static files

saimon · January 24, 2008, 1:14pm

Hi all,

I’ve just added a patch that I’d appreciate some feedback on:

http://rubyforge.org/tracker/index.php?func=detail&aid=17446&group_id=1306&atid=5147

Regards,

Saimon

Saimon M.
Freelance Web D.
(Available for hire - For details visit http://saimonmoore.net)

Skype: saimonmoore
Yahoo IM: saimonmoore
Google IM: saimonmoore

saimon · January 24, 2008, 4:43pm

On Jan 24, 2008, at 6:13 AM, Saimon M. wrote:

Hi all,

I’ve just added a patch that I’d appreciate some feedback on:

http://rubyforge.org/tracker/index.php?
func=detail&aid=17446&group_id=1306&atid=5147

I’ll admit by cold-addled brain isn’t following logic perfectly this
morning, but I feel like this really would be better handled by the
upstream HTTP handler giving enough information to your app so it
serves the right cached file (or runs the right controller, or
whatever).

I think passing the Host: header along with the request_routing
plugin (assuming you’re on Rails) will make everything happy:

http://agilewebdevelopment.com/plugins/request_routing

Or am I missing something?

-Nate

saimon · January 25, 2008, 3:29am

Hi Nathan,

The problem is that mongrel is serving the static file long before
rails knows anything about it.

e.g. As I mention in the patch, you have a web server setup with
multiple virtual hosts each mapped to a separate doc root.

example.com => /var/www/apps/example.com/public

es.example.com => /var/www/apps/example.com/public/es

Rails is using page caching and the cache is clean.

1st request:

http://example.com =>

web server looks for /var/www/apps/example.com/public/index.html
passes request to mongrel which does the same thing, doesn’t find
it and calls rails to render it.
Now /var/www/apps/example.com/public/index.html exists as was
cached by rails.

2nd request:

http://es.example.com =>

web server looks for /var/www/apps/example.com/public/es/index.html
passes request to mongrel which looks for /var/www/apps/example.com/
public/index.html
(which unlike the web server, has only one doc root),
finds the file under /var/www/apps/example.com/public/index.html
(from the previous request) and serves that back before rails has even
got a chance to know about it. It has served the wrong file.

I hope this example makes it clearer.

With the ability to force mongrel to forward all requests it receives
to rails, on the second request rails (using the host info) would then
cache the index.html file into /var/www/apps/example.com/public/es/
index.html.

As I was writing this, I just thought of another possibility being
writing a custom mongrel handler, but as mongrel_rails is a handler
itself, it seems a bit pointless so I still think my patch has merits.

Thoughts?

Regards,

Saimon

On Jan 24, 2008, at 4:42 PM, Nathan V. wrote:

morning, but I feel like this really would be better handled by the

-Nate

Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users

–
Saimon M.
Freelance Web D.
(Available for hire - For details visit http://saimonmoore.net)

Skype: saimonmoore
Yahoo IM: saimonmoore
Google IM: saimonmoore

saimon · January 25, 2008, 3:40pm

SO, it sounds like you have Apache (or another web server) in front of a
pack of Mongrels, which should be serving static files. It sounds like
your front-end web server is passing requests back to mongrel when this
is not your intent. If so, it sounds like you have a misconfiguration in
your front-end web server configuration.

Is this the case?

==
Will G.

Saimon M. wrote …

saimon · January 25, 2008, 4:44pm

Will,

Yes it’s the case that a front-end web server (nginx) is sitting in
front of the mongrels but no it’s not the case that it is misconfigured.

As I hoped my example was illustrating, the web server knows about
different virtual hosts but mongrel doesn’t.

Let me try and give another example with even more detail:

Assume:

Mongrel (without patch), which serves any static files it finds in
it’s doc root (RAILS_ROOT/public)
Web server (WS) virtual host mapping:
example.com => /var/www/apps/example.com/public (web
server’s doc root for virtual host 1)
es.example.com => /var/www/apps/example.com/public/es (web
server’s doc root for virtual host 2)

directory structure (DS) when cache is cleared:

RAILS_ROOT/public
RAILS_ROOT/public/es

Steps on 1st request: http://example.com

WS: can’t find index.html in /var/www/apps/example.com/public
WS forwards request to mongrel process (M)
M can’t find index.html in /var/www/apps/example.com/public
Mongrel creates a rails process to handle request for /
Rails, sees example.com host and / path and renders index.html in /
var/www/apps/example.com/public
Mongrel serves /var/www/apps/example.com/public/index.html back

There’s no problem at this point. But bear with me…

DS is now:

RAILS_ROOT/public
RAILS_ROOT/public/index.html
RAILS_ROOT/public/es

Steps on 2nd request: http://example.com

WS: finds index.html in /var/www/apps/example.com/public
WS serves request

Again, no problem…just served up previously cached file from same
domain.

DS is still:

RAILS_ROOT/public
RAILS_ROOT/public/index.html
RAILS_ROOT/public/es

Steps on 3rd request: http://es.example.com

WS: can’t find index.html in /var/www/apps/example.com/public/es <=
Note the doc root is different
WS forwards request to mongrel process (M)
M finds index.html in /var/www/apps/example.com/public
Mongrel serves /var/www/apps/example.com/public/index.html which is
the page associated with the http://example.com domain.
This is NOT the required behaviour as it’s serving the cached file
for http://example.com rather than http://es.example.com as was
requested.

DS is still:

RAILS_ROOT/public
RAILS_ROOT/public/index.html
RAILS_ROOT/public/es

Alternative 3rd request (http://es.example.com) with * patched mongrel

(mongrel set NOT to serve any static files):

WS: can’t find index.html in /var/www/apps/example.com/public/es
WS forwards request to mongrel process (M)
M ignores index.html in /var/www/apps/example.com/public as it
directly forwards request to rails
Mongrel creates a rails process to handle request
Rails, sees es.example.com host and / path and renders index.html
in /var/www/apps/example.com/public/es
Mongrel serves /var/www/apps/example.com/public/es/index.html back

DS is now:

RAILS_ROOT/public
RAILS_ROOT/public/index.html
RAILS_ROOT/public/es
RAILS_ROOT/public/es/index.html

Note: Now the right page has been served for the right domain. After
the file has been cached for http://es.example.com, the WS will
happily serve that up.

Phew, I hope this finally illustrates the problem with more clarity.

Regards,

Saimon

On Jan 25, 2008, at 3:32 PM, Will G. wrote:

it and calls rails to render it.
example.com/
to rails, on the second request rails (using the host info) would

[email protected]

http://rubyforge.org/mailman/listinfo/mongrel-users
–
Saimon M.
Freelance Web D.
(Available for hire - For details visit http://saimonmoore.net)

Skype: saimonmoore
Yahoo IM: saimonmoore
Google IM: saimonmoore

saimon · January 25, 2008, 5:10pm

Saimon,

I think you were clear enough the first time, and I don’t think anyone
can
have missed your point now.

I think your patch is a good idea, and not only because it solves your
problem. As Zed has pointed out very succinctly a few times, having the
app
container serve files is a bad, bad, bad idea. I daresay that most of
us
don’t let it do that, or at least we intend to not let it do that, but
we
might have made a mistake somewhere. Now, if your patch was accepted, I
would certainly upgrade my production mongrel configurations to say “do
not
serve files”, and if I had made a mistake somewhere, I would find out
soon
enough. Fail early, as the Prags would say.

That being said, I haven’t checked your actual implementation, so I
cannot
vouch for it (as if I could anyway). But I think it should be (possibly
rewritten and) included in trunk.

Also, there may be another solution to your problem, but I think it’s
silly:
instead of having the root for example.com be /public, make it
/public/com.
Then make your app always use a sub-directory for the generated files.
I
bet it would work, but I like that patch approach better.

Best regards,

David

saimon · January 25, 2008, 5:12pm

Yep,you got it.

Since I sent in the ticket I had been thinking about the possibility
to write a custom mongrel handler (And in fact I’ll probably do it)
but I still think that this patch is of value. If only, to avoid
people having to write their own custom handlers for each use case
(though I can’t think of any others right now).

Also mongrel_rails is a custom mongrel handler of sorts, so I guess I
see it as just a patch on that.

But I can live with writing a custon mongrel handler, I just thought
this tiny patch may come in handy (and save them some time) for others
when confronted with the same problem.

Regards,

Saimon

On Jan 25, 2008, at 4:53 PM, Will G. wrote:

No need to patch Mongrel directly, just extend the existing

As I hoped my example was illustrating, the web server knows about

Rails, sees example.com host and / path and renders index.html
RAILS_ROOT/public/es
DS is still:
Note the doc root is different

WS forwards request to mongrel process (M)
RAILS_ROOT/public/index.html

a misconfiguration in your front-end web server configuration.

rails knows anything about it.
1st request:
2nd request:
(from the previous request) and serves that back before rails has
index.html.
Saimon

http://rubyforge.org/tracker/index.php?
I think passing the Host: header along with the request_routing
http://rubyforge.org/mailman/listinfo/mongrel-users

Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users

Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users

–
Saimon M.
Freelance Web D.
(Available for hire - For details visit http://saimonmoore.net)

Skype: saimonmoore
Yahoo IM: saimonmoore
Google IM: saimonmoore

saimon · January 25, 2008, 5:40pm

On Jan 25, 2008, at 9:34 AM, Saimon M. wrote:

requested.
Wait… my brain is still missing something; I apologize.

Why isn’t nginx sending the stuff from es.example.com to
mongrel.example.com/es? You don’t need to send your proxied
requests to mongrel’s root… do you?

I guess you’d need to override the behavior of Rails’s url_for() or
generate routes asymmetrically, which would suck.

My gut (as a non-devteam-member) is that you’re doing something a
little strange, so a custom handler is more sane than patching a
switch onto Mongrel.

A more interesting proposal would be to say “Mongrel is ONLY an app
server and doesn’t serve static files AT ALL” – to make it do so
would require a custom handler. That, however, would be a Big Scary
Change ™ and I think users would be… unhappy.

-Nate

saimon · January 25, 2008, 4:58pm

Ah, so you have Rails generating a static file.

It looks like you have the same Rails app mounted at two locations. If
so, you need to have a way for Nginx to tell Mongrel “hey, this is the
same app, but now I need it in Spanish”.

Investigate having Nginx pass an environment variable (HTTP header) back
to Mongrel. Then create a custom Mongrel handler that searches for the
correct index.html file based on that header.

No need to patch Mongrel directly, just extend the existing DirHandler
(or write a new one based on the existing one), and register it with
Mongrel.

Saimon M. wrote …

saimon · January 26, 2008, 12:00pm

At 08:09 AM 1/25/2008, [email protected] wrote:

to write a custom mongrel handler (And in fact I’ll probably do it)
others
when confronted with the same problem.

Regards,

Saimon

Hi Saimon,

Maybe I’m totally missing the boat here, but couldn’t you solve this on
the Rails end instead? David V. approached this point in his reply
but maybe I can extend it a bit.

Couldn’t you configure your Rails app to become “cache-folder-specific”
depending on your language locale? So at the top of your Rails app you
detect what language you’re in (based on the incoming site, the route
or whatever else) and you set your root caching folder appropriately
from within Rails? So instead of having a default page cache be written
into:

[rails_root]/public

and a spanish cache written to

[rails_root]/public/es

Why not move the default page cache to (assuming en is default):

[rails_root]/public/en

That way, Mongrel will never find “index.html” in your public root and
so will just not interfere? You just program nginx to look in
“[rails_root]/public/en” for your default “www.site.com” hosts and to
“…/public/es” for “es.site.com” - etc…

Am I missing something? I usually am. I hope this helps anyway.

Steve

p.s. I don’t think your ideas for a Mongrel patch are wrong in any way,
but I’m just wondering if you can solve this with nginx.conf and Rails
and leave the Mongrel container out of it completely?

saimon · January 26, 2008, 11:59am

At 08:09 AM 1/25/2008, [email protected] wrote:

to write a custom mongrel handler (And in fact I’ll probably do it)
others
when confronted with the same problem.

Regards,

Saimon

Hi Saimon,

Maybe I’m totally missing the boat here, but couldn’t you solve this on
the Rails end instead? David V. approached this point in his reply
but maybe I can extend it a bit.

Couldn’t you configure your Rails app to become “cache-folder-specific”
depending on your language locale? So at the top of your Rails app you
detect what language you’re in (based on the incoming site, the route
or whatever else) and you set your root caching folder appropriately
from within Rails? So instead of having a default page cache be written
into:

[rails_root]/public

and a spanish cache written to

[rails_root]/public/es

Why not move the default page cache to (assuming en is default):

[rails_root]/public/en

That way, Mongrel will never find “index.html” in your public root and
so will just not interfere? You just program nginx to look in
“[rails_root]/public/en” for your default “www.site.com” hosts and to
“…/public/es” for “es.site.com” - etc…

Am I missing something? I usually am. I hope this helps anyway.

Steve

p.s. I don’t think your ideas for a Mongrel patch are wrong in any way,
but I’m just wondering if you can solve this with nginx.conf and Rails
and leave the Mongrel container out of it completely?

saimon · January 29, 2008, 1:17pm

Yep Steve,

You’re absolutely right. I certainly can do it like that. I’m just
dumbfounded why I didn’t ‘see’ this obviously easier path.

So much for my ‘reason’

In any case, I think David V. brought up a very valid point about
mongrel not serving static files at all.

Thanks everyone for your feedback.

Regards,

Saimon

On Jan 25, 2008, at 6:42 PM, Steve M. wrote:

Since I sent in the ticket I had been thinking about the possibility
this tiny patch may come in handy (and save them some time) for
on
into:

way,
but I’m just wondering if you can solve this with nginx.conf and Rails
and leave the Mongrel container out of it completely?

Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users

–
Saimon M.
Freelance Web D.
(Available for hire - For details visit http://saimonmoore.net)

Skype: saimonmoore
Yahoo IM: saimonmoore
Google IM: saimonmoore