WEBrick and Mongrel die on linux unless run as root

Rails
1

I’ve created a straightforward Rails app for a client and it runs great
on my FreeBSD 6 box. When we deploy it on his Redhat 7.2 server, both
WEBrick and Mongrel start and run fine, but will only stay up for a
couple of hours. We’ll find them dead with absolutely no information in
the logs. No core dumps. Nothing in /var/log/messages. Just gone. The
process will die even if idle, so I don’t think there’s anything in the
actual app that’s causing the problem.

In desperation, I started Mongrel as root and the darn thing is staying
up. I know zilch about Linux admin. Is there something I’m missing here?
There are no obvious per-user process limits (memory, CPU, etc.) as far
as I can tell.

2

On Fri, 2006-08-25 at 03:14 +0200, Bob S. wrote:

There are no obvious per-user process limits (memory, CPU, etc.) as far
as I can tell.

I’m not gonna even speculate but RHL 7.2 has been EOL for at least 4
years. Why not use CentOS 4 ?

You don’t say which version of ruby/rails you are using and I’m
presuming that you are avoiding apache which is bound to be really old
too.

Craig

3

Craig W. wrote:

On Fri, 2006-08-25 at 03:14 +0200, Bob S. wrote:

There are no obvious per-user process limits (memory, CPU, etc.) as far
as I can tell.

I’m not gonna even speculate but RHL 7.2 has been EOL for at least 4
years. Why not use CentOS 4 ?

Because a) I’m a programmer not a sysadmin, and b) I don’t have any idea
what RHL 7.2 and CentOS 4 even are, sadly :~) Heck, it may not even by
RHL 7.2 for all I know. It’s what cat /etc/redhat-release says, but who
knows if that’s right?

His box seems wacky to me. Non-root users can’t run ps, or who, or top.
You get an error that /proc must be mounted (?). Is this normal for
Linux?

You don’t say which version of ruby/rails you are using and I’m
presuming that you are avoiding apache which is bound to be really old
too.

I did a fresh build of Ruby 1.8.4 and am using Rails 1.1.2 (yes, I need
to upgrade to 1.1.6). However, this problem does not happen on FreeBSD,
and it only happens if I start the server as a non-privileged user.
Doesn’t happen on Win32 either.

Apache not playing a part. Just running WEBrick or mongrel_rails as the
server, port 3000, development environment. Just vanilla stuff.

I know I’m OT for the Rails list here, sorry…

4

Can the user that mongrel is running under access the log files? Read
and
write?

Sounds permissions related to me.

5

Bob S. wrote:

Because a) I’m a programmer not a sysadmin, and b) I don’t have any idea
what RHL 7.2 and CentOS 4 even are, sadly :~)

Wow. I would suggest HIRING a sysadmin then.

Joe

6

HH wrote:

Can the user that mongrel is running under access the log files? Read
and
write?

Yes, the application starts up fine and writes a pid file and log files.
You can use it with no problems, but after a couple of hours or so, it
just goes away. You can restart it and get a couple more hours just fine
and then poof.

7

Joe R. wrote:

Bob S. wrote:

Because a) I’m a programmer not a sysadmin, and b) I don’t have any idea
what RHL 7.2 and CentOS 4 even are, sadly :~)

Wow. I would suggest HIRING a sysadmin then.

But I was hoping to get free help from you :~) I just wanted to see if
it was an obvious Linux thing that anyone with an IQ higher than pond
slime would know but that I was overlooking in my shameful,
mind-boggling, blithering ignorance. It seems that’s not the case.

8

On Fri, 2006-08-25 at 04:20 +0200, Bob S. wrote:

RHL 7.2 for all I know. It’s what cat /etc/redhat-release says, but who
knows if that’s right?

I’d accept that it is correct.

RHL = Red Hat Linux

RHL versioned 7.2 => 7.3 => 8.0 => 9 and then became known as Fedora
(free)

Red Hat Enterprise Linux / vague comparisons
2.1 (similar to RHL 7.3) - now in maintenance mode, security updates
only
3.8 (similar to RHL 9)
4.4 (similar to Fedora Core 3)
version 5.0 scheduled for release December 2006

CentOS is freeware version of Red Hat Enterprise Linux
http://www.centos.org
CentOS has versions derived from above.

Fedora Core version 4 just went EOL and version 5 is current, version 6
is scheduled for release October 6, 2006

In essence, between Red Hat Linux and its successor, Fedora, Fedora Core
5 represents the 8th release since Red Hat Linux 7.2 - that’s like
trying to use Windows 95 (rev B)

His box seems wacky to me. Non-root users can’t run ps, or who, or top.
You get an error that /proc must be mounted (?). Is this normal for
Linux?

ls -l /bin/ps /usr/bin/top /usr/bin/who

-r-xr-xr-x 1 root root 66416 Dec 31 2005 /bin/ps
-r-xr-xr-x 1 root root 52348 Dec 31 2005 /usr/bin/top
-rwxr-xr-x 1 root root 23440 Aug 23 2005 /usr/bin/who

that’s on a CentOS 4 system. As you can see, all users can execute them.
No, it’s not normal for Linux. Was it normal for RHL 7.2? I don’t think
so but I don’t have any remaining systems running anything that old to
check.

You don’t say which version of ruby/rails you are using and I’m
presuming that you are avoiding apache which is bound to be really old
too.

I did a fresh build of Ruby 1.8.4 and am using Rails 1.1.2 (yes, I need
to upgrade to 1.1.6). However, this problem does not happen on FreeBSD,
and it only happens if I start the server as a non-privileged user.
Doesn’t happen on Win32 either.

on CentOS and Fedora systems, I run as user - never an issue.

Apache not playing a part. Just running WEBrick or mongrel_rails as the
server, port 3000, development environment. Just vanilla stuff.

I know I’m OT for the Rails list here, sorry…

we are talking about Rails - it’s on topic

Craig

9

Jeremy K. wrote:

On 8/24/06, Bob S. [email protected] wrote:

Because a) I’m a programmer not a sysadmin, and b) I don’t have any idea
what RHL 7.2 and CentOS 4 even are, sadly :~) Heck, it may not even by
RHL 7.2 for all I know. It’s what cat /etc/redhat-release says, but who
knows if that’s right?

His box seems wacky to me. Non-root users can’t run ps, or who, or top.
You get an error that /proc must be mounted (?). Is this normal for
Linux?

Sounds like you may be hitting a system limit and your processes are
being
killed.

Run ulimit -a as your user and as root to see the difference.

That’s what I thought, but here’s what I get:

[root@blah]# ulimit -a
core file size (blocks) 0
data seg size (kbytes) unlimited
file size (blocks) unlimited
max locked memory (kbytes) unlimited
max memory size (kbytes) unlimited
open files 1024
pipe size (512 bytes) 8
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 4082
virtual memory (kbytes) unlimited
[root@nimbus prop]# su - bob
[bob@blah]$ ulimit -a
core file size (blocks) 0
data seg size (kbytes) unlimited
file size (blocks) unlimited
max locked memory (kbytes) unlimited
max memory size (kbytes) unlimited
open files 1024
pipe size (512 bytes) 8
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 4082
virtual memory (kbytes) unlimited

I think there’s something on the box that’s killing of processes. I did
a ps and found three daemons running that are all encrypted Perl scripts
(the first line is “use Filter::decrtypt”, and the rest is gibberish):

monstd
syswatchd
healthd

These all live in /usr/prop/sbin. There’s also a bunch of cron jobs that
run stuff in /usr/prop/sbin.

After some googling, I figured out how to:

rpm -q --file /usr/prop/sbin/syswatchd

helix-syswatchd-ded72-1-1

Does this mean anything to anyone? I appreciate all the help.

10

On 8/24/06, Bob S. [email protected] wrote:

Because a) I’m a programmer not a sysadmin, and b) I don’t have any idea
what RHL 7.2 and CentOS 4 even are, sadly :~) Heck, it may not even by
RHL 7.2 for all I know. It’s what cat /etc/redhat-release says, but who
knows if that’s right?

His box seems wacky to me. Non-root users can’t run ps, or who, or top.
You get an error that /proc must be mounted (?). Is this normal for
Linux?

Sounds like you may be hitting a system limit and your processes are
being
killed.

Run ulimit -a as your user and as root to see the difference.

jeremy

11

On Fri, 2006-08-25 at 04:20 +0200, Bob S. wrote:

You get an error that /proc must be mounted (?). Is this normal for
Linux?

this might help…

$ grep proc /etc/fstab
proc /proc proc defaults 0 0

add that line to fstab (obviously as root), make sure that you
have /proc directory like

$ ls -ld /proc
dr-xr-xr-x 142 root root 0 Aug 23 03:02 /proc

reboot and see if that makes happy rails/mongrel

Craig

12

On Fri, 2006-08-25 at 05:25 +0200, Bob S. wrote:

core file size (blocks) 0
[root@nimbus prop]# su - bob
max user processes 4082
These all live in /usr/prop/sbin. There’s also a bunch of cron jobs that
run stuff in /usr/prop/sbin.

After some googling, I figured out how to:

rpm -q --file /usr/prop/sbin/syswatchd

helix-syswatchd-ded72-1-1

Does this mean anything to anyone? I appreciate all the help.

/usr/prop # not part of Red Hat packages - 3rd party software perhaps

Craig

13

On Fri, 2006-08-25 at 04:53 +0200, Bob S. wrote:

HH wrote:

Can the user that mongrel is running under access the log files? Read
and
write?

Yes, the application starts up fine and writes a pid file and log files.
You can use it with no problems, but after a couple of hours or so, it
just goes away. You can restart it and get a couple more hours just fine
and then poof.

I suppose then you create a cron job that runs a script that restarts
mongrel process every hour or so - otherwise, get a system admin.

Craig

14

Craig W. wrote:

I suppose then you create a cron job that runs a script that restarts
mongrel process every hour or so - otherwise, get a system admin.

Thanks, looks like we’ll do the latter. I appreciate all the help.