Memory limits on TextDrive

Hey folks-

I’ve recently been running up against the memory limits on TextDrive for
a Ruby on Rails app under development.

It’s running on FCGI/lighttpd with 2 listeners. I can restart it and
within a few clicks on random pages I can get it to fail for memory
allocation problems.

While I haven’t written off the idea of a memory leak, I have been
unable to reproduce the problem on my development machine. I can run
mongrel or webrick and don’t see memory usage go up after a LOT of
browsing around and hitting it through the webservices API.

I think my app just gets me close to the 48MB limit. So, my questions:

  1. Is the TxD memory limit too low? My app isn’t enormous, but it is
    bigger than a simple blog or other basic Rails app like Strongspace or
    something. I’ve seen memory usage posts from other folks that put their
    app in the 40MB-70MB range. If this is the case, and TxD claims to be
    the premier RoR host, why is their limit so low?

  2. What techniques can I use to reduce memory usage?

  3. Would another server technology be easier on memory?

  4. I really like TxD, but they claim they can’t increase memory
    allotment for me (even just to TEST the damn thing for a day). So, are
    there other hosts that would be more amenable to this app?

Business hosting is a possibility (at TxD) and it will be necessary once
the app is released, but before I go through the trouble of moving the
app, I wanted to test some stuff out.

Jake

Jake J. wrote:

Hey folks-

I’ve recently been running up against the memory limits on TextDrive for
a Ruby on Rails app under development.

I thought I’d mention – this has recently been a problem because my app
just got bigger. But when I migrated to Edge Rails, it got even worse.
I wonder if there will be even more such problems when 1.2 is released
and people’s code just gets bigger.

Jake

On 10/30/06, Jake J. [email protected] wrote:

Hey folks-

I’ve recently been running up against the memory limits on TextDrive for
a Ruby on Rails app under development.

It’s running on FCGI/lighttpd with 2 listeners. I can restart it and
within a few clicks on random pages I can get it to fail for memory
allocation problems.

Why do you need two listeners? Most apps would not.


_Deirdre http://deirdre.net/

I am also running up against the memory limits of Text Drive but only
since I upgraded my app to use edge rails. Are you running on edge?
If so, I would recommend running on 1.1.6 and see how that goes. I
never had a problem with memory, except for very occasionally on
TextDrive.

Since I upgraded to edge, I am seeing memory spike up to 3.5 MBs on a
single request that only grabs 10 records out of 139 or them
(something about the total impacts the size). I am in the process of
downgrading my app back to 1.1.6 and will post my results to the list
if it fixes my memory issues. In theory, this could be something that
I added in this release but from my initial analysis it does not
appear to be.

Also, a lot of people say you don’t need more than one fcgi and while
in theory I would agree, when you are on a server like TextDrive that
will kill your fcgis if they hit system resources the second fcgi is
a good idea so your site is not immediately dead. You have two
chances.

Tom

On 10/30/06, Deirdre Saoirse M. [email protected] wrote:

allocation problems.

Why do you need two listeners? Most apps would not.


_Deirdre http://deirdre.net/


Tom D.

http://atomgiant.com
http://gifthat.com

Tom D. wrote:

I am also running up against the memory limits of Text Drive but only
since I upgraded my app to use edge rails. Are you running on edge?
If so, I would recommend running on 1.1.6 and see how that goes. I
never had a problem with memory, except for very occasionally on
TextDrive.

Since I upgraded to edge, I am seeing memory spike up to 3.5 MBs on a
single request that only grabs 10 records out of 139 or them
(something about the total impacts the size). I am in the process of
downgrading my app back to 1.1.6 and will post my results to the list
if it fixes my memory issues. In theory, this could be something that
I added in this release but from my initial analysis it does not
appear to be.

Also, a lot of people say you don’t need more than one fcgi and while
in theory I would agree, when you are on a server like TextDrive that
will kill your fcgis if they hit system resources the second fcgi is
a good idea so your site is not immediately dead. You have two
chances.

Tom

Tom–

Yes, the memory limit came into play when I upgraded to edge. I
definitely think edge has higher memory requirements. Unfortunately,
it’s something I needed to do, so couldn’t be avoided.

We moved to a container on TxD and that solved the memory problems.
It’s also more expensive, but reasonable for this app. The increased
speed is a huge benefit.

I think I just recognize the limitations of shared hosting. The move to
edge just accelerated the need to switch off of it, but that was
inevitable. Perhaps TxD will increase their memory limits when 1.2
releases.

Jake

Deirdre Saoirse M. wrote:

On 10/30/06, Jake J. [email protected] wrote:

Hey folks-

I’ve recently been running up against the memory limits on TextDrive for
a Ruby on Rails app under development.

It’s running on FCGI/lighttpd with 2 listeners. I can restart it and
within a few clicks on random pages I can get it to fail for memory
allocation problems.

Why do you need two listeners? Most apps would not.

That point is irrelevant actually. Each process is limited to 48 MB on
TxD. I don’t know what their limit it on the sum.

Jake

Yeah, from my limited testing, I witnessed edge rails requiring in the
neighborhood of 6-8MBs more per fcgi. That is a lot to jump from one
point release to another so I suspect it was an accident.

I posted another email to the list to see if anyone has any feedback
on it. I would hope they can bring the memory requirements down
closer to the 1.1.6 level.

Tom

On 11/7/06, Jake J. [email protected] wrote:

(something about the total impacts the size). I am in the process of

speed is a huge benefit.


Tom D.

http://atomgiant.com
http://gifthat.com

I have a shared TD account.

How do I see my memory consumption?

Is it the Size column listed in the output from the top command?

last pid: 87116; load averages: 2.60, 3.94, 5.15
up 83+06:27:38 22:00:11
17 processes: 1 running, 16 sleeping
CPU states: 11.8% user, 4.2% nice, 17.5% system, 0.4% interrupt,
66.1%idle
Mem: 2976M Active, 391M Inact, 410M Wired, 201M Cache, 112M Buf, 13M
Free
Swap: 12G Total, 5239M Used, 7049M Free, 42% Inuse, 320K Out

PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU
COMMAND
28834 bikle10 4 10 42452K 8K select 2 0:06 0.00%
0.00%ruby
28832 bikle10 4 10 4144K 776K select 2 0:03 0.00%
0.00%lighttpd

-Dan

On 11/7/06, Tom D. [email protected] wrote:

single request that only grabs 10 records out of 139 or them
chances.
It’s also more expensive, but reasonable for this app. The increased
Posted via http://www.ruby-forum.com/.


[email protected]
http://bikle.com

Hi Dan,

Yeah top should work. You can also do “ps auxww | grep dispatch”.

The size is the virtual memory size. The RES is the resident memory.
Virtual memory is limited to 80MB on TextDrive. Resident memory is
limited to 48MB. In my case, I was hitting the resident memory
limits.

Tom

On 11/7/06, Dan B. [email protected] wrote:

CPU states: 11.8% user, 4.2% nice, 17.5% system, 0.4% interrupt, 66.1%

on it. I would hope they can bring the memory requirements down

If so, I would recommend running on 1.1.6 and see how that goes. I

Yes, the memory limit came into play when I upgraded to edge. I
releases.

Tom D.

http://atomgiant.com
http://gifthat.com


Tom D.

http://atomgiant.com
http://gifthat.com

Nope, TextDrive no longer allows you to use ‘top’. I’ve run into the
same problem with them - hitting the 48M limit on my production app.
After trying in vain to get the memory consumption down and seeing my
site bouncing like a yoyo, I’m getting a VPS at Slicehost. I’ll setup
Mongrel and be much happier, I think.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs