Scheduled tasks

Rails
1

Hi

What is the best way to go about creating a service that has access to
my rails environment, but that runs on its own?

My specific problem is creating a process that runs in the background
and updates certain models every 5 minutes. Is there a standard way of
creating something like this?

Thanks
Ryan

2

On 10 Mar 2008, at 22:00, Ryan wrote:

What is the best way to go about creating a service that has access to
my rails environment, but that runs on its own?

My specific problem is creating a process that runs in the background
and updates certain models every 5 minutes. Is there a standard way of
creating something like this?

  • cron tab and script/runner
  • backgroundrb’s scheduler
  • small ruby daemon with loop with sleep

You can choose what suits you best.

Best regards

Peter De Berdt

3

I’ll chime in a small variant of Peter’s suggestions. If you have a
ruby or other program running, you might want to run it from inittab
to keep it running no matter what.

I vote for crontab calling the shots and calling script/runner.

H

4

Same here, crontabs would be my personal favorite too. It takes a few
seconds to start up the rails process, but at least it releases memory
again afterwards. Backgroundrb being a persistent process leaked
memory for me, which lead to server crashes over time (earlier
versions, haven’t tried the last one).

On 11 Mar 2008, at 17:20, blasterpal wrote:

  • cron tab and script/runner
  • backgroundrb’s scheduler
  • small ruby daemon with loop with sleep

Best regards

Peter De Berdt

5

Thanks for the help.

What if I wanted it to constantly be working. For example, monitoring
a database table for jobs to process?

6

On 11 Mar 2008, at 22:08, Ryan wrote:

Thanks for the help.

What if I wanted it to constantly be working. For example, monitoring
a database table for jobs to process?

Depending on what you mean exactly, you could use triggers in the
database (if the processing can happen there)
http://dev.mysql.com/doc/refman/5.0/en/triggers.html

If you’re talking about your rails app saving something to the db and
you want to process it without locking a mongrel or the user, you
could use backgroundrb, starling or some other asynchronous processor
out there. There’s a chapter devoted to such setups in the excellent
Advanced Rails Recipes book.

Best regards

Peter De Berdt

7

Has anybody used rufus-scheduler? Opinions? I just downloaded it and
am trying to get it to work. I’m somewhat new to rails so the fix is
not obvious.

I need to run a task every 5 minutes that gathers data and adds it to my
database, so the periodic task needs access to ActiveRecord to save
objects.

8

Steven L. wrote:

Has anybody used rufus-scheduler? Opinions? I just downloaded it and
am trying to get it to work. I’m somewhat new to rails so the fix is
not obvious.

I need to run a task every 5 minutes that gathers data and adds it to my
database, so the periodic task needs access to ActiveRecord to save
objects.

If anybody is interested, here are the errors. It doesn’t seem to be
finding an ActiveRecord object it’s trying to access:

running incremental_scan:
trigger() caught exception
A copy of QueryController has been removed from the module tree but is
still active!
/usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:237:in
load_missing_constant' /usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:469:inconst_missing’
/home/adminline/cvswatch/app/controllers/query_controller.rb:43:in
`incremental_scan’

The QueryController contains a method called incremental_scan.
Incremental_scan is configured to run every 5 minutes. However when the
scheduler calls it, it throws an exception when it tries to do a
.find(1) from an ActiveRecord object.

9

Ok, I got it to work using script/runner but first I had to move my
method from my controller to a model and make the methods static.

Thanks.

Steve

10

On Tue, 2008-03-11 at 19:12 +0100, Peter De Berdt wrote:

Same here, crontabs would be my personal favorite too. It takes a few
seconds to start up the rails process, but at least it releases memory
again afterwards. Backgroundrb being a persistent process leaked
memory for me, which lead to server crashes over time (earlier
versions, haven’t tried the last one).
te

Hmm, I doubt that BackgrounDRb itself leaked memory. However with newer
version there is an option to keep scheduled workers not persistent. As
in, your worker will be stared afresh each time schedule is arrived.
Just define your usual configuration file and in worker:

class FooWorker < BackgrounDRb::MetaWorker
set_worker_name :foo_worker
reload_on_schedule true
end

IMHO BackgrounDRb has advantage that you can easily monitor status of
your workers. You wouldn’t need a separate crontab entry and since
everything is under RAILS_ROOT you can manage it with Capistrano or
Vlad.

11

Steven L. wrote:

Has anybody used rufus-scheduler? Opinions? I just downloaded it and
am trying to get it to work. I’m somewhat new to rails so the fix is
not obvious.

I need to run a task every 5 minutes that gathers data and adds it to my
database, so the periodic task needs access to ActiveRecord to save
objects.

Yes, I am just in the process of using rufus-scheduler.

To use ActiveRecord models from your rails application you need to bring
in the rails environment. Put your script that uses rufus-scheduler in
your rails root application directory. Near the top of my little script:

rails_root = File.expand_path(File.join(File.dirname(FILE)))
RAILS_HOME = rails_root

require RAILS_HOME + ‘/config/environment’

Then you can use schedules like:

scheduler.schedule_every(“1m00s”) do
puts Time.now.to_s + “…popping up every 1 minute…”
Session.delete_all(“updated_at < now()-‘20 minutes’::interval”)
end

Does this help?

Cheers,
Gary.

12

On 12 Mar 2008, at 01:52, hemant kumar wrote:

Hmm, I doubt that BackgrounDRb itself leaked memory.

Well, I said I didn’t use the latest version yet, and if I look in the
changelog, it clearly says:

2008-02-28 hemant kumar hemant@shire

  • fixed some meory leaks.

My problems had to do with exporting records to csv (with lots of
calculations being done on them before exporting). I tested the same
report generator from a normal Rails process and it was stable in
memory use, handing it over to BackgroundRB caused the server to run
out of memory in a matter of days (after it had been running without a
hitch for over a year).

That said, I believe backgroundrb is a good solution and the
advantages the other posters claim are indeed true. And given that
it’s easy to integrate and if it fails for you, just as easy to take
out again and replace with something else, I see no reason not to try
it (I might well give it another shot in the future :-))

Best regards

Peter De Berdt

13

On Wed, 2008-03-12 at 11:00 +0100, Peter De Berdt wrote:

  • fixed some meory leaks.

My problems had to do with exporting records to csv (with lots of
calculations being done on them before exporting). I tested the same
report generator from a normal Rails process and it was stable in
memory use, handing it over to BackgroundRB caused the server to run
out of memory in a matter of days (after it had been running without a
hitch for over a year).

Gosh, I feel naked. I dunno if you were affected by those bugs. But
anyways, since a fix was posted about 20 days back(and was in git repo
for much longer time), I think its okay.

That said, I believe backgroundrb is a good solution and the
advantages the other posters claim are indeed true. And given that
it’s easy to integrate and if it fails for you, just as easy to take
out again and replace with something else, I see no reason not to try
it (I might well give it another shot in the future :-))

Yup, fully agree. Choose what works for you. Heck, I will dump bdrb in a
zip if it didn’t work for me. I just wanted to draw your attention to
the facility of reloading workers on schedule, so as they are not
persistent and gets loaded only on scheduled time.

14

Everywhere I look, I see people saying negative things about
backgroundrb, about how it is unstable and is a memory leak or adds a
bunch of hassles. I assume this must have been true some time in the
past (perhaps when backgroundrb was actually using drb).

However, I have been using backgroundrb (with scheduling) [latest
version] and it seems stable, easy to use and very low memory. I think
Hemant must have made some radical revisions because nothing bad I hear
about it rings true anymore as far as I know. For one it doesn’t even
use drb.

I would really recommend backgroundrb as it is easy to control through
capistrano and monitor using monit. Also it has full access to the rails
environment, models, etc.

Just my two cents.

15

On Wed, 2008-03-12 at 10:10 +0100, Nathan E. wrote:

I would really recommend backgroundrb as it is easy to control through
capistrano and monitor using monit. Also it has full access to the rails
environment, models, etc.

There were/are couple of problems around bdrb.

  1. Google search about BackgrounDRb refers to documentation/tutorials
    which are very old.
  2. There have been too many bleeding versions and hence the confusion.
  3. I am a lousy and lazy programmer.

We are improving things bit by it and hope its usable for everyone as
its usable for you and me.

16

Gary D. wrote:

To use ActiveRecord models from your rails application you need to bring
in the rails environment. Put your script that uses rufus-scheduler in
your rails root application directory. Near the top of my little script:

rails_root = File.expand_path(File.join(File.dirname(FILE)))
RAILS_HOME = rails_root

require RAILS_HOME + ‘/config/environment’

Then you can use schedules like:

scheduler.schedule_every(“1m00s”) do
puts Time.now.to_s + “…popping up every 1 minute…”
Session.delete_all(“updated_at < now()-‘20 minutes’::interval”)
end

Does this help?

Yes, I learned several things by reading this. Thanks!

Steve