NAME
rq v2.3.3
SYNOPSIS
rq (queue | export RQ_Q=q) mode [mode_args]* [options]*
URIS
http://codeforpeople.com/lib/ruby/rq/
http://raa.ruby-lang.org/project/rq/
Linux Clustering with Ruby Queue: Small Is Beautiful | Linux Journal
DESCRIPTION
ruby queue (rq) is a zero-admin zero-configuration tool used to
create
instant unix clusters. rq requires only a central nfs filesystem in
order
to manage a simple sqlite database as a distributed priority work
queue.
this simple design allows researchers to install and configure, in
only a
few minutes and without root privileges, a robust unix cluster
capable of
distributing processes to many nodes - bringing dozens of powerful
cpus to
their knees with a single blow. clearly this software should be kept
out of
the hands of free radicals, seti enthusiasts, and one mr. j safran.
the central concept of rq is that n nodes work in isolation to pull
jobs
from an centrally mounted nfs priority work queue in a synchronized
fashion.
the nodes have absolutely no knowledge of each other and all
communication
is done via the queue meaning that, so long as the queue is available
via
nfs and a single node is running jobs from it, the system will
continue to
process jobs. there is no centralized process whatsoever - all nodes
work
to take jobs from the queue and run them as fast as possible. this
creates
a system which load balances automatically and is robust in face of
node
failures.
although the rq system is simple in it’s design it features powerful
functionality such as priority management, predicate and sql query ,
compact
streaming command-line processing, programmable api, hot-backup, and
input/capture of the stdin/stdout/stderr io streams of remote jobs.
to date
rq has had no reported runtime failures and is in operation at dozens
of
research centers around the world.
HISTORY
2.3.3:
- fixed bug in updater related to io files (see tmp_stdin in
jobqueue.rb)
- [IMPORTANT] removed feature where stdin jobs could contain
comments (‘#’).
input lines are now taken literally except blank lines, which
are
still ignored.
- added ability to dump stdin/stdout/stderr for any job
rq q stdout 42
cat jids | rq q stdout -
- added ability to dump stdin/stdout/stderr pathnames for any job
rq q stdout4 42
cat jids | rq q stderr4 -
- rotate was made more robust and tolerates nfs errors to some
degree
enjoy.
-a