Forum: Ruby on Rails [ANN] RailsCron 0.2 plugin

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Kyle Maxwell (Guest)
on 2006-01-25 21:29
(Received via mailing list)
I received feedback from some of you, saying that it would be cool if
RailsCron was even easier to manage.  So I implemented a graceful
start/restart inside of the plugin's init.rb.  If you don't like it,
comment it out and send me the feedback.

Also, I finished an update to RailsCron that allows you to attach
asynchronous processing methods to your ActiveRecord models. i.e.:

class EmailQueue < ActiveRecord::Base
	background :deliver, :every => 10 #seconds, or 1.minute, etc

	def self.deliver
		#process the queue
	end
end

If your deliver method takes a minute to process (like some of my
async methods do), then RailsCron will NOT load up multiple threads of
the same command every 10 seconds.  Rather RailsCron checks to see if
the command is currently running, and doesn't start a new one.  You
can overrride this with:

RailsCron.options[:overlap] = true

Is there a better name to use than overlap?  Anyhow, have fun, and
feedback is always welcome and encouraged!

Repository:
http://svn.kylemaxwell.com/rails_cron/

Bug Tracker:
http://dev.kylemaxwell.com

--
Kyle Maxwell
Chief Technologist
E Factor Media // FN Interactive
kyle@efactormedia.com
1-866-263-3261
Ezra Zygmuntowicz (Guest)
on 2006-01-25 21:35
(Received via mailing list)
Kyle-

	This is totally sweet. I am using it to implement some new features
on http://yakimaherald.com. i will give you feedback and maybe a
patch or two once I get it worked out.

Thanks
-Ezra

On Jan 25, 2006, at 12:26 PM, Kyle Maxwell wrote:

>
>
>
> --
> Kyle Maxwell
> Chief Technologist
> E Factor Media // FN Interactive
> kyle@efactormedia.com
> 1-866-263-3261
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails

-Ezra Zygmuntowicz
Yakima Herald-Republic
WebMaster
http://yakimaherald.com
509-577-7732
ezra@yakima-herald.com
Ben Reubenstien (Guest)
on 2006-01-25 21:38
(Received via mailing list)
Hi Kyle ~

This looks like it might solve several items that I need to automate.  I
will be sure to check it out and send you some feedback.  I usually
would
set up a cron job on a linux box to execute the action I needed.

Thx!

~ Ben
Jeff Blasius (Guest)
on 2006-01-26 04:43
(Received via mailing list)
Hello Kyle,
I know this is the plugin I need. Can you maybe give me a simple
example of what I might need in the model and controller? What I'm
trying to do is process then purge emails recorded in a table.
Thank You,
           jeff
Tim Lucas (Guest)
on 2006-01-26 05:13
(Received via mailing list)
Awesome awesome work Kyle!

On 26/01/2006, at 7:26 AM, Kyle Maxwell wrote:

> RailsCron.options[:overlap] = true
>
> Is there a better name to use than overlap?  Anyhow, have fun, and
> feedback is always welcome and encouraged!

maybe it would make more sense to have this as an option to the
"background" method. i.e.

class EmailQueue < ActiveRecord::Base
	background :deliver, :every => 10, :concurrent => true
end

(would be good if we had a 'seconds' method for fixnum which just
returned itself, so you would write 10.seconds)

-- tim
Cloves Carneiro Junior (Guest)
on 2006-01-26 06:02
(Received via mailing list)
HTTP URL Validation is a Rails plugin that allows you
to validate a URL
entered in a form. It validates if the URL exists by
hitting it with a HEAD
request. There's the option to also check that the URL
returns content of
a specified type. HereĆ¢??s how you can use it your
model:

Class Article < ActiveRecord::Base
  validates_http_url :url, :content_type =>
"text/html"
  validates_http_utl :photo, :content_type => "image"
end

This example will make sure the value entered for the
URL field points to a
publicly accessible HTML page, and the photo field
points to an image.

You can get the plugin at
http://www.ccjr.name/blog/2006/01/25/http-url-validator/

Cheers,

Cloves Carneiro Jr
http://www.ccjr.name/blog






__________________________________________________________
Find your next car at http://autos.yahoo.ca
F18e9db7b1cded3d16d5424bad89d02a?d=identicon&s=25 Nshbrown N. (nshb)
on 2006-01-26 07:26
(Received via mailing list)
One useful addition worth considering is adding this RegEX as a define
or even as a helper to AR in the format of "validates_format_of_url
:url" to be a shorthand for the REGEX.

Source: http://www.nshb.net/node/252

class Company < ActiveRecord::Base
  validates_format_of :url, :with =>
/^(http|https):\/\/[a-z0-9]+([\-\.]{1}[a-z0-9]+)*\.[a-z]{2,5}(([0-9]{1,5})?\/.*)?$/ix
end

-Nb

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Nathaniel S. H. Brown                           http://nshb.net
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jeroen Houben (Guest)
on 2006-01-26 12:06
(Received via mailing list)
Nathaniel S. H. Brown wrote:
> One useful addition worth considering is adding this RegEX as a define or even as a 
helper to AR in the format of "validates_format_of_url :url" to be a shorthand for the 
REGEX.
>
> Source: http://www.nshb.net/node/252
>
> class Company < ActiveRecord::Base
>   validates_format_of :url, :with => 
/^(http|https):\/\/[a-z0-9]+([\-\.]{1}[a-z0-9]+)*\.[a-z]{2,5}(([0-9]{1,5})?\/.*)?$/ix
> end
>


Technically a URL can also start with ftp:// smb:// etc

Does this also allow stuff like:
http://username:password@www.example.com/?q=r&me=y...

Sorry, my REGEX skills are quite poor..

Jeroen
F18e9db7b1cded3d16d5424bad89d02a?d=identicon&s=25 Nshbrown N. (nshb)
on 2006-01-26 12:31
(Received via mailing list)
It only allows for URL's without the HTTP AUTH syntax you mentioned.

/^(http|https|ftp|smb):\/\/(a-z0-9]+:[a-z0-9]+@)?[a-z0-9]+([\-\.]{1}[a-z0-9]
+)*\.[a-z]{2,5}(([0-9]{1,5})?\/.*)?$/ix

Without testing, the above should work in such a case.

-Nb

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Nathaniel S. H. Brown                           http://nshb.net
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Alex Young (Guest)
on 2006-01-26 12:40
(Received via mailing list)
Nathaniel S. H. Brown wrote:
> It only allows for URL's without the HTTP AUTH syntax you mentioned.
>
> /^(http|https|ftp|smb):\/\/(a-z0-9]+:[a-z0-9]+@)?[a-z0-9]+([\-\.]{1}[a-z0-9]
> +)*\.[a-z]{2,5}(([0-9]{1,5})?\/.*)?$/ix

Depending on how complete you want it to be, you might want to get rid
of explicit scheme matching completely.  RFC 1738 indicates that
[A-Za-z.+-]+ should work instead of (http|https|ftp|smb).
Patrice Neff (Guest)
on 2006-01-26 12:52
(Received via mailing list)
El 26/01/2006, a las 06:04 AM, Jeroen Houben
escribiĆ³:
> Technically a URL can also start with ftp:// smb:// etc

Most of the time you require a user to enter an URL only HTTP (and
sometimes FTP and mailto) is allowed. At least in my applications.

Patrice_______________________________________________
Rails mailing list
Rails@lists.rubyonrails.org
http://lists.rubyonrails.org/mailman/listinfo/rails
Jeroen Houben (Guest)
on 2006-01-26 13:04
(Received via mailing list)
Alex Young wrote:
> Nathaniel S. H. Brown wrote:
>> It only allows for URL's without the HTTP AUTH syntax you mentioned.
>>
>> /^(http|https|ftp|smb):\/\/(a-z0-9]+:[a-z0-9]+@)?[a-z0-9]+([\-\.]{1}[a-z0-9]
>>
>> +)*\.[a-z]{2,5}(([0-9]{1,5})?\/.*)?$/ix
>
> Depending on how complete you want it to be, you might want to get rid
> of explicit scheme matching completely.  RFC 1738 indicates that
> [A-Za-z.+-]+ should work instead of (http|https|ftp|smb).

I agree. It should be up to the user though if they want to allow more
exotic protocols. I personally would never include this in a plugin, I
would just copy and paste the regex from Nathan's homepage and modify it
to fit my needs.

Jeroen
Alex Young (Guest)
on 2006-01-26 13:22
(Received via mailing list)
Jeroen Houben wrote:
>> [A-Za-z.+-]+ should work instead of (http|https|ftp|smb).
>
> I agree. It should be up to the user though if they want to allow more
> exotic protocols. I personally would never include this in a plugin, I
> would just copy and paste the regex from Nathan's homepage and modify it
> to fit my needs.
You could use a scheme whitelist that the first bracket group gets
compared against, couldn't you?  Something like:

class UriMatcher < Regexp
   def initialize(whitelist)
     super(###uri_regex_here###)
     @whitelist = Set.new(whitelist)
   end
   def =~(str)
     return @whitelist.include? self.match(str)[1]
   end
end

class Company < ActiveRecord::Base
   schemes = ['http', 'https', 'ftp']
   validates_format_of :url, :with => UriMatcher.new(schemes)
end

I guess the overriding principle here is to Do The Simplest Thing That
Can Possibly Work :-)
Rob Biedenharn (Guest)
on 2006-01-26 15:26
(Received via mailing list)
At 1/26/2006 06:38 AM, you wrote:
>Alex
Or go all the way with the regexp from RFC 2396
<http://www.ietf.org/rfc/rfc2396.txt>Uniform Resource Identifiers
(URI): Generic Syntax on page 29.  (the remainder is best viewed in a
monospaced font or just visit the RFC on the web.
-Rob



RFC 2396                   URI Generic Syntax                August 1998


B. Parsing a URI Reference with a Regular Expression

    As described in Section 4.3, the generic URI syntax is not
sufficient
    to disambiguate the components of some forms of URI.  Since the
    "greedy algorithm" described in that section is identical to the
    disambiguation method used by POSIX regular expressions, it is
    natural and commonplace to use a regular expression for parsing the
    potential four components and fragment identifier of a URI
reference.

    The following line is the regular expression for breaking-down a URI
    reference into its components.

       ^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?
        12            3  4          5       6  7        8 9

    The numbers in the second line above are only to assist readability;
    they indicate the reference points for each subexpression (i.e.,
each
    paired parenthesis).  We refer to the value matched for
subexpression
    <n> as $<n>.  For example, matching the above expression to

       http://www.ics.uci.edu/pub/ietf/uri/#Related

    results in the following subexpression matches:

       $1 = http:
       $2 = http
       $3 = //www.ics.uci.edu
       $4 = www.ics.uci.edu
       $5 = /pub/ietf/uri/
       $6 = <undefined>
       $7 = <undefined>
       $8 = #Related
       $9 = Related

    where <undefined> indicates that the component is not present, as is
    the case for the query component in the above example.  Therefore,
we
    can determine the value of the four components and fragment as

       scheme    = $2
       authority = $4
       path      = $5
       query     = $7
       fragment  = $9

    and, going in the opposite direction, we can recreate a URI
reference
    from its components using the algorithm in step 7 of Section 5.2.





Berners-Lee, et. al.        Standards Track                    [Page 29]
Ed70bd2729179633e1567d179b76aa00?d=identicon&s=25 Timo H. (thoepfner)
on 2006-01-26 16:23
(Received via mailing list)
Hi Kyle,

looks great.

There's a small bug in RailsCron#task_list: @@last_db_update is not
updated, so the DB is always queried. Changing

	      @@task_list = find(:all)

to

	      @@last_db_update = Time.now.to_i
	      @@task_list = find(:all)

solves it.

I agree to Tim, that :overlap should be settable on a per-task basis
(and that :concurrent would make a nice option-name for it).

Thanks for the nice plugin.

Timo

Am 25.01.2006 um 21:26 schrieb Kyle Maxwell:
Kyle Maxwell (Guest)
on 2006-01-26 19:16
(Received via mailing list)
On 1/25/06, Jeff Blasius <jeff.blasius@gmail.com> wrote:
> > RailsCron was even easier to manage.  So I implemented a graceful
> >                 #process the queue
> >
> > Kyle Maxwell
> >
> >
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>

I just posted an example on my blog:
http://www.kylemaxwell.com/articles/2006/01/26/sim...

--
Kyle Maxwell
Chief Technologist
E Factor Media // FN Interactive
kyle@efactormedia.com
1-866-263-3261
Kyle Maxwell (Guest)
on 2006-01-26 19:37
(Received via mailing list)
On 1/26/06, Timo Hoepfner <th-dev@onlinehome.de> wrote:
>
>               @@last_db_update = Time.now.to_i
>               @@task_list = find(:all)
>
> solves it.

Yeah, I feel a little sheepish...  It's fixed in the trunk, and I'll
probably tag up 0.2.1 with this, and the :concurrent option sometime
today or tomorrow.

--
Kyle Maxwell
Chief Technologist
E Factor Media // FN Interactive
kyle@efactormedia.com
1-866-263-3261
F18e9db7b1cded3d16d5424bad89d02a?d=identicon&s=25 Nshbrown N. (nshb)
on 2006-01-26 22:23
(Received via mailing list)
That regex seems broken.

It says that a URI can start with everything other than any of the
following
items ":/?#". Meaning that a URI can start with a "@" or a "%" or any of
the
other million keys that aren't those four?

Alex Young's mention of RFC 1738 looks much closer to what a URI should
start with [a-z.+-]+

/^([a-z.+-]+):\/\/(a-z0-9]+:[a-z0-9]+@)?[a-z0-9]+([\-\.]{1}[a-z0-9]+)*\.[a-z
]{2,5}(([0-9]{1,5})?\/.*)?$/ix

The "I" flag negates the second case of A-Z.

I really like the concept of the URI Matcher, but it would be far
simpler to
have a helper method for AR with usage like so:

validates_format_of_uri :link, :schemes => [ 'mailto', 'http', 'https' ]

That is what I was suggesting go into the Plugin. Or even simply mapping
the
above Regex into a define like so:

REGEX_URI =
/^([a-z.+-]+):\/\/(a-z0-9]+:[a-z0-9]+@)?[a-z0-9]+([\-\.]{1}[a-z0-9]+)*\.[a-z
]{2,5}(([0-9]{1,5})?\/.*)?$/ix

So you can use it within the original code:

validates_format_of :uri, :with => REGEX_URI

This would be a simple method of implementation, and would stick to the
DRY
principle as I am sure if you have one URI column in your database, you
are
likely to have another.

-Nb

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Nathaniel S. H. Brown                           http://nshb.net
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Be5ecce553223fe2a84f07f5e8df4917?d=identicon&s=25 Nate D. (ndrake)
on 2006-01-26 22:46
I'm a newbie, and having trouble installing the plugin.

I did "ruby script/plugin source
http://svn.kylemaxwell.com/rails_cron/tags/0.2/", and it is added ok.
When I do "ruby script/plugin list", I see the following for the new
source:

lib
http://svn.kylemaxwell.com/rails_cron/tags/0.2/lib/
tasks
http://svn.kylemaxwell.com/rails_cron/tags/0.2/tasks/

This doesn't seem correct.  There are other plugins listed (i.e.
account_location, acts_as_taggable, etc), so I was thinking there would
be something like "rails_cron" instead of "lib" and "tasks".  What am I
doing wrong?

Thanks in advance!

Nate
Alex Young (Guest)
on 2006-01-26 23:18
(Received via mailing list)
Nathaniel S. H. Brown wrote:
> That regex seems broken.
>
> It says that a URI can start with everything other than any of the following
> items ":/?#". Meaning that a URI can start with a "@" or a "%" or any of the
> other million keys that aren't those four?
>
Yup.  It's a superset.  The regex from the RFC only serves to separate
the constituent parts of a URI, not to validate them.  From page 12,
though:

    Scheme names consist of a sequence of characters beginning with a
    lower case letter and followed by any combination of lower case
    letters, digits, plus ("+"), period ("."), or hyphen ("-").  For
    resiliency, programs interpreting URI should treat upper case
letters
    as equivalent to lower case in scheme names (e.g., allow "HTTP" as
    well as "http").

       scheme        = alpha *( alpha | digit | "+" | "-" | "." )
Kyle Maxwell (Guest)
on 2006-01-28 03:04
(Received via mailing list)
On 1/25/06, Kyle Maxwell <kyle@kylemaxwell.com> wrote:
>
>
>
> --
> Kyle Maxwell
> Chief Technologist
> E Factor Media // FN Interactive
> kyle@efactormedia.com
> 1-866-263-3261
>

I just tagged 0.2.1 which fixes several minor issues, and adds the
:concurrent option suggested by the list.

background :method, :concurrent => true

STUFF HAS BEEN REMOVED!!
- The overlap option has been superceded by concurrent.
- RailsCron DOES NOT start automatically.  Please use the rake tasks.
This was a problematic feature, but if you can code it without side
efffects, the patch is welcome.

--
Kyle Maxwell
Chief Technologist
E Factor Media // FN Interactive
kyle@efactormedia.com
1-866-263-3261
A6a8ad1b2e50469e4968ff5ffecc0dc5?d=identicon&s=25 Lon B. (spdemac)
on 2006-02-14 20:42
Are there any plans to support distributed jobs?

For example, when jobs grows too big to be accomplished between the cron
intervals.

I have been thinking of something like, acts_as_queue or
acts_as_queue_job.

Something to allow multiple threads and/or separate physical servers to
check out jobs for processing, without stomping on each other.

Using servers as an example:

Server 1 checks out a batch of jobs, processes the jobs and clears them
from the queue as they are finished.

Server 2 grabs the next batch, and so on.

Is Server 1 fails, the next machine grabs the unfinished jobs based on a
maximum checkout time value set during check out.

If a server exceeds the maximum checkout time value during processing,
it drops the remaining jobs and grabs a new batch.

Certainly there are many nuances I am missing. But this is my first pass
at a solution.

Any comments or code would be greatly appreciated. ;-)
Kyle Maxwell (Guest)
on 2006-02-14 20:58
(Received via mailing list)
On 2/14/06, Lon Baker <lon@speedymac.com> wrote:
>
> If a server exceeds the maximum checkout time value during processing,
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>

I currently don't have any plans to support distributed jobs, because
my current apps don't require it.  RailsCron currently is
multithreaded.

If I was going to do simple distributed jobs, I would put:

return unless ENV["HOSTNAME"] == "specific_server"

at the top of each action to let each server run a subset of the task
list.  If you wanted multiple servers to run the same action on a
queue, then it isn't too bad to implement that queue with an exclusive
checkout flag.  This type of thing is separate logic from the idea of
RailsCron, and is suitable for the type of acts_as_queue plugin you
are talking about.  I do something similar for sending bulk email, but
haven't been able to extract a generalized plugin from it.

--
Kyle Maxwell
Chief Technologist
E Factor Media // FN Interactive
kyle@efactormedia.com
1-866-263-3261
Cede46f8ea9231ee030f9f3ce1ca6fd8?d=identicon&s=25 Hussein M. (mucki)
on 2006-02-14 22:26
very very useful plugin.
But if someone is using migration, your plugin is not useable. Instead
of generating a tabel for the cronjob, it would be better to generate a
migration-script.
This is now a standard in rails.


Hussein Morsy
Kyle Maxwell (Guest)
on 2006-02-14 23:21
(Received via mailing list)
On 2/14/06, Hussein Morsy <hussein@morsy.de> wrote:
> Posted via http://www.ruby-forum.com/.
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>

A migration would be better, but this works fine, and doesn't
interfere with your migrations.  Maybe in the future.

--
Kyle Maxwell
Chief Technologist
E Factor Media // FN Interactive
kyle@efactormedia.com
1-866-263-3261
This topic is locked and can not be replied to.