Rails and background tasks/threads

Alex Y. wrote:

It’s basically just keeping the non-serializable stuff in your own
hash that won’t be serialized.

If that worked, you wouldn’t need a session at all, you could just
keep everything in memory all the time.

Well, yeah, but all the other crap that you can serialize you might as
well have in a
session and let it get serialized. This whole thread was more or less a
thought experiment
anyway. I don’t personally currently have a need to stick threads,
procs, or singletons in
the session… but I expect I might some day. I’m just working through
ruby’s rough spots
mentally…

Ultimately, my point is that ruby/rails needs a way to keep
application-wide data in
memory. This is very useful in the Java world. In fact, I’d say that a
big chunk of the
reason why Java did so well in the web world was that they went straight
for the container
model. Plopping a multi-threaded, long-running server process into a
world of
start-process-stop CGI insanity made people go “whoa… cool”.

b

On Feb 15, 2006, at 6:32 AM, Eric Ching wrote:

Wow, thanks for all the great responses. I think I need to plug
away at
some more Rails and give the sugestions a try (rinda/drb). Any
example
would be greatly appreciated.

I want to use Rails because there is so much about it I do like, but
without being able to tightly control background threads/tasks I
really
cannot jump into it 100% - that’s the make or break for me.

Eric-

Here is a simple drb server/client for you to play with. This is an

example of exposing an ActiveRecord model over drb. You can actually
publish any object you like over drb. this example is a bit contrived
but it can show you how drb works a bit. Run the server in a terminal
and then open irb and paste in the client part and play with that. I
am going to work on getting my drb+ajax progress bars plugin into
releaseable shape this weekend so stay tuned.

#!/usr/local/bin/ruby
Server
require ‘rubygems’
require ‘drb/drb’
require_gem ‘activerecord’

ActiveRecord::Base.establish_connection(
:adapter => “mysql”,
:username => “root”,
:host => “localhost”,
:password => “xxxxxxx”,
:database => “remote_drb”
)

ActiveRecord::Schema.define(:version => 1) do
create_table “remote_records”, :force => true do |t|
t.column :title, :string
t.column :desc, :string
end
end

class RemoteRecord < ActiveRecord::Base
include DRb::DRbUndumped
end

DRb.start_service(“druby://127.0.0.1:3500”, RemoteRecord)
puts DRb.uri
DRb.thread.join

END
#client run in irb
require ‘rubygems’
require ‘drb/drb’
require_gem ‘activerecord’

DRb.start_service
RemoteRecord = DRbObject.new(nil, ‘druby://127.0.0.1:3500’)
[
[‘foo’,‘bar’],
[‘qux’, ‘baz’],
[‘nik’, ‘nuk’]
].each do |record|
RemoteRecord.create :title => record[0], :desc => record[1]
end

test = RemoteRecord.find :all
test.each{|o| p “#{o.title} : #{o.desc}” }

results in:

“foo : bar”

“qux : baz”

“nik : nuk”

Cheers-

-Ezra Z.
Yakima Herald-Republic
WebMaster

509-577-7732
[email protected]

Sweet… moving this to my “saved” folder!

b

Ezra (and everyone else :slight_smile:

Ok, here’s how I am using Linda (cleaned up as best I can to make things
more clear). Basically, put all of these files into one directory to get
started, and then after things are working then you can start moving
different pieces to different servers.

First of all, the background that you should read (or, the background
that I
read that was helpful):
Linda (coordination language) - Wikipedia (good
background on Linda (and why you’d want to use it), which Rinda
implements)
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/15023
http://segment7.net/projects/ruby/drb/rinda/ringserver.html
http://www.ruby-doc.org/stdlib/libdoc/rinda/rdoc/classes/Rinda/TupleSpace.ht
ml
http://www.ntecs.de/blog/Tech/ComputerScience

and, of course, the pickax has a page on Rinda.

And finally, my code (not very well documented, since it assumes that
you’ve
read all of the other URLs I pasted above and that now you just want an
example of how it works all together as well as an example of several
things
that I couldn’t really find easily in the documentation).

(there are 4 files below)

— the port where everyone talks to each other needs to be defined —
In file, rinda_port_uri.rb

RINDA_PORT_URI = “druby://your.hostname.com:12345”
#(you can also use “localhost” if you are all running on the same
machine, but as soon as you put ANY process on a different server, then
they
have to all have the exact same druby:// line (as far as I can tell…
:slight_smile:
Ruby -rsocket -e ‘p Socket.gethostname’
# this command was helpful for me to find out what ruby thinks my
server was named.

File: rinda_blackboard.rb # this is the main “server” that enables
agents
to get/send messages for things they want someone to do.

require ‘drb/drb’
require ‘drb/acl’
require ‘rinda/tuplespace’
require ‘rinda_port_uri’ # defines the constant used for the Port

acl = ACL.new(%w(deny all # block out other servers from accessing
your
Linda blackboard
allow
10.0.0.7
allow
10.0.0.4
allow
60.59.145.219 #the documentation says that you can put wildcards
‘60.59.145.*’ but that didn’t work for me…)
allow
60.59.145.220
allow
localhost))

DRb.install_acl(acl)

Ruby -rsocket -e ‘p Socket.gethostname’
DRb.start_service(RINDA_PORT_URI, Rinda::TupleSpace.new(10))
#puts DRb.uri
#(0…5).each {|x| # this block will start up 5 agents

puts “Starting agent #{x}”

IO.popen(“ruby rinda_agent.rb”)

#}
puts “blackboard is open on #{RINDA_PORT_URI}”
DRb.thread.join
#puts ‘[return] to exit’
#gets

File: rinda_agent.rb # this is where you do the work… you can have
multiple agents running, and on different servers all pointing to the
same
rinda_blackboard

this is a “example wrapper” thing… If I want an agent to do some

work

for me, then I (personally) want to pass:

- a message,

- a hash containing variables and stuff

- a reference ID that uniquely identifies this message request

- the “return message” I will be looking for that I’d like the agent

to

reply using (so that I know what to look for)

- a return hash containing results from the agent

(that’s just me, what I want…)

require ‘drb/drb’
require ‘rinda/tuplespace’
require ‘rinda_port_uri’ # defines the constant used for the Port
require ‘json/lexer’
require ‘json/objects’
# I like using JSON to tranfer info between agents, but you can use
a simple hash if you wish.

DRb.start_service
ts = Rinda::TupleSpaceProxy.new(DRbObject.new(nil,RINDA_PORT_URI))

class Gen

def worker1(var_in, ref_id, msg_out)
	... do something here
	an_amazing_result = ...
	#return the results of the work
	[msg_out,an_amazing_result]
end

end

loop do
puts
puts
puts
puts
puts “waiting…(for PPT stuff)”

# Read the next command
op, var_in, ref_id,msg_out,var_out = ts.take(

[%r{^(msg_1_I_listen_for|msg_2_I_listen_for)$},nil,nil,nil,nil],55) #
time
out after 55 seconds if no activity… good for debugging initially (can
be
omitted in production)

puts "Received! #{op} #{ref_id} #{var_in}"

gen = Gen.new
msg_out,var_out = gen.send(op, var_in, ref_id)

#— DONE --------------------
ts.write([msg_out,ref_id, var_out]) unless (msg_out.nil? or
msg_out.length == 0)
puts “done! #{msg_out} #{ref_id} #{var_out}”
end

file: rinda_watcher.rb

#finally, once you’ve got your blackboard where you can post messages to
get
work done, I wanted to be able to SEE what the heck was being put on
there
:slight_smile: so this file is a easy to implement, hacky, fast way of doing
that…

require ‘drb/drb’
require ‘rinda/tuplespace’
require ‘rinda_port_uri’ # defines the constant used for the Port
require ‘pp’

DRb.start_service
puts “trying to connect to #{RINDA_PORT_URI}…”
ts = Rinda::TupleSpaceProxy.new(DRbObject.new(nil,RINDA_PORT_URI))
puts “---------------------Watching”

option = 1
case option
when 1
result = ts.read_all([nil,nil])
pp result unless result.nil?
result = ts.read_all([nil,nil,nil])
pp result unless result.nil?
result = ts.read_all([nil,nil,nil,nil])
pp result unless result.nil?
result = ts.read_all([nil,nil,nil,nil,nil])
pp result unless result.nil?
result = ts.read_all([nil,nil,nil,nil,nil,nil])
pp result unless result.nil?
puts
when 2
result =
ts.read_all([%r{prepare_to_generate_ppt_.*},
nil, #session[:username],
nil, #@current_ppt_name,
nil, #@new_ppt_name,
nil, #@path_to_template,
nil, #json_msg
nil, #user password
])

		pp result unless result.nil?
when 3
		0.upto(10) {|ignore|
			hit = 0
			result = ts.read_all([nil,nil])
			unless result.nil? or result.size ==0
				pp result
				ts.take([nil,nil])
				hit +=1
			end

			result = ts.read_all([nil,nil,nil])
			unless result.nil? or result.size ==0
				pp result
				ts.take([nil,nil,nil])
				hit +=1
			end

			result = ts.read_all([nil,nil,nil,nil])
			unless result.nil? or result.size ==0
				pp result
				ts.take([nil,nil,nil,nil])
				hit +=1
			end

			result = ts.read_all([nil,nil,nil,nil,nil])
			unless result.nil? or result.size ==0
				pp result
				ts.take([nil,nil,nil,nil,nil])
				hit +=1
			end

			result =

ts.read_all([nil,nil,nil,nil,nil,nil])
unless result.nil? or result.size ==0
pp result
ts.take([nil,nil,nil,nil,nil,nil])
hit +=1
end

			result =

ts.read_all([nil,nil,nil,nil,nil,nil,nil])
unless result.nil? or result.size ==0
pp result

ts.take([nil,nil,nil,nil,nil,nil,nil])
hit +=1
end

			if hit > 0
				puts '----------------------------'
				puts
			end
		}

End

-Greg

[email protected]
916 792 4538

Add me to the list of people eagerly awaiting the arrival of this
plugin. :slight_smile:

Thanks for the code samples, much appreciated. :slight_smile: