Trying to add threading to parse a .txt file

Hi All,

I have code that will parse a text file line by line and then send the
variable I want to a sub routine to send pages to people. My question
is can I use threads to parse that text file with out duplicating the
output. Below is the code it may make more sense to look at it.

#!/usr/bin/ruby

trap(‘SIGHUP’,‘IGNORE’)

i = 0

def autopage(pagenum,count)

begin

snpp -s SERVER ADDRESS -f 'EOC Notify' -m 'test' #{pagenum}
rescue
puts “#{pagenum} has NOT been paged.”
else
puts "#{pagenum} has been paged. #{count} "
end
end

File.open(’//var//lib//asterisk//agi-bin//pager_list.txt’, ‘r’) do |
f1|
while line = f1.gets
result = {}
pagenum,fname,lname=line.split(" ")
if line.split.empty?
else
if pagenum == “####”
else
count = (i += 1)
autopage(pagenum,count)
end
end
end
end

Thanks!!

On Mon, Jun 9, 2008 at 2:30 PM, [email protected] wrote:

Hi All,

I have code that will parse a text file line by line and then send the
variable I want to a sub routine to send pages to people. My question
is can I use threads to parse that text file with out duplicating the
output. Below is the code it may make more sense to look at it.

Why do you want to use threads?


Avdi

Home: http://avdi.org
Developer Blog: Avdi Grimm, Code Cleric
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

On Jun 9, 3:35 pm, Avdi G. [email protected] wrote:


Avdi

Home:http://avdi.org
Developer Blog:Avdi Grimm, Code Cleric
Twitter:http://twitter.com/avdi
Journal:http://avdi.livejournal.com

As this .txt file is 200 lines long. I can send 200 individual pages
now in 3:min33sec i would like to cut that time by a forth by using 4
threads. I just don’t think I know enough enough about threads yet to
do so. The closest thing I have come up with so far is to have 2
threads parsing two .txt file with 100 lines each in them.

On Mon, Jun 9, 2008 at 4:34 PM, [email protected] wrote:

As this .txt file is 200 lines long. I can send 200 individual pages
now in 3:min33sec i would like to cut that time by a forth by using 4
threads. I just don’t think I know enough enough about threads yet to
do so. The closest thing I have come up with so far is to have 2
threads parsing two .txt file with 100 lines each in them.

Chances are, with the MRI threading model, you’re not going to get any
speed increase from threading this. Ruby IO tends to block the entire
interpreter.


Avdi

Home: http://avdi.org
Developer Blog: Avdi Grimm, Code Cleric
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

On Jun 9, 2:29 pm, [email protected] wrote:

puts "#{pagenum} has been paged. #{count} "
    if pagenum == "####"
    else
       count = (i += 1)
       autopage(pagenum,count)
       end
     end
   end
 end

Thanks!!

The parsing of the file itself, should not be threaded. What you
should do is every time you read a line and determine that you need to
send a page, put the paging information into a Queue.

Then you can creat four, ten, or some other number of threads to
consume the paging information from the queue. After pulling
something from the queue, they’ll use the information to call your
autopage method.

For a good example, see the the short producer/consumer program
created by Robert Kellner that appears in the ‘thread’ library section
of the Pick Axe book (2nd ed). In my version of the book it’s on page
722, but page numbers seem to vary b/w printings and format.

If you don’t mind my offering some additional unsolicited notes on
your code:

You have a few "if"s with empty "then"s. Ruby has an “unless” which
is essentially the opposite of “if” that would make this code cleaner
(IMHO).

You can put a “rescue” at the top level inside a method without having
a begin/end block. But you’re right in that in other cases you do
need the begin/end block.

Hope that’s helpful,

Eric

====

LearnRuby.com offers Rails & Ruby HANDS-ON public & ON-SITE
workshops.
Ruby Fundamentals Wkshp June 16-18 Ann Arbor, Mich.
Ready for Rails R. Wkshp June 23-24 Ann Arbor, Mich.
Ruby on Rails Wkshp June 25-27 Ann Arbor, Mich.
Ruby Plus Rails Combo Wkshp June 23-27 Ann Arbor, Mich
Please visit http://LearnRuby.com for all the details.