Problem with concurrency

Hi guys,

I’m having an issue with a multi-threaded system I’m developing.

Sorry if my question sounds too generic and unspecific but:
Are there any known pitfalls that one might come across when using
threads?

This is the code that runs the threads:
(I’m new to this so the code is borrowed from another post from this
list.)

def run_mods( page_data, structure )

    mod_queue = Queue.new

    @threads = ( [email protected][:threads] ).map {
        |i|
        Thread.new( mod_queue ) {
            |q|
            until( q == ( curr_mod = q.deq ) )

                if( !run_module?( curr_mod , structure ) )
                    print_verbose( 'Skipping ' + curr_mod.to_s +
                        ', nothing to audit.' )
                    next
                end

                print_debug( )
                print_debug( 'Thread-' + i.to_s + " " +

curr_mod.inspect )
print_debug( )

                print_status( curr_mod.to_s )

                mod_new = curr_mod.new( page_data, structure )

                mod_new.prepare   if curr_mod.method_defined?(

‘prepare’ )
mod_new.run
mod_new.clean_up if curr_mod.method_defined?(
‘clean_up’ )

                while( handle_interrupt(  ) )
                end

            end
        }
    }

    # enque the loaded mods
    for mod in ls_loaded_mods
        mod_queue.enq mod
    end

    @threads.size.times { mod_queue.enq mod_queue }
    @threads.each { |t| t.join }

end

The modules call their parent and the parent uses Net::HTTP.
Then they get the HTTP response via the parent and manipulate it in some
way.
My main problem is that from what I’ve gathered some HTTP calls fail
without an error.

Some modules just don’t produce any results when the thread count is set
higher than 1.

I normally wouldn’t bother you with something so broad but I’ve been
debugging this for 24 hours straight so I figured that someone else
might have run to a similar problem and save me 24 more hours of
debugging.

Has anyone else had a similar experience?

On 07/29/2010 07:55 AM, Tasos L. wrote:

list.)

                 while( handle_interrupt(  ) )

without an error.
Try this at the top of your program:

Thread.abort_on_exception = true

Then you will probably see errors.

Kind regards

robert

Robert K. wrote:

On 07/29/2010 07:55 AM, Tasos L. wrote:

list.)

                 while( handle_interrupt(  ) )

without an error.
Try this at the top of your program:

Thread.abort_on_exception = true

Then you will probably see errors.

Kind regards

robert

Hi Robert,

Thanks for the reply, however I didn’t see any errors.

This is going to be a bitch to debug…

Tasos L. wrote:

Robert K. wrote:

On 07/29/2010 07:55 AM, Tasos L. wrote:

list.)

                 while( handle_interrupt(  ) )

without an error.
Try this at the top of your program:

Thread.abort_on_exception = true

Then you will probably see errors.

Kind regards

robert

Hi Robert,

Thanks for the reply, however I didn’t see any errors.

This is going to be a bitch to debug…

Huh…it seems that the problem was actually caused by the localhost
webserver.
My laptop had a crazy uptime and all its memory (virtual and ram) had
been somewhat exhausted.

After a reboot everything started to work fine.

I can’t believe all those hours of debugging went to waist.

Robert K. wrote:

On 07/29/2010 10:16 AM, Tasos L. wrote:

Thread.abort_on_exception = true

Then you will probably see errors.

I can’t believe all those hours of debugging went to waist.

If it can happen once chances are that it will happen again. It’s
probably worth to dig a bit and try to find out what caused the memory
issue in the first place. Maybe you have a leaky application. What OS
btw?

Kind regards

robert

Well, the laptop was on for 2-3 days constantly running fullscreen flash
video, eclipse, apache2 & mysql ( being constantly hammered by Arachni
due to the debugging itself) and around 13 tabs of Firefox spread over
several windows.

So in hindsight it’s not much of a surprise.

I’m running the latest Kubuntu with Xfce, so I guess it has now become
Xubuntu…

On 07/29/2010 10:16 AM, Tasos L. wrote:

Thread.abort_on_exception = true

Then you will probably see errors.

I can’t believe all those hours of debugging went to waist.

If it can happen once chances are that it will happen again. It’s
probably worth to dig a bit and try to find out what caused the memory
issue in the first place. Maybe you have a leaky application. What OS
btw?

Kind regards

robert