Forum: JRuby Difference in behaviour using 'require' within threads

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
0513c92286df6a35d3de502676a2e58d?d=identicon&s=25 John Snowdon (Guest)
on 2008-10-25 15:58
(Received via mailing list)
Hi list,

Following on from my previous question regarding threading I have
observed what appears to be a difference in behaviour of 'require' when
running code within a number of different threads. I'm not sure whether
this is expected behaviour, or a consequence of JRuby's native threading
when running on the JVM.

I have one file with two methods defined:

main.rb
-
def mainmethod(work)
  require "dbmethods.rb"
  while (work)
    threadcount += 1
    threads.push thread.new(threadcount) { |threadid|
        anothermethod(threadid)
      }
    end
  end
end

def anothermethod(id)
  require "moremethods.rb"
  puts "called by" + id
  externalmethod()
end

Another file, has a third method defined:

moremethods.rb
-
def externalmethod
  stuff()
end

The code works and it executes anothermethod() and externalmethod() is
then called, but I *always* get a "No method" error relating to the
function externalmethod part way through the execution of the code of
anothermethod(). This can vary from the very first thread that calls
externalmethod, or it can run a couple of times and return a valid
result until the 3rd, 4th or 5th time.

If I change mainfunc() to use something along the lines of:

while (work)
  anothermethod
end

ie. without the threads, then the require of moremethods.rb works
*every* time and I never get a "No method" error.
In addition, if I instead change the require to be in the main body of
mainfunc(), rather than anothermethod() I also never see the error.

Could anyone explain why I see this behaviour? My naivety says that if I
'require' another library or file, then that dependency is loaded before
execution continues.

The actual code spawning the threads from the main body of the
application looks like this:

while (threadcount < maxthreads)
  workerthreads.push Thread.new(threadcount) { |threadid|
    userconn = sqlConn()
    ldapconn = ldapConn()
    if (verbose)
      puts "thread #{threadid} started"
    end
    until ((accountqueue.length < 1) && (accounts.length <1))
      login = accountqueue.deq
      if (verbose)
        puts "thread #{threadid} Got some work : #{login}"
      end
      status = ldapAccountBuildStaff(verbose, userconn, threadid, login)
      if (status)
        added.enq(login)
      else
        failed.enq(login)
      end
    end
    if (verbose)
      puts "thread #{threadid} exiting"
    end
    userconn.close
    ldapconn.unbind
  }
  threadcount += 1
end

... and the ldapAccountBuildStaff method is rather simple:

def ldapAccountBuildStaff(verbose, sqlconn, threadid, login)
  # Builds a Staff LDAP object
  # Load locally defined libraries
  require "libs/ldapstaffaccount.rb"
  if (verbose >1)
    puts "thread #{threadid} ldapAccountBuildStaff : #{login}"
  end
  personattrs = ldapStaffAttrs(verbose, sqlconn, login)
  return personattrs
end

The "No method" error relating to ldapStaffAttrs (which is defined in
libs/ldapstaffaccount.rb) is what I keep getting.

As I say, I can resolve the issue either by running from a simple loop,
or by moving the require to the main body, rather than the
ldapAccountBuildStaff method. But I would appreciate any insight on why
the error could be occurring!

John

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
13313ac2ec7ba7c43b1b952db034ff3b?d=identicon&s=25 Thomas E Enebo (Guest)
on 2008-10-26 08:12
(Received via mailing list)
See:
http://jira.codehaus.org/browse/JRUBY-3078

We also have an older thread about a week or two old in our mailing
list about thread safety in require.  It is not a simple problem, but
we think we can fix it.

This is really an effect of JRuby having native threads.  With Java
native threads all require/loads are happening concurrently.  We need
to synchronize on registration, but we cannot naively synchronize or
we can deadlock in several scenarios (one of which may not be that
uncommon).

-Tom

On Sat, Oct 25, 2008 at 8:55 AM, John Snowdon
<j.p.snowdon@newcastle.ac.uk> wrote:
>  while (work)
>  puts "called by" + id
>
>
>                end
>                        end
> ... and the ldapAccountBuildStaff method is rather simple:
> end
>    http://xircles.codehaus.org/manage_email
>
>
>



--
Blog: http://www.bloglines.com/blog/ThomasEEnebo
Email: enebo@acm.org , tom.enebo@gmail.com

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
0513c92286df6a35d3de502676a2e58d?d=identicon&s=25 John Snowdon (Guest)
on 2008-10-27 08:55
(Received via mailing list)
Thanks Tom, that certainly describes the symptoms I have seen.

Moving the relevant require statements to the body of the main function
has solved the issue as expected. It's not a show stopper for me as I've
only got 6 or so additional libraries that I require throughout the life
of the thread.

I'll keep an eye on 3078 and 3057.

Cheers

John
---
 Unix & Web Infrastructure Management
 Faculty of Medical Sciences Computing
 University of Newcastle

 Email : j.p.snowdon@ncl.ac.uk
 Web: http://www.ncl.ac.uk/medev


>We also have an older thread about a week or two old in our mailing
>
>> I have one file with two methods defined:
>>    end
>>
>externalmethod, or it can run a couple of times and return a valid
>> In addition, if I instead change the require to be in the main body of
>>        workerthreads.push Thread.new(threadcount) { |threadid|
>: #{login}"
>>                        puts "thread #{threadid} exiting"
>>        # Builds a Staff LDAP object
>> The "No method" error relating to ldapStaffAttrs (which is defined in
>> To unsubscribe from this list, please visit:
>Email: enebo@acm.org , tom.enebo@gmail.com
>
>---------------------------------------------------------------------
>To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
This topic is locked and can not be replied to.