Hi guys,
I’m trying to make the code I have written that populates an LDAP server
from a list of user accounts a little more efficient.
The logic behind the application is fairly straightforward:
For every username in an array
Check whether the user is already in ldap
If the user is not in ldap then
Check the type of user (staff/student/invalid)
Collect all relevant attributes for the user type from database
Return user attributes
If user attributes are not null then build them into an LDAP:MOD
object
Return LDAP:MOD object
Add user to LDAP
Append successful accounts to list of added accounts
Append failed accounts to list of remaining accounts
Return to main loop
Coded up, this looks like:
accounts.each do |login|
if (ldapAdd(ldapconn, userconn, login))
added.push login
else
failed.push login
end
end
The application works fine as a sequential process, and when run in AOT
and server vm mode the speed is comparable to some legacy Python code
that we have been using - however I’m eager to see if I can get some
extra speed from JRuby’s native threading (I’m also doing it for my own
curiosity 
The logic for the threaded version of the application is very similar:
until (accounts.empty?) do
if (threadcount < maxthreads)
threadlock.synchronize do
threadcount += 1
end
login = accounts.pop
workerthreads.push Thread.new(login) { |loginthread|
status = ldapAdd(ldapconn, sqlconn, loginthread)
threadlock.synchronize do
if (status)
added.push loginthread
else
failed.push loginthread
end
threadcount -= 1
end
}
end
end
Testing, retrieving and building LDAP attributes for each person should
be able to be done in parallel for at least a small number of threads at
any given time. The code above works ok and I can indeed see the
interleaved output from ldapAdd and its subsequent children (check
account, retrieve attribs, assemble ldap object, add ldap object) for a
number of accounts simulatenously - however I’m not getting the speed or
concurrency that I think I ought to.
One query I had was relating to the ldap and sql connection objects that
are passed to each worker thread; I create a single connection object
for both LDAP (a jruby-ldap LDAP::SSLConn) and JDBC
(Java::com.mysql.jdbc.Driver) outside the main loop and pass them to the
thread (as ldapconn & sqlconn). Are these connection objects thread
safe, or will workers using these objects block? I guess that could be
the cause…
Having written all of this out I now figure I can spawn a number of long
running threads that consume data from the account array, each with
their own ldap and sql connection objects and not have the overhead of
creating and destroying threads each time around the main loop. However
it would still be interesting to know if those connection objects are
thread safe, or if there are alternatives, such as using Java connection
pooling technology within JRuby?
Cheers
John
Unix & Web Infrastructure Management
Faculty of Medical Sciences Computing
University of Newcastle
Email : [email protected]
Web: http://www.ncl.ac.uk/medev
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email