Why does this code deadlock?


#1

Hi,

I’m getting a deadlock on this small program, on both Windows
and Linux:

require 'net/ftp'
Thread.abort_on_exception = true
ftp = Net::FTP.new('ftp.idsoftware.com')
p( Thread.new {$SAFE=4; ftp.login('anonymous', 'removed_email_address@domain.invalid')}.value )

ruby -v ftp_safe_test.rb

ruby 1.8.4 (2005-12-24) [i386-mswin32]
ftp_safe_test.rb:4:in `value’: Thread(0x27855c8): deadlock (fatal)
from ftp_safe_test.rb:4

$ ruby -v safe_ftp_test.rb
ruby 1.8.4 (2005-12-24) [i686-linux]
safe_ftp_test.rb:4:in `value’: Thread(0x4f060748): deadlock (fatal)
from safe_ftp_test.rb:4

Now, I didn’t necessarily expect the code to WORK at $SAFE=4,
but I was anticipating a SecurityError rather than a deadlock.

(I’ve tried it with Thread.abort_on_exception both true and
false, same result.)

Any thoughts?

Thanks,

Regards,

Bill


#2

Hi Bill-

On Apr 17, 2007, at 4:34 PM, Bill K. wrote:

ruby 1.8.4 (2005-12-24) [i686-linux]
Any thoughts?

Thanks,

Regards,

Bill

Are you using ruby 1.8.6 by chance? If so install the laets
fastthread gem and require it in your script to see if it fixes the
deadlock. Unfortunately the fastthread code in the ruby1.8.6 release
has some bugs that are fixed in the fastthread gem

Cheers-
– Ezra Z.
– Lead Rails Evangelist
– removed_email_address@domain.invalid
– Engine Y., Serious Rails Hosting
– (866) 518-YARD (9273)


#3

Hi Ezra,


has some bugs that are fixed in the fastthread gem
The above was 1.8.4 with the standard thread.rb.

Trying ruby 1.8.6 (2007-03-16 patchlevel 3) [i686-linux]
…I still get the deadlock, with or without fastthread:

ruby -v ftp_safe_test.rb
ruby 1.8.6 (2007-03-16 patchlevel 3) [i686-linux]
ftp_safe_test.rb:5:in `value’: Thread(0x50df1704): deadlock (fatal)
from ftp_safe_test.rb:5

Regards,

Bill


#4

From: “Nobuyoshi N.” removed_email_address@domain.invalid

At Wed, 18 Apr 2007 08:34:03 +0900,
Bill K. wrote in [ruby-talk:248300]:

require ‘net/ftp’
Thread.abort_on_exception = true
ftp = Net::FTP.new(‘ftp.idsoftware.com’)
th = Thread.new {$SAFE=4; ftp.login(‘anonymous’, ‘removed_email_address@domain.invalid’)}
Thread.critical = false
p th.value

Nobu, you RULE!!! :smiley:

Thanks!

Bill


#5

Hi,

At Wed, 18 Apr 2007 08:34:03 +0900,
Bill K. wrote in [ruby-talk:248300]:

require ‘net/ftp’
Thread.abort_on_exception = true
ftp = Net::FTP.new(‘ftp.idsoftware.com’)
th = Thread.new {$SAFE=4; ftp.login(‘anonymous’, ‘removed_email_address@domain.invalid’)}
Thread.critical = false
p th.value


#6

Hi,

In message “Re: Why does this code deadlock?”
on Wed, 18 Apr 2007 16:21:48 +0900, Nobuyoshi N.
removed_email_address@domain.invalid writes:

|At Wed, 18 Apr 2007 08:34:03 +0900,
|Bill K. wrote in [ruby-talk:248300]:
|> require ‘net/ftp’
|> Thread.abort_on_exception = true
|> ftp = Net::FTP.new(‘ftp.idsoftware.com’)
| th = Thread.new {$SAFE=4; ftp.login(‘anonymous’, ‘removed_email_address@domain.invalid’)}
| Thread.critical = false
| p th.value

Who set Thread.critical? It should be fixed, I think.

          matz.

#7

From: “Yukihiro M.” removed_email_address@domain.invalid

| Thread.critical = false
| p th.value

Who set Thread.critical? It should be fixed, I think.

It seems to be something called by ftp.login: (I added some
sleeps to coordinate the two threads.)

require 'fastthread'
require 'net/ftp'
Thread.abort_on_exception = true
puts "1. crit=#{Thread.critical}"
ftp = Net::FTP.new('ftp.idsoftware.com')
puts "2. crit=#{Thread.critical}"
th = Thread.new {sleep(2); $SAFE=4; sleep(2); ftp.login('anonymous', 
'removed_email_address@domain.invalid')}
puts "3. crit=#{Thread.critical}"
sleep(3)
puts "4. crit=#{Thread.critical}"
sleep(2)
puts "5. crit=#{Thread.critical}"
# Thread.critical = false
p th.value

$ ruby -v ftp_safe_test.rb
ruby 1.8.6 (2007-03-16 patchlevel 3) [i686-linux]

  1. crit=false
  2. crit=false
  3. crit=false
  4. crit=false
  5. crit=true
    ftp_safe_test.rb:14:in `value’: Thread(0x525ad708): deadlock (fatal)
    from ftp_safe_test.rb:14

If I uncomment the `Thread.critical = false’, it prints:

$ ruby -v ftp_safe_test.rb
ruby 1.8.6 (2007-03-16 patchlevel 3) [i686-linux]

  1. crit=false
  2. crit=false
  3. crit=false
  4. crit=false
  5. crit=true
    /opt/lib/ruby/1.8/monitor.rb:284:in mon_acquire': Insecure: can't modify instance variable (SecurityError) from ftp_safe_test.rb:14:invalue’
    from ftp_safe_test.rb:14

This is in MonitorMixin::ConditionVariable:

def mon_acquire(queue)
while @mon_owner && @mon_owner != Thread.current
queue.push(Thread.current)
Thread.stop
Thread.critical = true
end
@mon_owner = Thread.current
end

It looks like several methods in that class may want
ensure blocks enforcing their final `Thread.critical = false’
statements? Such as mon_try_enter, mon_enter, mon_exit,
etc.

Regards,

Bill


#8

On 4/18/07, Bill K. removed_email_address@domain.invalid wrote:

|> ftp = Net::FTP.new(‘ftp.idsoftware.com’)
require ‘fastthread’
puts “5. crit=#{Thread.critical}”

  1. crit=true
  2. crit=false
    while @mon_owner && @mon_owner != Thread.current
    queue.push(Thread.current)
    Thread.stop
    Thread.critical = true
    end
    @mon_owner = Thread.current
    end

ftp.login calls synchronize in thread.rb, which calls lock in
thread.rb, which sets critical:

def lock
while (Thread.critical = true; @locked)
@waiting.push Thread.current
Thread.stop
end
@locked = true
Thread.critical = false
self
end

Maybe I’m barking up the wrong tree.


#9

From: “Leslie V.” removed_email_address@domain.invalid

Thread.critical = false
self
end

Hmm. Not shure if this has changed, but in the 1.8.6 sources,
net/ftp uses MonitorMixin:

require “monitor”

module Net

class FTP
include MonitorMixin

...

end
end

Tracing the execution with a set_trace_func shows
Net::FTP#login -> MonitorMixin#synchronize -> MonitorMixin#mon_enter
setting Thread.critical:

call /opt/lib/ruby/1.8/net/ftp.rb:371      login Net::FTP
line /opt/lib/ruby/1.8/net/ftp.rb:372      login Net::FTP
line /opt/lib/ruby/1.8/net/ftp.rb:372      login Net::FTP

c-call /opt/lib/ruby/1.8/net/ftp.rb:372 == String
c-return /opt/lib/ruby/1.8/net/ftp.rb:372 == String
c-call /opt/lib/ruby/1.8/net/ftp.rb:372 == String
c-return /opt/lib/ruby/1.8/net/ftp.rb:372 == String
line /opt/lib/ruby/1.8/net/ftp.rb:376 login Net::FTP
line /opt/lib/ruby/1.8/net/ftp.rb:377 login Net::FTP
call /opt/lib/ruby/1.8/monitor.rb:235 synchronize MonitorMixin
line /opt/lib/ruby/1.8/monitor.rb:236 synchronize MonitorMixin
call /opt/lib/ruby/1.8/monitor.rb:209 mon_enter MonitorMixin
line /opt/lib/ruby/1.8/monitor.rb:210 mon_enter MonitorMixin
c-call /opt/lib/ruby/1.8/monitor.rb:210 critical= Thread
c-return /opt/lib/ruby/1.8/monitor.rb:210 critical= Thread
line /opt/lib/ruby/1.8/monitor.rb:211 mon_enter MonitorMixin
call /opt/lib/ruby/1.8/monitor.rb:278 mon_acquire MonitorMixin
line /opt/lib/ruby/1.8/monitor.rb:279 mon_acquire MonitorMixin
line /opt/lib/ruby/1.8/monitor.rb:284 mon_acquire MonitorMixin
c-call /opt/lib/ruby/1.8/monitor.rb:284 current Thread
c-return /opt/lib/ruby/1.8/monitor.rb:284 current Thread
c-call /opt/lib/ruby/1.8/monitor.rb:284 new Class
c-call /opt/lib/ruby/1.8/monitor.rb:284 initialize Exception
c-return /opt/lib/ruby/1.8/monitor.rb:284 initialize Exception
c-return /opt/lib/ruby/1.8/monitor.rb:284 new Class
c-call /opt/lib/ruby/1.8/monitor.rb:284 backtrace Exception
c-return /opt/lib/ruby/1.8/monitor.rb:284 backtrace Exception
c-call /opt/lib/ruby/1.8/monitor.rb:284 set_backtrace Exception
c-return /opt/lib/ruby/1.8/monitor.rb:284 set_backtrace Exception
raise /opt/lib/ruby/1.8/monitor.rb:284 mon_acquire MonitorMixin
return /opt/lib/ruby/1.8/monitor.rb:279 mon_acquire MonitorMixin
return /opt/lib/ruby/1.8/monitor.rb:210 mon_enter MonitorMixin
return /opt/lib/ruby/1.8/monitor.rb:236 synchronize MonitorMixin
return /opt/lib/ruby/1.8/net/ftp.rb:372 login Net::FTP

Regards,

Bill


#10

Hi,

In message “Re: Why does this code deadlock?”
on Thu, 19 Apr 2007 13:20:15 +0900, Nobuyoshi N.
removed_email_address@domain.invalid writes:

|> Who set Thread.critical? It should be fixed, I think.
|
|mon_acquire called from mon_enter fails because of
|SecurityError, and leaves Thread.critical true.

Could you commit the patch, please?

          matz.

#11

Hi,

At Wed, 18 Apr 2007 16:53:25 +0900,
Yukihiro M. wrote in [ruby-talk:248325]:

|At Wed, 18 Apr 2007 08:34:03 +0900,
|Bill K. wrote in [ruby-talk:248300]:
|> require ‘net/ftp’
|> Thread.abort_on_exception = true
|> ftp = Net::FTP.new(‘ftp.idsoftware.com’)
| th = Thread.new {$SAFE=4; ftp.login(‘anonymous’, ‘removed_email_address@domain.invalid’)}
| Thread.critical = false
| p th.value

Who set Thread.critical? It should be fixed, I think.

mon_acquire called from mon_enter fails because of
SecurityError, and leaves Thread.critical true.

Index: lib/monitor.rb

— lib/monitor.rb (revision 12191)
+++ lib/monitor.rb (working copy)
@@ -106,12 +106,15 @@ def wait(timeout = nil)
ensure
Thread.critical = true

  • if timer && timer.alive?
  • Thread.kill(timer)
  • begin
  • if timer && timer.alive?
  •  Thread.kill(timer)
    
  • end
  • if @waiters.include?(Thread.current) # interrupted?
  •  @waiters.delete(Thread.current)
    
  • end
  • @monitor.instance_eval {mon_enter_for_cond(count)}
  • ensure
  • Thread.critical = false
    end
  • if @waiters.include?(Thread.current) # interrupted?
  • @waiters.delete(Thread.current)
  • end
  •    @monitor.instance_eval {mon_enter_for_cond(count)}
    
  • Thread.critical = false
    end
    end
    @@ -211,4 +214,5 @@ def mon_enter
    mon_acquire(@mon_entering_queue)
    @mon_count += 1
  • ensure
    Thread.critical = false
    end
    @@ -300,6 +304,7 @@ def mon_exit_for_cond
    count = @mon_count
    @mon_count = 0
  • mon_release
    return count
  • ensure
  • mon_release
    end
    end