Why does this code deadlock?

kelleysislander · April 18, 2007, 1:34am

Hi,

I’m getting a deadlock on this small program, on both Windows
and Linux:

require 'net/ftp'
Thread.abort_on_exception = true
ftp = Net::FTP.new('ftp.idsoftware.com')
p( Thread.new {$SAFE=4; ftp.login('anonymous', '[email protected]')}.value )

ruby -v ftp_safe_test.rb

ruby 1.8.4 (2005-12-24) [i386-mswin32]
ftp_safe_test.rb:4:in `value’: Thread(0x27855c8): deadlock (fatal)
from ftp_safe_test.rb:4

$ ruby -v safe_ftp_test.rb
ruby 1.8.4 (2005-12-24) [i686-linux]
safe_ftp_test.rb:4:in `value’: Thread(0x4f060748): deadlock (fatal)
from safe_ftp_test.rb:4

Now, I didn’t necessarily expect the code to WORK at $SAFE=4,
but I was anticipating a SecurityError rather than a deadlock.

(I’ve tried it with Thread.abort_on_exception both true and
false, same result.)

Any thoughts?

Thanks,

Regards,

Bill

kelleysislander · April 18, 2007, 5:05am

Hi Bill-

On Apr 17, 2007, at 4:34 PM, Bill K. wrote:

ruby 1.8.4 (2005-12-24) [i686-linux]
Any thoughts?

Thanks,

Regards,

Bill

Are you using ruby 1.8.6 by chance? If so install the laets
fastthread gem and require it in your script to see if it fixes the
deadlock. Unfortunately the fastthread code in the ruby1.8.6 release
has some bugs that are fixed in the fastthread gem

Cheers-
– Ezra Z.
– Lead Rails Evangelist
– remove[email protected]
– Engine Y., Serious Rails Hosting
– (866) 518-YARD (9273)

kelleysislander · April 18, 2007, 8:03am

Hi Ezra,

has some bugs that are fixed in the fastthread gem
The above was 1.8.4 with the standard thread.rb.

Trying ruby 1.8.6 (2007-03-16 patchlevel 3) [i686-linux]
…I still get the deadlock, with or without fastthread:

ruby -v ftp_safe_test.rb
ruby 1.8.6 (2007-03-16 patchlevel 3) [i686-linux]
ftp_safe_test.rb:5:in `value’: Thread(0x50df1704): deadlock (fatal)
from ftp_safe_test.rb:5

Regards,

Bill

kelleysislander · April 18, 2007, 9:32am

From: “Nobuyoshi N.” [email protected]

At Wed, 18 Apr 2007 08:34:03 +0900,
Bill K. wrote in [ruby-talk:248300]:

require ‘net/ftp’
Thread.abort_on_exception = true
ftp = Net::FTP.new(‘ftp.idsoftware.com’)
th = Thread.new {$SAFE=4; ftp.login(‘anonymous’, ‘[email protected]’)}
Thread.critical = false
p th.value

Nobu, you RULE!!!

Thanks!

Bill

kelleysislander · April 18, 2007, 9:24am

Hi,

At Wed, 18 Apr 2007 08:34:03 +0900,
Bill K. wrote in [ruby-talk:248300]:

require ‘net/ftp’
Thread.abort_on_exception = true
ftp = Net::FTP.new(‘ftp.idsoftware.com’)
th = Thread.new {$SAFE=4; ftp.login(‘anonymous’, ‘[email protected]’)}
Thread.critical = false
p th.value

kelleysislander · April 18, 2007, 9:55am

Hi,

In message “Re: Why does this code deadlock?”
on Wed, 18 Apr 2007 16:21:48 +0900, Nobuyoshi N.
[email protected] writes:

|At Wed, 18 Apr 2007 08:34:03 +0900,
|Bill K. wrote in [ruby-talk:248300]:
|> require ‘net/ftp’
|> Thread.abort_on_exception = true
|> ftp = Net::FTP.new(‘ftp.idsoftware.com’)
| th = Thread.new {$SAFE=4; ftp.login(‘anonymous’, ‘[email protected]’)}
| Thread.critical = false
| p th.value

Who set Thread.critical? It should be fixed, I think.

          matz.

kelleysislander · April 18, 2007, 11:03am

From: “Yukihiro M.” [email protected]

| Thread.critical = false
| p th.value

Who set Thread.critical? It should be fixed, I think.

It seems to be something called by ftp.login: (I added some
sleeps to coordinate the two threads.)

require 'fastthread'
require 'net/ftp'
Thread.abort_on_exception = true
puts "1. crit=#{Thread.critical}"
ftp = Net::FTP.new('ftp.idsoftware.com')
puts "2. crit=#{Thread.critical}"
th = Thread.new {sleep(2); $SAFE=4; sleep(2); ftp.login('anonymous', 
'[email protected]')}
puts "3. crit=#{Thread.critical}"
sleep(3)
puts "4. crit=#{Thread.critical}"
sleep(2)
puts "5. crit=#{Thread.critical}"
# Thread.critical = false
p th.value

$ ruby -v ftp_safe_test.rb
ruby 1.8.6 (2007-03-16 patchlevel 3) [i686-linux]

crit=false
crit=false
crit=false
crit=false
crit=true
ftp_safe_test.rb:14:in `value’: Thread(0x525ad708): deadlock (fatal)
from ftp_safe_test.rb:14

If I uncomment the `Thread.critical = false’, it prints:

$ ruby -v ftp_safe_test.rb
ruby 1.8.6 (2007-03-16 patchlevel 3) [i686-linux]

crit=false
crit=false
crit=false
crit=false
crit=true
/opt/lib/ruby/1.8/monitor.rb:284:in mon_acquire': Insecure: can't modify instance variable (SecurityError) from ftp_safe_test.rb:14:invalue’
from ftp_safe_test.rb:14

This is in MonitorMixin::ConditionVariable:

def mon_acquire(queue)
while @mon_owner && @mon_owner != Thread.current
queue.push(Thread.current)
Thread.stop
Thread.critical = true
end
@mon_owner = Thread.current
end

It looks like several methods in that class may want
ensure blocks enforcing their final `Thread.critical = false’
statements? Such as mon_try_enter, mon_enter, mon_exit,
etc.

Regards,

Bill

kelleysislander · April 18, 2007, 10:16pm

On 4/18/07, Bill K. [email protected] wrote:

|> ftp = Net::FTP.new(‘ftp.idsoftware.com’)
require ‘fastthread’
puts “5. crit=#{Thread.critical}”

crit=true

crit=false
while @mon_owner && @mon_owner != Thread.current
queue.push(Thread.current)
Thread.stop
Thread.critical = true
end
@mon_owner = Thread.current
end

ftp.login calls synchronize in thread.rb, which calls lock in
thread.rb, which sets critical:

def lock
while (Thread.critical = true; @locked)
@waiting.push Thread.current
Thread.stop
end
@locked = true
Thread.critical = false
self
end

Maybe I’m barking up the wrong tree.

kelleysislander · April 18, 2007, 11:02pm

From: “Leslie V.” [email protected]

Thread.critical = false
self
end

Hmm. Not shure if this has changed, but in the 1.8.6 sources,
net/ftp uses MonitorMixin:

require “monitor”

module Net
…
class FTP
include MonitorMixin

...

end
end

Tracing the execution with a set_trace_func shows
Net::FTP#login -> MonitorMixin#synchronize -> MonitorMixin#mon_enter
setting Thread.critical:

call /opt/lib/ruby/1.8/net/ftp.rb:371      login Net::FTP
line /opt/lib/ruby/1.8/net/ftp.rb:372      login Net::FTP
line /opt/lib/ruby/1.8/net/ftp.rb:372      login Net::FTP

c-call /opt/lib/ruby/1.8/net/ftp.rb:372 == String
c-return /opt/lib/ruby/1.8/net/ftp.rb:372 == String
c-call /opt/lib/ruby/1.8/net/ftp.rb:372 == String
c-return /opt/lib/ruby/1.8/net/ftp.rb:372 == String
line /opt/lib/ruby/1.8/net/ftp.rb:376 login Net::FTP
line /opt/lib/ruby/1.8/net/ftp.rb:377 login Net::FTP
call /opt/lib/ruby/1.8/monitor.rb:235 synchronize MonitorMixin
line /opt/lib/ruby/1.8/monitor.rb:236 synchronize MonitorMixin
call /opt/lib/ruby/1.8/monitor.rb:209 mon_enter MonitorMixin
line /opt/lib/ruby/1.8/monitor.rb:210 mon_enter MonitorMixin
c-call /opt/lib/ruby/1.8/monitor.rb:210 critical= Thread
c-return /opt/lib/ruby/1.8/monitor.rb:210 critical= Thread
line /opt/lib/ruby/1.8/monitor.rb:211 mon_enter MonitorMixin
call /opt/lib/ruby/1.8/monitor.rb:278 mon_acquire MonitorMixin
line /opt/lib/ruby/1.8/monitor.rb:279 mon_acquire MonitorMixin
line /opt/lib/ruby/1.8/monitor.rb:284 mon_acquire MonitorMixin
c-call /opt/lib/ruby/1.8/monitor.rb:284 current Thread
c-return /opt/lib/ruby/1.8/monitor.rb:284 current Thread
c-call /opt/lib/ruby/1.8/monitor.rb:284 new Class
c-call /opt/lib/ruby/1.8/monitor.rb:284 initialize Exception
c-return /opt/lib/ruby/1.8/monitor.rb:284 initialize Exception
c-return /opt/lib/ruby/1.8/monitor.rb:284 new Class
c-call /opt/lib/ruby/1.8/monitor.rb:284 backtrace Exception
c-return /opt/lib/ruby/1.8/monitor.rb:284 backtrace Exception
c-call /opt/lib/ruby/1.8/monitor.rb:284 set_backtrace Exception
c-return /opt/lib/ruby/1.8/monitor.rb:284 set_backtrace Exception
raise /opt/lib/ruby/1.8/monitor.rb:284 mon_acquire MonitorMixin
return /opt/lib/ruby/1.8/monitor.rb:279 mon_acquire MonitorMixin
return /opt/lib/ruby/1.8/monitor.rb:210 mon_enter MonitorMixin
return /opt/lib/ruby/1.8/monitor.rb:236 synchronize MonitorMixin
return /opt/lib/ruby/1.8/net/ftp.rb:372 login Net::FTP

Regards,

Bill

kelleysislander · April 19, 2007, 7:17am

Hi,

In message “Re: Why does this code deadlock?”
on Thu, 19 Apr 2007 13:20:15 +0900, Nobuyoshi N.
[email protected] writes:

|> Who set Thread.critical? It should be fixed, I think.
|
|mon_acquire called from mon_enter fails because of
|SecurityError, and leaves Thread.critical true.

Could you commit the patch, please?

          matz.

kelleysislander · April 19, 2007, 6:20am

Hi,

At Wed, 18 Apr 2007 16:53:25 +0900,
Yukihiro M. wrote in [ruby-talk:248325]:

|At Wed, 18 Apr 2007 08:34:03 +0900,
|Bill K. wrote in [ruby-talk:248300]:
|> require ‘net/ftp’
|> Thread.abort_on_exception = true
|> ftp = Net::FTP.new(‘ftp.idsoftware.com’)
| th = Thread.new {$SAFE=4; ftp.login(‘anonymous’, ‘[email protected]’)}
| Thread.critical = false
| p th.value

Who set Thread.critical? It should be fixed, I think.

mon_acquire called from mon_enter fails because of
SecurityError, and leaves Thread.critical true.

Index: lib/monitor.rb

— lib/monitor.rb (revision 12191)
+++ lib/monitor.rb (working copy)
@@ -106,12 +106,15 @@ def wait(timeout = nil)
ensure
Thread.critical = true

if timer && timer.alive?
Thread.kill(timer)

begin
if timer && timer.alive?
```
 Thread.kill(timer)
```
end
if @waiters.include?(Thread.current) # interrupted?
```
 @waiters.delete(Thread.current)
```
end
@monitor.instance_eval {mon_enter_for_cond(count)}
ensure
Thread.critical = false
end

if @waiters.include?(Thread.current) # interrupted?
@waiters.delete(Thread.current)
end

   @monitor.instance_eval {mon_enter_for_cond(count)}

Thread.critical = false
end
end
@@ -211,4 +214,5 @@ def mon_enter
mon_acquire(@mon_entering_queue)
@mon_count += 1

ensure
Thread.critical = false
end
@@ -300,6 +304,7 @@ def mon_exit_for_cond
count = @mon_count
@mon_count = 0

mon_release
return count

ensure
mon_release
end
end