Forum: Ruby Why does this code deadlock?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
4feed660d3728526797edeb4f0467384?d=identicon&s=25 Bill Kelly (Guest)
on 2007-04-18 01:34
(Received via mailing list)
Hi,

I'm getting a deadlock on this small program, on both Windows
and Linux:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
require 'net/ftp'
Thread.abort_on_exception = true
ftp = Net::FTP.new('ftp.idsoftware.com')
p( Thread.new {$SAFE=4; ftp.login('anonymous', 'abc@def.com')}.value )

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


  # ruby -v ftp_safe_test.rb
  ruby 1.8.4 (2005-12-24) [i386-mswin32]
  ftp_safe_test.rb:4:in `value': Thread(0x27855c8): deadlock (fatal)
          from ftp_safe_test.rb:4


  $ ruby -v safe_ftp_test.rb
  ruby 1.8.4 (2005-12-24) [i686-linux]
  safe_ftp_test.rb:4:in `value': Thread(0x4f060748): deadlock (fatal)
          from safe_ftp_test.rb:4


Now, I didn't necessarily expect the code to WORK at $SAFE=4,
but I was anticipating a SecurityError rather than a deadlock.

(I've tried it with Thread.abort_on_exception both true and
false, same result.)

Any thoughts?

Thanks,

Regards,

Bill
6076c22b65b36f5d75c30bdcfb2fda85?d=identicon&s=25 Ezra Zygmuntowicz (Guest)
on 2007-04-18 05:05
(Received via mailing list)
Hi Bill-

On Apr 17, 2007, at 4:34 PM, Bill Kelly wrote:

>
>  ruby 1.8.4 (2005-12-24) [i686-linux]
> Any thoughts?
>
> Thanks,
>
> Regards,
>
> Bill

  Are you using ruby 1.8.6 by chance? If so install the laets
fastthread gem and require it in your script to see if it fixes the
deadlock. Unfortunately the fastthread code in the ruby1.8.6 release
has some bugs that are fixed in the fastthread gem

Cheers-
-- Ezra Zygmuntowicz
-- Lead Rails Evangelist
-- ez@engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)
4feed660d3728526797edeb4f0467384?d=identicon&s=25 Bill Kelly (Guest)
on 2007-04-18 08:03
(Received via mailing list)
Hi Ezra,

>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
> has some bugs that are fixed in the fastthread gem
The above was 1.8.4 with the standard thread.rb.

Trying ruby 1.8.6 (2007-03-16 patchlevel 3) [i686-linux]
...I still get the deadlock, with or without fastthread:

  ruby -v ftp_safe_test.rb
  ruby 1.8.6 (2007-03-16 patchlevel 3) [i686-linux]
  ftp_safe_test.rb:5:in `value': Thread(0x50df1704): deadlock (fatal)
          from ftp_safe_test.rb:5


Regards,

Bill
F1d6cc2b735bfd82c8773172da2aeab9?d=identicon&s=25 Nobuyoshi Nakada (Guest)
on 2007-04-18 09:24
(Received via mailing list)
Hi,

At Wed, 18 Apr 2007 08:34:03 +0900,
Bill Kelly wrote in [ruby-talk:248300]:
> require 'net/ftp'
> Thread.abort_on_exception = true
> ftp = Net::FTP.new('ftp.idsoftware.com')
  th = Thread.new {$SAFE=4; ftp.login('anonymous', 'abc@def.com')}
  Thread.critical = false
  p th.value
4feed660d3728526797edeb4f0467384?d=identicon&s=25 Bill Kelly (Guest)
on 2007-04-18 09:32
(Received via mailing list)
From: "Nobuyoshi Nakada" <nobu@ruby-lang.org>
>
> At Wed, 18 Apr 2007 08:34:03 +0900,
> Bill Kelly wrote in [ruby-talk:248300]:
>> require 'net/ftp'
>> Thread.abort_on_exception = true
>> ftp = Net::FTP.new('ftp.idsoftware.com')
>  th = Thread.new {$SAFE=4; ftp.login('anonymous', 'abc@def.com')}
>  Thread.critical = false
>  p th.value

Nobu, you RULE!!!  :D


Thanks!

Bill
0ec4920185b657a03edf01fff96b4e9b?d=identicon&s=25 Yukihiro Matsumoto (Guest)
on 2007-04-18 09:55
(Received via mailing list)
Hi,

In message "Re: Why does this code deadlock?"
    on Wed, 18 Apr 2007 16:21:48 +0900, Nobuyoshi Nakada
<nobu@ruby-lang.org> writes:

|At Wed, 18 Apr 2007 08:34:03 +0900,
|Bill Kelly wrote in [ruby-talk:248300]:
|> require 'net/ftp'
|> Thread.abort_on_exception = true
|> ftp = Net::FTP.new('ftp.idsoftware.com')
|  th = Thread.new {$SAFE=4; ftp.login('anonymous', 'abc@def.com')}
|  Thread.critical = false
|  p th.value

Who set Thread.critical?  It should be fixed, I think.

              matz.
4feed660d3728526797edeb4f0467384?d=identicon&s=25 Bill Kelly (Guest)
on 2007-04-18 11:03
(Received via mailing list)
From: "Yukihiro Matsumoto" <matz@ruby-lang.org>
> |  Thread.critical = false
> |  p th.value
>
> Who set Thread.critical?  It should be fixed, I think.

It seems to be something called by ftp.login:  (I added some
sleeps to coordinate the two threads.)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
require 'fastthread'
require 'net/ftp'
Thread.abort_on_exception = true
puts "1. crit=#{Thread.critical}"
ftp = Net::FTP.new('ftp.idsoftware.com')
puts "2. crit=#{Thread.critical}"
th = Thread.new {sleep(2); $SAFE=4; sleep(2); ftp.login('anonymous',
'abc@def.com')}
puts "3. crit=#{Thread.critical}"
sleep(3)
puts "4. crit=#{Thread.critical}"
sleep(2)
puts "5. crit=#{Thread.critical}"
# Thread.critical = false
p th.value
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

$ ruby -v ftp_safe_test.rb
ruby 1.8.6 (2007-03-16 patchlevel 3) [i686-linux]
1. crit=false
2. crit=false
3. crit=false
4. crit=false
5. crit=true
ftp_safe_test.rb:14:in `value': Thread(0x525ad708): deadlock (fatal)
        from ftp_safe_test.rb:14


If I uncomment the `Thread.critical = false', it prints:

$ ruby -v ftp_safe_test.rb
ruby 1.8.6 (2007-03-16 patchlevel 3) [i686-linux]
1. crit=false
2. crit=false
3. crit=false
4. crit=false
5. crit=true
/opt/lib/ruby/1.8/monitor.rb:284:in `mon_acquire': Insecure: can't
modify instance variable (SecurityError)
        from ftp_safe_test.rb:14:in `value'
        from ftp_safe_test.rb:14


This is in MonitorMixin::ConditionVariable:

  def mon_acquire(queue)
    while @mon_owner && @mon_owner != Thread.current
      queue.push(Thread.current)
      Thread.stop
      Thread.critical = true
    end
    @mon_owner = Thread.current
  end


It looks like several methods in that class may want
ensure blocks enforcing their final `Thread.critical = false'
statements?  Such as mon_try_enter, mon_enter, mon_exit,
etc.


Regards,

Bill
37ee5fa90f5eaeef62553629382497f7?d=identicon&s=25 Leslie Viljoen (Guest)
on 2007-04-18 22:16
(Received via mailing list)
On 4/18/07, Bill Kelly <billk@cts.com> wrote:
> > |> ftp = Net::FTP.new('ftp.idsoftware.com')
> require 'fastthread'
> puts "5. crit=#{Thread.critical}"
> 5. crit=true
> 3. crit=false
>     while @mon_owner && @mon_owner != Thread.current
>       queue.push(Thread.current)
>       Thread.stop
>       Thread.critical = true
>     end
>     @mon_owner = Thread.current
>   end

ftp.login calls synchronize in thread.rb, which calls lock in
thread.rb, which sets critical:

def lock
   while (Thread.critical = true; @locked)
     @waiting.push Thread.current
     Thread.stop
   end
   @locked = true
   Thread.critical = false
   self
end

Maybe I'm barking up the wrong tree.
4feed660d3728526797edeb4f0467384?d=identicon&s=25 Bill Kelly (Guest)
on 2007-04-18 23:02
(Received via mailing list)
From: "Leslie Viljoen" <leslieviljoen@gmail.com>
>   Thread.critical = false
>   self
> end

Hmm. Not shure if this has changed, but in the 1.8.6 sources,
net/ftp uses MonitorMixin:

require "monitor"

module Net
  ...
  class FTP
    include MonitorMixin

    ...
  end
end

Tracing the execution with a set_trace_func shows
Net::FTP#login -> MonitorMixin#synchronize -> MonitorMixin#mon_enter
setting Thread.critical:

    call /opt/lib/ruby/1.8/net/ftp.rb:371      login Net::FTP
    line /opt/lib/ruby/1.8/net/ftp.rb:372      login Net::FTP
    line /opt/lib/ruby/1.8/net/ftp.rb:372      login Net::FTP
  c-call /opt/lib/ruby/1.8/net/ftp.rb:372         ==   String
c-return /opt/lib/ruby/1.8/net/ftp.rb:372         ==   String
  c-call /opt/lib/ruby/1.8/net/ftp.rb:372         ==   String
c-return /opt/lib/ruby/1.8/net/ftp.rb:372         ==   String
    line /opt/lib/ruby/1.8/net/ftp.rb:376      login Net::FTP
    line /opt/lib/ruby/1.8/net/ftp.rb:377      login Net::FTP
    call /opt/lib/ruby/1.8/monitor.rb:235 synchronize MonitorMixin
    line /opt/lib/ruby/1.8/monitor.rb:236 synchronize MonitorMixin
    call /opt/lib/ruby/1.8/monitor.rb:209  mon_enter MonitorMixin
    line /opt/lib/ruby/1.8/monitor.rb:210  mon_enter MonitorMixin
  c-call /opt/lib/ruby/1.8/monitor.rb:210  critical=   Thread
c-return /opt/lib/ruby/1.8/monitor.rb:210  critical=   Thread
    line /opt/lib/ruby/1.8/monitor.rb:211  mon_enter MonitorMixin
    call /opt/lib/ruby/1.8/monitor.rb:278 mon_acquire MonitorMixin
    line /opt/lib/ruby/1.8/monitor.rb:279 mon_acquire MonitorMixin
    line /opt/lib/ruby/1.8/monitor.rb:284 mon_acquire MonitorMixin
  c-call /opt/lib/ruby/1.8/monitor.rb:284    current   Thread
c-return /opt/lib/ruby/1.8/monitor.rb:284    current   Thread
  c-call /opt/lib/ruby/1.8/monitor.rb:284        new    Class
  c-call /opt/lib/ruby/1.8/monitor.rb:284 initialize Exception
c-return /opt/lib/ruby/1.8/monitor.rb:284 initialize Exception
c-return /opt/lib/ruby/1.8/monitor.rb:284        new    Class
  c-call /opt/lib/ruby/1.8/monitor.rb:284  backtrace Exception
c-return /opt/lib/ruby/1.8/monitor.rb:284  backtrace Exception
  c-call /opt/lib/ruby/1.8/monitor.rb:284 set_backtrace Exception
c-return /opt/lib/ruby/1.8/monitor.rb:284 set_backtrace Exception
   raise /opt/lib/ruby/1.8/monitor.rb:284 mon_acquire MonitorMixin
  return /opt/lib/ruby/1.8/monitor.rb:279 mon_acquire MonitorMixin
  return /opt/lib/ruby/1.8/monitor.rb:210  mon_enter MonitorMixin
  return /opt/lib/ruby/1.8/monitor.rb:236 synchronize MonitorMixin
  return /opt/lib/ruby/1.8/net/ftp.rb:372      login Net::FTP


Regards,

Bill
F1d6cc2b735bfd82c8773172da2aeab9?d=identicon&s=25 Nobuyoshi Nakada (Guest)
on 2007-04-19 06:20
(Received via mailing list)
Hi,

At Wed, 18 Apr 2007 16:53:25 +0900,
Yukihiro Matsumoto wrote in [ruby-talk:248325]:
> |At Wed, 18 Apr 2007 08:34:03 +0900,
> |Bill Kelly wrote in [ruby-talk:248300]:
> |> require 'net/ftp'
> |> Thread.abort_on_exception = true
> |> ftp = Net::FTP.new('ftp.idsoftware.com')
> |  th = Thread.new {$SAFE=4; ftp.login('anonymous', 'abc@def.com')}
> |  Thread.critical = false
> |  p th.value
>
> Who set Thread.critical?  It should be fixed, I think.

mon_acquire called from mon_enter fails because of
SecurityError, and leaves Thread.critical true.


Index: lib/monitor.rb
===================================================================
--- lib/monitor.rb  (revision 12191)
+++ lib/monitor.rb  (working copy)
@@ -106,12 +106,15 @@     def wait(timeout = nil)
       ensure
   Thread.critical = true
-  if timer && timer.alive?
-    Thread.kill(timer)
+  begin
+    if timer && timer.alive?
+      Thread.kill(timer)
+    end
+    if @waiters.include?(Thread.current)  # interrupted?
+      @waiters.delete(Thread.current)
+    end
+    @monitor.instance_eval {mon_enter_for_cond(count)}
+  ensure
+    Thread.critical = false
   end
-  if @waiters.include?(Thread.current)  # interrupted?
-    @waiters.delete(Thread.current)
-  end
-        @monitor.instance_eval {mon_enter_for_cond(count)}
-  Thread.critical = false
       end
     end
@@ -211,4 +214,5 @@   def mon_enter
     mon_acquire(@mon_entering_queue)
     @mon_count += 1
+  ensure
     Thread.critical = false
   end
@@ -300,6 +304,7 @@   def mon_exit_for_cond
     count = @mon_count
     @mon_count = 0
-    mon_release
     return count
+  ensure
+    mon_release
   end
 end
0ec4920185b657a03edf01fff96b4e9b?d=identicon&s=25 Yukihiro Matsumoto (Guest)
on 2007-04-19 07:17
(Received via mailing list)
Hi,

In message "Re: Why does this code deadlock?"
    on Thu, 19 Apr 2007 13:20:15 +0900, Nobuyoshi Nakada
<nobu@ruby-lang.org> writes:

|> Who set Thread.critical?  It should be fixed, I think.
|
|mon_acquire called from mon_enter fails because of
|SecurityError, and leaves Thread.critical true.

Could you commit the patch, please?

              matz.
This topic is locked and can not be replied to.