Review requested: Binding.call_stack(), Binding.of_caller()-

[Dumaiu]

Hi, all–

I’d like to submit for evaluation an elaboration I’ve made on the
famous ‘Binding.of_caller()’ of Florian Groß. I have tried to follow
the the description of the interface for Binding.call_stack() suggested
in ruby-talk 12097, “RCR: replacing ‘caller’”, although in standard
Ruby instead of an extension.
The technique of using set_trace_func() to capture a Binding one
stack frame up has been around at least since 2001 [1]. Here I raise
an exception and ride it out to main scope, catching Bindings as the
stack unrolls. File and line info is taken from caller() as usual, and
the input from the two traces is knitted together.

#<call_stack.rb>
=begin
Binding::call_stack() returns a CallStack object–an Array
subclass–containing one StackFrame object per calling frame. Unless
specified otherwise, the last StackFrame will contain information from
the outermost scope, the ‘main’ quasi-object. A StackFrame, like a
Struct, is a bundle of attributes, as follows:

‘object’: Name of module|class in which the calling function is
defined. Will be ‘main’ if call_stack() is invoked at global scope.
‘method’: Name of method whose calling scope we currently inhabit.
Will be nil at global scope.
‘binding’: Binding within calling method’s scope. This is the really
useful one, carrying on Binding.of_caller()'s performance.
‘file’: Current file.
‘line’: Number of line at which current method was called, as
returned by Kernel::caller().

As far as I have understood the logic behind caller(), I have tried to
integrate the Binding retrieval smoothly thereinto. Here are some
examples:

<stacktest.rb>

require ‘call_stack’

puts Binding::call_stack.first.to_a # Retrieve StackFrame and convert
to array for output

</stacktest.rb>

produces:

main
nil
#Binding:0xb16fa4f00l
stacktest.rb
4

Not interesting. But given this

<stacktest.rb>

require ‘call_stack’

main_local = ‘main scope’

class C
def first()
first_local = ‘in C.first()’
return second
end

def second()
  second_local = 'in C.second()'
  return Binding::call_stack # <---- Nested call
end

end# C

Print CallStack array:

C.new.first().each { |frame|
# Dump current frame:
puts “Class: ‘%s’” % [frame.object]
puts “Method name: ‘%s’” % [frame.method]
puts “Filename: ‘%s’” % [frame.file]
puts “Line no.: %d” % [frame.line]
puts “Locals in binding: #{eval ‘local_variables()’,
frame.binding}\n\n”
}# each

</stacktest.rb>

we see

Class: ‘C’
Method name: ‘second’
Filename: ‘stacktest.rb’
Line no.: 18
Locals in binding: second_local

Class: ‘C’
Method name: ‘first’
Filename: ‘stacktest.rb’
Line no.: 13
Locals in binding: first_local

Class: ‘main’
Method name: ‘’
Filename: ‘stacktest.rb’
Line no.: 22
Locals in binding: main_local

The first frame holds the environment in which call_stack() was itself
called, that is, the method C#second(). The second frame holds it’s
calling environment, C#first(), and the third frame, global binding.
The eval()'d calls to Kernel::local_variables() suggest how these
captured bindings can be used to mess with other people’s stuff.
Three configuration attributes control the classes used internally:

• Binding::CallStack::frame_class : The class held here is instantiated
  into containers for the data returned by call_stack(). Default value
  is Binding::StackFrame.
• Binding::CallStack::tracing_exception : The exception thrown to
  unroll the stack. Because it arrests itself, this is one exception we
  don’t want to be caught, to which end its type is generated from the
  system clock when the file is loaded. But you can change it.
• Binding::stack_class : The class instantiated for return by
  call_stack(). In this case, it must quack like an Array with two extra
  methods–call(), invoked each time a frame is added; and ret(), invoked
  just before the object is returned.

=end

BINDING

class Binding
require ‘test/unit/assertions’
class << self
include Test::Unit::Assertions # for Binding::call_stack
end

*************** STACKFRAME ***************

class StackFrame
include Test::Unit::Assertions

# TODO: Make read-only:
attr_accessor :object
attr_accessor :method
attr_accessor :binding
attr_accessor :file
attr_accessor :line

# to_a(): Return data as new array.
def to_a()
  return [ self.object,
    self.method,
    self.binding,
    self.file,
    self.line
  ]
end# to_a()

# to_s(): Return join()ed string.
def to_s()
  return to_a().to_s()
end# to_s()

# inspect(): Return as array of values.
def inspect()
  return to_a().inspect()
end# inspect()

# to_h(): Return in key/value form.  Note: A pair will only exist

if accessor assignment has been used.
def to_h()
h = Hash::new
var = nil # temp
instance_variables().each { |var|
var =~ /^@ (\w+) $/x
assert( $1 )
h[$1] = instance_variable_get(var)
}# each

  return h
end# to_h()

end# StackFrame

*************** CALLSTACK ***************

Actually an Array, not a Stack proper.

class CallStack < Array
include Test::Unit::Assertions
require ‘date’

@version = '0.0.1'
class << self
  attr_reader :version
end


# Class instance vars:
@frame_class = Binding::StackFrame
@tracing_exception = "xCallStack: #{DateTime.now.to_s}".intern()
class << self
  attr_accessor :frame_class
  attr_accessor :tracing_exception
end

# Ctor: call_stack() passes its binding hither, in case we need to

read locals therefrom (a utility device). Read-only!
# Note: A null value for call_stack_binding will yield an empty
object.
def initialize( call_stack_binding = nil )
if( call_stack_binding )
@call_stack_binding = call_stack_binding # Store for use in
call().

    push StackFrame::new() # Disposable: eliminated in ret().
  end# if
  # else empty
end# ctor()

# call(): Repeatedly used by call_stack() to add frames to the

array (hence the name).
# [ class, method, binding, file, line ]
def call( trace_event, trace_file, trace_line, trace_method,
trace_bind, trace_object )
assert( eval( “defined? aCallers”, @call_stack_binding ) ==
‘local-variable’ ) # Ensure we have the proper name for locvar in
call_stack().
aCallers = eval( “aCallers”, @call_stack_binding ) # temp to
avoid repeat eval() calls

  push StackFrame::new()
  # The nature of set_trace_func() is to return the class and

method name for the previous stack frame.
at(-2).object = trace_object
at(-2).method = trace_method

  last.binding = trace_bind
  aCallers.first =~ /^ ( [\w\.]+ ) : ( \d+ ) \D*/x
  assert( $1 && $2 )
  last.file = $1
  last.line = $2

  return self
end# call()

# ret(): Called after all iterations are finished, just before the

CallStack is passed back to the client.
def ret()
shift() # Remove placeholder first element.
assert( length > 0 )

  assert( eval( "defined? nCount", @call_stack_binding ) ==

‘local-variable’ ) # Ensure we have the proper name for locvar in
call_stack().

  # If the final frame isn't at main scope, depending on the

arguments to call_stack(), it’s also a placeholder and must be removed:
if( eval( “nCount”, @call_stack_binding ) )
pop()
else
# Flesh out final element:
last.object = ‘main’
last.method = nil
end# if

  return self
end# ret()

end# CallStack

*************** CALL_STACK ***************

call_stack(): Retrieve CallStack object (array of StackFrame

objects).
def Binding::call_stack( nSkipFrames = 0, nCount = nil )
# Validate vars:
[nSkipFrames, nCount].each { |var|
if( var && (!var.is_a?(Integer) || (var < 0 )) ) then raise(
ArgumentError, “Optional argument to call_stack() should be a
nonnegative Integer.” ); end
}# each

# Store temporarily to obviate reinvocation.  Needed below, and

also by the current implementation of class CallStack, so I opted to
store it here and pass the CallStack ctor this, call_stack()'s,
binding, to allow for greater flexibility if CallStack is extended.
aCallers = caller()

# If at toplevel or nSkipFrames is greater than the number of

frames available, /or/ the client explicitly requests zero frames, exit
early:
if( (aCallers.length-nSkipFrames <= 0) || (nCount == 0) ) then
return Binding::stack_class::new(); end

# Trim the first entry, the client function, which we don't want to

include; then omit the next nSkipFrames entries.

# If nCount sufficiently large, no reason to use it at all:
if( nCount && ( nSkipFrames+nCount >= aCallers.length) ) then

nCount = nil; end

# Trim end if nCount provided:
if( nCount )
  assert( nCount < aCallers.length )
  nCount += nSkipFrames+1 # If nCount must be used in its "proper"

function, we have to pad out the number of trace calls to one beyond
the specified, in order to receive object and method info (see
CallStack::call(), above).
aCallers.slice!(nCount … -1)
end# end

aCallStack = Binding::stack_class::new( binding() ) # Pass current

Binding, giving CallStack access to locals.
callcc { |cc|
# Hope springs eternal(!):
#old_tracefunc = get_trace_func()
set_trace_func(
proc {
# event, file, line, id, bind, classname
|event, *remaining_args|

      if( event == 'return' )
        # If frame capture yet enabled:
        if( nSkipFrames == 0 )
          # capture current:
          aCallStack.call( event, *remaining_args )
        else # skip to next
          nSkipFrames -= 1
        end# if

        aCallers.shift()

        # If at toplevel, end ride:
        if( aCallers.length == 0 )
          #set_trace_func( get_trace_func() )
          set_trace_func( nil )

          # -----> Stack unrolling finishes here:
          cc.call()
        end# if
      end# if
    }# proc
  )# set_trace_func

  # Stack unrolling begins here: ----->
  throw( Binding::CallStack::tracing_exception )
}# callcc

assert(nSkipFrames)

# Resume execution in client func:
aCallStack.ret()
return( aCallStack )

end# call_stack()

Class instance var:

@stack_class = Binding::CallStack
class << self
attr_accessor :stack_class
end
end# Binding

</call_stack.rb>

I would appreciate your input with regard to the following:

• The three class attributes mentioned above (and the version member)
  allow noninvasive tweaking, but I’m not sure whether I should be using
  class variables or class instance vars for them.
• I noticed that Herr Groß renders his script thread critical(). I
  know not enough about thread safety to incorporate this without dumbly
  parroting him.
• Use of set_trace_func() breaks compatibility with the debugger, and
  I haven’t had much luck with irb, either. Under what other
  circumstances do Binding.of_caller() and relatives not work well?

Also, my rdoc skills are rudimentary and there’s a sort of half-assed
Hungarian notation going on, because I haven’t really worked out a
style for myself. (
Finally: I’ve only been in Ruby for a little over a month, so if it
looks somewhere like I don’t know what I’m doing, that’s probably the
case.
If I have wasted your time, please don’t look back.

Yours,

Jonathan J-S

Off to bed.

• [1]: I’ve started thinking of this as the ‘red carpet idiom.’ Any
  takers?

[Dumaiu]

Wow, impressive. I can’t comment very much on the implementation, I’m
still trying to wrap my head around what you’ve done. Would you mind
writing a little bit about how you are using continuations to walk the
stack in the Binding::call_stack method? How does that interact with
set_trace_func and the throw at the bottom of the callcc block? (e.g.
throw( Binding::CallStack::tracing_exception ))

Looks pretty cool to me!

[Dumaiu]

Dumaiu wrote:

I’d like to submit for evaluation an elaboration I’ve made on the
famous ‘Binding.of_caller()’ of Florian GroÃ?. I have tried to follow
the the description of the interface for Binding.call_stack() suggested
in ruby-talk 12097, “RCR: replacing ‘caller’”, although in standard
Ruby instead of an extension.
The technique of using set_trace_func() to capture a Binding one
stack frame up has been around at least since 2001 [1]. Here I raise
an exception and ride it out to main scope, catching Bindings as the
stack unrolls. File and line info is taken from caller() as usual, and
the input from the two traces is knitted together.

Wow, that’s some pretty innovative, clever and great thinking going on
there. I haven’t even thought about doing this, yet. Nice idea.

However, I see a problem with it: If you throw() through methods that
have an ensure block the ensure block will get executed. This is
demonstrated in the attached example. (I also attached your code because
it was damaged in your mail by being wrapped.)

I thought we might be able to fly through ensure blocks without
executing them by raising the fatal exception, but it appears that Ruby
will even run them in that case…

If you can find a way of making it work correctly with the attached test
then you have an implementation that is a lot more powerful than mine
which means I can finally drop my implementation…

I would appreciate your input with regard to the following:

• The three class attributes mentioned above (and the version member)
  allow noninvasive tweaking, but I’m not sure whether I should be using
  class variables or class instance vars for them.

I’d just hard code the actual classes instead. I see no purpose in being
able to change the type of data it returns anyway. If you need to
convert it to another format, you can always just do that.

• I noticed that Herr GroÃ? renders his script thread critical(). I
  know not enough about thread safety to incorporate this without dumbly
  parroting him.

The trace func isn’t thread local, but you want it to be. The only way
of working around that is by temporarily stopping all other threads.

• Use of set_trace_func() breaks compatibility with the debugger, and
  I haven’t had much luck with irb, either. Under what other
  circumstances do Binding.of_caller() and relatives not work well?

When there already is a trace_func.

It’s also a problem when using -rtrace. Binding.of_caller ought to work
in IRB, but I’m not 100% sure.

Finally: I’ve only been in Ruby for a little over a month, so if it
looks somewhere like I don’t know what I’m doing, that’s probably the
case.

And you are sure you really want to get involved into this kind of black
magic already?

Well, it already seems to be paying off so who am I to ask?

[Dumaiu]

Hiya!

Sure, I'll talk some more.  It took *me* a while to wrap my head

around Florian G.‘s stuff. I just didn’t want to look like I was
trying to talk down to anyone.
As we know, Kernel::caller() returns an array of strings, and is
pretty frickin’ useless by itself. A lot of people, independent of one
another, have wanted a way to write library methods that can
“magically” access the scope in which they were invoked. Gross wrote a
library to do this, binding_of_caller.rb, which has been floating
around this newsgroup for a while; and if you haven’t seen it yet, you
shouldn’t have trouble finding it. The only way to generate Binding
objects automatically is set_trace_func(), so you’ll see it in every
attempt of this sort. You stick in a trace func that catches the
bindings from the ‘return’ events. Gross’s clever solution was to use
a continuation to mark a location, take a detour to collect Bindings,
and then resume. Scenario: you write a library function requiring
access to its caller’s binding. Your function calls his
Binding::of_caller(), puts a block into set_trace_func(), creates a
continuation, and allows itself to return. You have now moved back a
stack frame into your original function. The tracer notes this. Your
function then also returns immediately (enforced by
binding_of_caller.rb), and with the second ‘return’ event you get the
binding you’re after. Once the trace func sees this, it uses the
cached continuation to dive back into normal execution.
The necessity of Binding::of_caller() being a tail call bothered
me. Instead of

def library_func()
   bind = Binding::of_caller
   do_something( bind )
end

you must split your function into an outer and an inner component:

def library_func() # Header
   Binding::of_caller { |bind| library_func_impl(bind) } #

Continuation is only available as a block parameter.
# Nothing allowed here
end

def library_func_impl(bind) # Body
   do_something( bind )
end

That block is required. Not concealed enough! The only novelty in my
code is the use of an escape continuation, i.e., an exception, which
eliminates the tail-call thing: set_trace_func() sees nothing but a
pile of ‘return’ events until it reaches the top (bottom? top?) of the
stack, and then–just before the uncaught exception crashes the
program–the stored continuation is called. Roughly equivalent to the
previous example is

def library_func()
   bind = Binding::call_stack(1,1).first.binding
   do_something( bind )
end

call_stack() calls callcc(). callcc() calls set_trace_func() and
throws an exception. The exception unrolls the stack, triggering the
tracer automatically. The tracer returns flow to call_stack() and
then unsets itself. The real stuff is all in Binding::call_stack(),
the rest just ‘limbs and outward flourishes.’ I don’t know about you,
but I had to work very hard to understand Kernel::callcc(). It’s
spaghetti syntax, to be sure.
F.G.'s design is faster and optimal if you only need to regress one
frame. Once I realized I could get the whole stack, it seemed silly to
return an array of just Bindings only, so I made it more like I wished
Kernel::caller() had worked to begin with. The implementation is a
byzantine shot at balancing extensibility and efficiency. In
particular, the capability of passing start- and endpoints as
arguments, the last feature added and the least well tested,
complicated things.

Take care,

   -Jonathan

[Dumaiu]

Update. So I learn that using an exception as the unrolling
mechanism will trip any ensure() blocks it passes through: each will
get executed twice, once during and once after, and the client won’t
know why. Included is the patch–the entire file without the docs, for
simplicity’s sake. A row of asterisks marks the only addition.

#<call_stack.rb>

Pop open for a bit:

class Binding
require ‘test/unit/assertions’
class << self
include Test::Unit::Assertions # for Binding::call_stack
end

*************** STACKFRAME ***************

class StackFrame
include Test::Unit::Assertions

# TODO: Make read-only:
attr_accessor :object
attr_accessor :method
attr_accessor :binding
attr_accessor :file
attr_accessor :line

# to_a(): Return data as new array.
def to_a()
  return [ self.object,
    self.method,
    self.binding,
    self.file,
    self.line
  ]
end# to_a()

# to_s(): Return join()ed string.
def to_s()
  return to_a().to_s()
end# to_s()

# inspect(): Return as array of values.
def inspect()
  return to_a().inspect()
end# inspect()

# to_h(): Return in key/value form.  Note: A pair will only exist

if accessor assignment has been used.
def to_h()
h = Hash::new
var = nil # temp
instance_variables().each { |var|
var =~ /^@ (\w+) $/x
assert( $1 )
h[$1] = instance_variable_get(var)
}# each

  return h
end# to_h()

end# StackFrame

*************** CALLSTACK ***************

Actually an Array, not a Stack proper.

class CallStack < Array
include Test::Unit::Assertions
require ‘date’

@version = '0.1.1'
class << self
  attr_reader :version
end


# Class instance vars:
@frame_class = Binding::StackFrame
@tracing_exception = "xCallStack: #{DateTime.now.to_s}".intern()
class << self
  attr_accessor :frame_class
  attr_accessor :tracing_exception
end

# Ctor: call_stack() passes its binding hither, in case we need to

read locals therefrom (a utility device). Read-only!
# Note: A null value for call_stack_binding will yield an empty
object.
def initialize( call_stack_binding = nil )
if( call_stack_binding )
@call_stack_binding = call_stack_binding # Store for use in
call().

    push StackFrame::new() # Disposable: eliminated in ret().
  end# if
  # else empty
end# ctor()

# call(): Repeatedly used by call_stack() to add frames to the

array (hence the name).
# [ class, method, binding, file, line ]
def call( trace_event, trace_file, trace_line, trace_method,
trace_bind, trace_object )
assert( eval( “defined? aCallers”, @call_stack_binding ) ==
‘local-variable’ ) # Ensure we have the proper name for locvar in
call_stack().
aCallers = eval( “aCallers”, @call_stack_binding ) # temp to
avoid repeat eval() calls

  push StackFrame::new()
  # The nature of set_trace_func() is to return the class and

method name for the previous stack frame.
at(-2).object = trace_object
at(-2).method = trace_method

  last.binding = trace_bind
  aCallers.first =~ /^ ( [\w\.]+ ) : ( \d+ ) \D*/x
  assert( $1 && $2 )
  last.file = $1
  last.line = $2

  return self
end# call()

# ret(): Called after all iterations are finished, just before the

CallStack is passed back to the client.
def ret()
shift() # Remove placeholder first element.
assert( length > 0 )

  assert( eval( "defined? nCount", @call_stack_binding ) ==

‘local-variable’ ) # Ensure we have the proper name for locvar in
call_stack().

  # If the final frame isn't at main scope, depending on the

arguments to call_stack(), it’s also a placeholder and must be removed:
if( eval( “nCount”, @call_stack_binding ) )
pop()
else
# Flesh out final element:
last.object = ‘main’
last.method = nil
end# if

  return self
end# ret()

end# CallStack

*************** CALL_STACK ***************

call_stack(): Retrieve CallStack object (array of StackFrame

objects).
def Binding::call_stack( nSkipFrames = 0, nCount = nil )
# Validate vars:
[nSkipFrames, nCount].each { |var|
if( var && (!var.is_a?(Integer) || (var < 0 )) ) then raise(
ArgumentError, “Optional argument to call_stack() should be a
nonnegative Integer.” ); end
}# each

# Store temporarily to obviate reinvocation.  Needed below, and

also by the current implementation of class CallStack, so I opted to
store it here and pass the CallStack ctor this, call_stack()'s,
binding, to allow for greater flexibility if CallStack is extended.
aCallers = caller()

# If at toplevel or nSkipFrames is greater than the number of

frames available, /or/ the client explicitly requests zero frames, exit
early:
if( (aCallers.length-nSkipFrames <= 0) || (nCount == 0) ) then
return Binding::stack_class::new(); end

# Trim the first entry, the client function, which we don't want to

include; then omit the next nSkipFrames entries.
#aCallers.slice!(0, nSkipFrames) FIXME: Don’t let this creep back
in!

# If nCount sufficiently large, no reason to use it at all:
if( nCount && ( nSkipFrames+nCount >= aCallers.length) ) then

nCount = nil; end

# Trim end if nCount provided:
if( nCount )
  assert( nCount < aCallers.length )
  nCount += nSkipFrames+1 # If nCount must be used in its "proper"

function, we have to pad out the number of trace calls to one beyond
the specified, in order to receive object and method info (see
CallStack::call(), above).
aCallers.slice!(nCount … -1)
end# end

aCallStack = Binding::stack_class::new( binding() ) # Pass current

Binding, giving CallStack access to locals.
callcc { |cc|
# Hope springs eternal(!):
#old_tracefunc = get_trace_func()
set_trace_func(
proc {
# event, file, line, id, bind, classname
|event, *remaining_args|

      # Capturing frames:
      if( event == 'return' )

        # If frame capture yet enabled:
        if( nSkipFrames == 0 )
          # capture current:
          aCallStack.call( event, *remaining_args )
        else # skip to next
          nSkipFrames -= 1
        end# if

        aCallers.shift()

        # If at toplevel, end ride:
        if( aCallers.length == 0 )
          #set_trace_func( get_trace_func() )
          set_trace_func( nil )

          # -----> Stack unrolling finishes here:
          cc.call()
        end# if
      elsif( event == 'line' )
        # *** This breaks free of inadvertent ensure() or rescue()

blocks:
throw( Binding::CallStack::tracing_exception ) #

---

      end# if
    }# proc
  )# set_trace_func

  # Stack unrolling begins here: ----->
  throw( Binding::CallStack::tracing_exception )
}# callcc

assert(nSkipFrames)

# Resume execution in client func:
aCallStack.ret()
return( aCallStack )

end# call_stack()

Class instance var:

@stack_class = Binding::CallStack
class << self
attr_accessor :stack_class
end
end# Binding

</call_stack.rb>

If the unrolling goes uninterrupted, set_trace_func() gets only
‘return’ events. If it reports a ‘line’ event, we bully our way back
out with another exception. This also takes care of the related
potential injudicious use of rescue(). It does not, however, address
the fact that use of call_stack() will cause else() blocks to never
be executed, and this one may really have me stumped. It was my
understanding, my impression, at least, that recovering from an
exception is a traditional and accepted application for a continuation,
and the use of an else() block presupposes that every exception, or
whatever you want to call it, is an error–which is inconsistent with
the language’s inclusion of throw() and catch(). In all, this looks
like a good reason for avoiding else() blocks–and I’m not just being
sore (I hope).

-J