Binding.of_caller examples don't work


#1

I’m trying to use the Binding class in the extensions library
http://extensions.rubyforge.org/rdoc/index.html

In particular, I want the method of_caller, but none of the examples
work for me
get ‘undefined method or variable’ for all that should be in the
caller’s scope.

local_variable returns an empty array.

Is this broken?
…using Ruby 1.8.6

thanks,
-doug.


#2

On Jan 10, 3:49 pm, Doug removed_email_address@domain.invalid wrote:

…using Ruby 1.8.6
Binding.of_caller is no longer possible.

T.


#3

Trans wrote:

Binding.of_caller is no longer possible.

I just thought of a use for it!

Is a workaround available?


#4

2009/1/11 Phlip removed_email_address@domain.invalid:

Trans wrote:

Binding.of_caller is no longer possible.

Is a workaround available?

It seems that the original idea of Florian Groß is still valid. Here’s
a version working for my Ruby 1.8.6 on Windows:

Re-implementation of Binding.of_caller for Ruby 1.8.6

© 2009 Pit C.

Original idea and implementation by Florian G.

def Binding.of_caller

old_critical = Thread.critical
Thread.critical = true

restart_cc = result = error = nil

expected_events = %w"line c-call c-return return return"
tracer = lambda do |event, file, line, id, binding, classname|
  if expected_events.empty?
    set_trace_func(nil)
    result = binding
    restart_cc.call
  elsif event != expected_events.shift
    set_trace_func(nil)
    error =
      "Binding.of_caller used in non-method context" +
      " or trailing statements of method using it aren't in the 

block."
restart_cc.call
end
end

callcc { |cc| restart_cc = cc }

if result
  yield(result)
elsif error
  raise(ArgumentError, error)
else
  set_trace_func(tracer)
end

ensure
Thread.critical = old_critical
end

Regards,
Pit


#5

On Jan 11, 1:11 pm, “Pit C.” removed_email_address@domain.invalid wrote:

      " or trailing statements of method using it aren't in the block."
else
  set_trace_func(tracer)
end

ensure
Thread.critical = old_critical
end

Cool. But word or warning to Doug, I would not recommend using
#set_trace_func in a production application. It can slow things down
considerably.

T.


#6

On Sun, 11 Jan 2009 14:13:29 -0500, Trans wrote:

Cool. But word or warning to Doug, I would not recommend using
#set_trace_func in a production application. It can slow things down
considerably.

T.

First, the version in the extensions gem uses Florian Groß’s idea, so
it’s the same as what Philip posted, modulo the organization of
initialization code and the like.

Second, the trace function is only being used for a couple of calls
before the continuation is called, and execution jumps back into
Binding.of_caller at which point the trace function is removed. It
wouldn’t appear to me like it would hurt things too much. (Maybe it
requires turning on an otherwise disabled feature in JRuby?)

Third: it looks like this should be portable to Ruby 1.9. Ruby 1.9 no
longer has Thread.critical, replacing it with Thread.exclusive (which
takes a block). Although Thread.exclusive is not suitable for this
function because the block exits before the function exits, it should be
unnecessary to lock out other threads anymore, since you should be able
to do a Thread.current.set_trace_func instead of Kernel#set_trace_func
and have it affect only the current thread.

–Ken


#7

From: “Charles Oliver N.” removed_email_address@domain.invalid

authorize(decrypted_password)
vars.each do |name|
if name =~ /password/
use_for_evil(eval name, binding)
end
end
end

If we’re calling untrusted code and giving it full execution privileges,
then
I think Binding.of_caller is the least of our worries. The untrusted
code
could as easily rm -rf ~ or innumerable other evil things.

Untrusted code needs to be sandboxed.

Seems we could either make Binding.of_caller not work at all if $SAFE
is >= 3 … or preferably, if it’s possible for the interpreter to
determine
whether the caller’s binding would have a lower $SAFE level than the
current binding, then only raise a SecurityException in that case.

Would this address your security concerns?

Regards,

Bill


#8

On Sun, 11 Jan 2009 22:27:29 -0500, Charles Oliver N. wrote:

do_secure_task
use_for_evil(eval name, binding)

  • Charlie

It turns out that Binding.of_caller isn’t a hack I need. It’s a hack I
though I needed when I thought I’d have to use Ripper to fix
s-expression
generation in the SqlStatement for Ruby 1.9. Then I discovered that all
the operators I needed were now overloadable, so I went with that
instead.

Which reminds me of another question I had. You were discussing
somewhere
else how Thread.critical works in JRuby. Does Thread.critical /
Thread.exclusive in JRuby protect other threads against changes to the
core class methods? Can I run Thread.critical=true , dramatically
redefine operators on the core classes, redefine the operators back
their original implementations, run Thread.critical=false, and expect
that no other JRuby thread can get in and be broken becuase I’ve
dramatically gutted the core classes?


#9

Bill K. wrote:

If we’re calling untrusted code and giving it full execution privileges,
then
I think Binding.of_caller is the least of our worries. The untrusted code
could as easily rm -rf ~ or innumerable other evil things.

Untrusted code needs to be sandboxed.

Sandboxing is made nearly impossible if calls in your own chain can
escalate privileges by executing code from your binding. This is the
primary reason why prototype continuation support for the JVM probably
will never be shipped…the ability to implicitly access things
elsewhere in the call stack has all sorts of really nasty security
issues, even in a pretty secure environment like the JVM.

Seems we could either make Binding.of_caller not work at all if $SAFE
is >= 3 … or preferably, if it’s possible for the interpreter to
determine
whether the caller’s binding would have a lower $SAFE level than the
current binding, then only raise a SecurityException in that case.

Would this address your security concerns?

No, because JRuby doesn’t support $SAFE. It’s a blunt instrument that I
don’t think anyone has ever audited for actual safety. So as a result,
anyone believing $SAFE is actually safe is probably introducing even
more risk into their apps.

Also, $SAFE is a global thing, and you can’t localize it to a given
binding, so that wouldn’t work anyway.

Honestly, if you want a binding, pass one in.

  • Charlie

#10

Ken B. wrote:

Second, the trace function is only being used for a couple of calls
before the continuation is called, and execution jumps back into
Binding.of_caller at which point the trace function is removed. It
wouldn’t appear to me like it would hurt things too much. (Maybe it
requires turning on an otherwise disabled feature in JRuby?)

Tracing works, but JRuby and IronRuby do not support continuations, and
I think they work differently in 1.9. It’s a hack that’s unlikely to
work reliably, even across minor versions of a given implementation.

In general I think Binding.of_caller is a really, really bad idea
because it exposes internal details every method call in your code to
the whims of someone else’s code. Basically, it breaks the most sacred
encapsulation possible, a method call’s local variables.

Imagine a case like this:

def secure_action(encrypted_password)
decrypted_password = decrypt(encrypted_password)
authorize(decrypted_password)
do_secure_task
end

If it were possible to modify do_secure task, or if you were simply
calling a third-party library, it could access the runtime variables of
the caller:

def do_secure_task
binding = Binding.of_caller
vars = eval ‘local_variables’, binding
vars.each do |name|
if name =~ /password/
use_for_evil(eval name, binding)
end
end
end

I even implemented Binding.of_caller for fun once in JRuby, and removed
it when I realized how invasive and dangerous it was.

Just like retry was limited in 1.9 to restarting a rescued block of code
(rather than being able to reevaluate the original arguments and
receiver), Binding.of_caller should remain unsupported.

  • Charlie

#11

Ken B. wrote:

Which reminds me of another question I had. You were discussing somewhere
else how Thread.critical works in JRuby. Does Thread.critical /
Thread.exclusive in JRuby protect other threads against changes to the
core class methods? Can I run Thread.critical=true , dramatically
redefine operators on the core classes, redefine the operators back
their original implementations, run Thread.critical=false, and expect
that no other JRuby thread can get in and be broken becuase I’ve
dramatically gutted the core classes?

You could up until about a week ago, when I proactively modified
critical= to just be a reentrant global lock. So basically…

Before:

  • critical= causes the current thread to be the only thread running
  • …except on JRuby where threads run in parallel and have to reach a
    checkpoint before they stop
  • …or if code you call from within a critical section itself spins up
    threads

After:

  • critical= acquires or releases a lock on a global mutex
  • threads that don’t have critical sections continue running
  • threads that try to acquire the lock while another thread has it block

This was actually proposed by Shri B. of IronRuby and blogged here:

http://blogs.msdn.com/shrib/archive/2009/01/07/proposed-spec-for-ruby-s-thread-critical.aspx

She (he?) proposes that almost all current uses of critical= intend to
simply delimit a critical section of code, and so a single global mutex
is sufficient for those cases. And in light of the fact that native
calls and parallel-threaded impls can’t be guaranteed to deschedule
other threads, I think this is a much more concrete and reasonable
definition. So I agreed, said so on a critical-related ruby-core thread,
and went ahead with the change. Nothing has broken so far :slight_smile:

For your case, I think your best bet is to either make those changes
before other threads start running or just accept that some of them
won’t see your changes all at once. I believe what you want to do may
work on MRI, since it doesn’t actually run threads in parallel and
critical= stops it from scheduling new ones on its own.

But also note that critical= is gone in 1.9, so you should not use it
anyway. Use a Mutex whenever possible…it’s quick, easy, and safe.

  • Charlie

#12

On Mon, 12 Jan 2009 10:59:52 -0600, Ken B. wrote:

get in and be broken becuase I’ve dramatically gutted the core
threads

After:

  • critical= acquires or releases a lock on a global mutex * threads
    that don’t have critical sections continue running * threads that try
    to acquire the lock while another thread has it block

This was actually proposed by Shri B. of IronRuby and blogged here:

http://blogs.msdn.com/shrib/archive/2009/01/07/proposed-spec-for-ruby-
s-

For your case, I think your best bet is to either make those changes
old_critical=Thread.critical
Can I guarantee that other threads won’t see my badly mangled Fixnum and

(The methods being overridden in this case are pretty much any useful
binary operator on Fixnum and String, so this is pretty drastic, not to
mention that one of those operators is != which people have been
gripeing about a lot on ruby-core.)

–Ken

Oh, shoot. I just realized I’d have the same problem if code passed to
the s-expression builder called user-defined custom methods (which is
actually a desired behavior) to use their results. Those methods would
be
broken inside by the wretching monkey patching going on. I guess it’s
probably best to abandon the idea of the s-expression builder
completely,
or do it with a string, an explicit binding, and ripper.

–Ken


#13

On Mon, 12 Jan 2009 10:34:26 -0500, Charles Oliver N. wrote:

You could up until about a week ago, when I proactively modified

  • critical= acquires or releases a lock on a global mutex * threads that
    don’t have critical sections continue running * threads that try to
    acquire the lock while another thread has it block

This was actually proposed by Shri B. of IronRuby and blogged here:

http://blogs.msdn.com/shrib/archive/2009/01/07/proposed-spec-for-ruby-s-
thread-critical.aspx
before other threads start running or just accept that some of them
won’t see your changes all at once. I believe what you want to do may
work on MRI, since it doesn’t actually run threads in parallel and
critical= stops it from scheduling new ones on its own.

My goal is that other threads don’t see the changes, by operating as
follows:

In Ruby 1.8 pseudocode:

old_critical=Thread.critical
begin
Thread.critical=true

[Fixnum,String].each{|klass| override_methods_on klass}
result=builderblock.call
[Fixnum,String].each{|klass| restore_methods_on klass}
ensure
Thread.critical=old_critical
end

Can I guarantee that other threads won’t see my badly mangled Fixnum and
String? Or is there some possiblity that another thread would continue
executing and call (now badly broken) methods on Fixnum and String for a
while before it hit a checkpoint that made it hit the lock? From your
description it sounds like the latter is the case, in which case I need
to be looking at sandboxing solutions or warn people very loudly that
this part of the library is not thread safe and cannot be made thread
safe. (AIUI, there’s also no standard way of sandboxing right now.)

But also note that critical= is gone in 1.9, so you should not use it
anyway. Use a Mutex whenever possible…it’s quick, easy, and safe.

  • Charlie

In Ruby 1.9 pesudocode this is only slightly different:

Thread.exclusive do
[Fixnum,String].each{|klass| override_methods_on klass}
result=builderblock.call
[Fixnum,String].each{|klass| restore_methods_on klass}
end

(The methods being overridden in this case are pretty much any useful
binary operator on Fixnum and String, so this is pretty drastic, not to
mention that one of those operators is != which people have been
gripeing
about a lot on ruby-core.)

–Ken


#14

Ken B. wrote:

result=builderblock.call
to be looking at sandboxing solutions or warn people very loudly that
this part of the library is not thread safe and cannot be made thread
safe. (AIUI, there’s also no standard way of sandboxing right now.)

The latter…you can’t guarantee the other threads will necessarily stop
before they see your breaking changes. You need to look at sandboxing,
or help Matz implement selector namespaces (which I’d love to help with
too).

In Ruby 1.9 pesudocode this is only slightly different:

Thread.exclusive do
[Fixnum,String].each{|klass| override_methods_on klass}
result=builderblock.call
[Fixnum,String].each{|klass| restore_methods_on klass}
end

This only guarantees only one thread can execute that block of code; it
doesn’t stop other threads from running:

â—† ruby1.9 -e “require ‘thread’; Thread.new { sleep 1; puts ‘hello’; redo
}; Thread.exclusive do; sleep; end”
hello
hello
hello
hello
hello
^C

  • Charlie

#15

On Mon, 12 Jan 2009 16:31:15 -0500, Charles Oliver N. wrote:

â—† ruby1.9 -e “require ‘thread’; Thread.new { sleep 1; puts ‘hello’; redo
}; Thread.exclusive do; sleep; end”
hello
hello
hello
hello
hello
^C

  • Charlie

Okay. In that case, I’d better read Pickaxe for 1.9 from cover to cover
before I try to use 1.9.

(Ducks and runs)
–Ken


#16

From: “Charles Oliver N.” removed_email_address@domain.invalid

Bill K. wrote:

If we’re calling untrusted code and giving it full execution privileges, then
I think Binding.of_caller is the least of our worries. The untrusted code
could as easily rm -rf ~ or innumerable other evil things.

Untrusted code needs to be sandboxed.

Sandboxing is made nearly impossible if calls in your own chain can escalate privileges by executing code from your binding.

Since you mention below JRuby doesn’t support $SAFE, then I’m at a
loss as to which sandboxing mechanism you refer. (Does JRuby offer an
alternative sandboxing approach?)

It seems to me there’s trusted code, and untrusted code. We allow
trusted
code to do all sorts of potentially dangerous things, including
reopening and
redefining any method in any class, etc. etc. So again, if we’re
talking about
trusted code, I’m hard pressed to see the validity of holding
Binding.of_caller
up as an exemplar of something somehow more dangerous than the myriad
privileges already granted to trusted code in ruby.

Conversely, if we’re talking about untrusted code, Binding.of_caller
seems
a relatively unremarkable addition to an already existing great number
of
avenues by which untrusted code can affect the system unless (somehow)
sandboxed.

If no such sandboxing mechanism exists, then why pick on
Binding.of_caller
in particular?

No, because JRuby doesn’t support $SAFE. It’s a blunt instrument that I don’t think anyone has ever audited for actual safety. So
as a result, anyone believing $SAFE is actually safe is probably introducing even more risk into their apps.

Also, $SAFE is a global thing, and you can’t localize it to a given binding, so that wouldn’t work anyway.

It’s thread local. And it apparently somehow does get localized to a
given binding (?) because ruby remembers the $SAFE level at which a
lexical closure was compiled (or at least, a proc.)

For example, if a thread is at $SAFE=4, and calls a proc which was
compiled at $SAFE=0, then the proc executes at $SAFE=0.

I do share your concerns about $SAFE never having been formally
audited for security, and I’d agree it would be foolish to assume there
don’t exist some undiscovered holes in the system.

(On the other hand, in practical terms, we live with as yet undiscovered
security holes in windows, linux, IIS, apache, etc. every day too.)

That said, it appears $SAFE=4 was designed with the intent to
provide a secure sandbox. And it seems to me the $SAFE mechanism
could as easily disallow inappropriate uses of Binding.of_caller, just
as it currently disallows the reopening of classes and modules, etc.

Honestly, if you want a binding, pass one in.

Been there, done that.

As Matz pointed out with respect to other dangerous features:

"open class" is so strong (often too strong), we can break things 

easily.
In other word, Ruby trust you to give you sharp knives, where Python
don’t.
From the Python point of view, it’s wrong, I guess.

Again, I don’t see why Binding.of_caller is particularly sharper than
many
other knives in ruby. (If I try to imagine writing malicious code I
know will
be executed with full privileges, Binding.of_caller isn’t the first tool
that
comes to mind when I contemplate all the nefarious possibilities.)

Regards,

Bill


#17

Bill K. wrote:

Sandboxing is made nearly impossible if calls in your own chain can
escalate privileges by executing code from your binding.

Since you mention below JRuby doesn’t support $SAFE, then I’m at a
loss as to which sandboxing mechanism you refer. (Does JRuby offer an
alternative sandboxing approach?)

The JVM has its own built-in security mechanisms, and we do nothing to
override or subvert them. We’ve considered adding portions of $SAFE like
limiting eval and loading of code from the filesystem, but largely $SAFE
is so amorphous and untested we’ve opted to leave it basically
unimplemented. Also, a substantial portion of $SAFE’s guarantees come
from tainting, which requires checking taint on objects over and over
and over again. I don’t think it’s a realistic or reliable security
mechanism as it exists today.

It seems to me there’s trusted code, and untrusted code. We allow trusted
code to do all sorts of potentially dangerous things, including
reopening and
redefining any method in any class, etc. etc. So again, if we’re
talking about
trusted code, I’m hard pressed to see the validity of holding
Binding.of_caller
up as an exemplar of something somehow more dangerous than the myriad
privileges already granted to trusted code in ruby.

Without Binding.of_caller, no trusted or untrusted Ruby code can access
a method’s local variables unless that method passes a block or binding
explicitly. of_caller opens up every method to that possibility. You
don’t think that exposure is problematic?

Conversely, if we’re talking about untrusted code, Binding.of_caller seems
a relatively unremarkable addition to an already existing great number of
avenues by which untrusted code can affect the system unless (somehow)
sandboxed.

It goes well above and beyond existing mechanisms since it exposes
details of a caller’s bindings that would never be accessible through
any other means.

Similar logic kicked retry out of 1.9, since it caused the original
receiver and arguments to be reevaluated, whether the caller that
behavior or not.

In general we should avoid adding the potential for locally-scoped side
effects that no amount of inspection can detect. It opens up a big can
of worms if you can’t look at a method and know each step along the way
that the variables are going to be what you set them do, and that no
call you make will invalidate that expectation.

There’s also logistical concerns. If you had Binding.of_caller
available, how do you ensure you’re not irreparably damaging the
caller’s context?

def do_call
binding = Binding.of_caller
eval “some_local_var = nil”, binding
end

def innocent
some_local_var = important_value
do_call

now, without me assigning it, some_local_var has been changed

end

This is a horrible breach of trust that should never, ever be made
possible in Ruby.

(On the other hand, in practical terms, we live with as yet undiscovered
security holes in windows, linux, IIS, apache, etc. every day too.)

$SAFE is one place where we simply can’t get by with the loose
specifications in place for the rest of Ruby. I’d welcome discussions on
how to make $SAFE or something similar a formally-specified security
mechanism for Ruby. But as it stands now, I think it’s too vague and
requires too many little bits of code sprinkled all over the codebase to
function correctly. I’m no security expert, but relying on a million
tiny little taint checks to ensure the sandbox remains secure seems
totally infeasible. If you have just one core method that doesn’t
propagate tainting correctly, you’re screwed.

That said, it appears $SAFE=4 was designed with the intent to
provide a secure sandbox. And it seems to me the $SAFE mechanism
could as easily disallow inappropriate uses of Binding.of_caller, just
as it currently disallows the reopening of classes and modules, etc.

You’re right about $SAFE locality…a binding does appear to keep a
reference to its original thread’s state, so it sees that thread’s
$SAFE. But of course that’s not really applicable here, since $SAFE
propagates down normal call stacks, and Binding.of_caller is intended
for normal call stacks. So caller and callee would always have the same
$SAFE level either way.

If you’re talking about doing Binding.of_caller from within a proc body
that came from another thread that has a different $SAFE level, with the
caller calling Proc#call and the callee using that call to get the
caller’s binding with Binding.of_caller…I think we’ve probably got
other complexity and threading issues to consider before we try to
secure Binding.of_caller.

Here’s an example of what you’d have to do to have a different caller
$SAFE than callee:

puts “safe outside: #{$SAFE}”
x = nil
Thread.new { x = proc { puts “safe in proc: #{$SAFE}” } }.join
$SAFE = 1
puts “safe escalated: #{$SAFE}”
x.call
new_safe = eval “$SAFE”, x
puts “safe from binding #{new_safe}”

This outputs 0, 1, 0, 1 as you’d expect. So the only way you’d
potentially have a different safe level for a caller than for a callee
is if you were evaluating code against a binding with a different save
level (at which point we’re explicitly using a binding anyway) or
calling a proc (where it would be pretty rare to see Binding.of_caller).
So I think $SAFE is not a particularly useful way to secure individual
frames in the call stack.

Again, I don’t see why Binding.of_caller is particularly sharper than many
other knives in ruby. (If I try to imagine writing malicious code I
know will
be executed with full privileges, Binding.of_caller isn’t the first tool
that
comes to mind when I contemplate all the nefarious possibilities.)

It’s WAY sharper because it invalidates infallible truths about
execution flow in a method body. Today, you can be guaranteed that
unless you explicitly pass a binding to another call (either through
eval/binding or by passing a block) none of your local variables will be
mutated unless you mutate them yourself. Binding.of_caller breaks that
assumption for all code everywhere. I don’t want to live in that world.

  • Charlie