Private visibility should be removed from Ruby 2

On 10/1/06, David V. [email protected] wrote:

(generating public wrappers for private methods of tested classes) would
they are necessary seems just overkill.
It almost sounds that you’re using Ruby as only a “better Java” :slight_smile:
Many people consider Smalltalk-style OO and metaprogramming
in Ruby extremely important.

Back to the point. It is very unusual for OO system to have
public/private distinction.
(http://www.approximity.com/ruby/Comparison_rb_st_m_java.html)
The OO language Smalltalk did not have it, and other OO systems
like Objective C, CLOS, Perl 5 and Python do not have it either.

The only listed languages that do are C++, Java and Ruby.
Conceptual model behind C++ is very far from Smalltalk-style OO.
In fact it has very little to do with OO, C++ objects are little
more than abstract data types. There is nothing remotely similar
to OO “message passing”, the “methods” are merely nicely
namespaced functions with some hacks for “inheritance” (and you
need to turn inheritance support explicitly using virtual keyword,
by default you get nothing more than nicely namespaced functions).
There is no such thing as interface of an objects. The same object
is going to behave in completely different way depending on
context - a single method is going to do something completely different
depending on whether you call it from object’s class, subclass of it
(and there is public and private inheritance, that changes object’s
interface), or a different place.
The same method can behave differently depending on type of variable
to which object is assigned - if Foo is a subclass of Bar, then the
following code:
Foo *f = new Foo();
Bar *b = f;
b->bar();
f->bar(); // bar redefined in Foo, bar not virtual
can do two completely different things.
The model is already so complicated, that adding private/public
distinction to everything doesn’t affect it much.

Java object model is halfway between C++ and Smalltalk.
Objects still do not have single clear interfaces. If Foo is
a subclass of Bar, and both have a private method bar(),
then calls to some_foo_obj->bar() will behave differently
depending on whether the call was done from methods declared in
Foo class or from methods declared in Bar class.
Private methods are also automatically final, and therefore
they cannot be overriden. So private methods in Java still
have more to do with global functions than with regular methods.
Fortunately they at least got rid of private inheritance.

I’m not saying “private methods” are useless in C++/Java.
They are simply not methods at all. They are nicely encapsulated
global helper functions for implementation of a class. They have
class-specific namespace and cannot be overriden, so a class
can define whatever private functions it wants to, and they do not
affect anything else.

The only language left that has Smalltalk-style OO and this kind of
access controls in Ruby. Instead of objects having single interface
like Smalltalk or multiple interfaces like C++/Java, Ruby objects has
exactly two - one for normal method calls, and a second for “implicit
self”
method calls. And we don’t get much - defining method “private”
doesn’t provide much “protection” as there is a single namespace
for all methods, so it can be accidentally redefined in a subclass.
On the other hand methods cannot call private methods of different
object of the same class (or its subclasses), so objects with
less-defined
interface and hidden implementation need to either hack around
access control inside own methods (ugly), or to have their internals
public.
A very simple and very common example would be == method
that checks if both objects have the same class, and if so,
whether their fields are identical.

While writing this reply I found something that really surprised me.
It is possible to get real private variables in Ruby, with private
per-class namespaces, not overridable, and working with other objects
of the same class, just like in C++/Java, by a few lines of
metaprogramming.
They are much better at encapsulation control than the official classes.

Here’s a snippet:

class Class
def define_private_method(name, &blk)
define_method(:xyzzy, &blk)
class_variable_set(“@@#{name}”.to_sym, instance_method(:xyzzy))
remove_method :xyzzy
end
end
class Foo
define_private_method(:bar) { puts “Hello” }
# Not the prettiest syntax possible.
# A slightly different idea:
# @@bar.call(self, *arguments)
def foo; @@bar.bind(self).call() end
end
Foo.new.foo

By movinc @@bar around or keeping it in some other variable (like
file-local, or package-local) we can easily implement advanced
visibility
control.

So I think it would be good idea to have method visibility controls
removed because:

  • They do not fit Smalltalk-style OO, so removing them would make
    everything simpler
  • We get very limited benefit from them due to lack of C++/Java-style
    features like per-class namespaces for private methods (or even
    instance variables), and ability to call private methods of other
    objects of the same type
  • They make metaprogramming, unit testing etc. more difficult
  • They provide very limited control over visibility
  • Very simple Ruby metaprogramming gives us much more powerful private
    variables anyway.
  • I think it’s unlikely for current visibility control system to lead
    to any cool things. As far as I can tell it was never used for
    implementing any magic. There is no obvious way to add any features
    (even to get them to the level of that metaprogramming snippet)
    without greatly complicating the language.
  • Ruby 2 is exactly the right time for doing such changes

Tomasz W. wrote:

[snip]

  • Very simple Ruby metaprogramming gives us much more powerful private
    variables anyway.
  • I think it’s unlikely for current visibility control system to lead
    to any cool things. As far as I can tell it was never used for
    implementing any magic. There is no obvious way to add any features
    (even to get them to the level of that metaprogramming snippet)
    without greatly complicating the language.
  • Ruby 2 is exactly the right time for doing such changes

At times I have had such thoughts too. After all Ruby is largely a
Document and Test oritented language b/c of it’s high degree of
dynamicisim. There is at least one feature of the private/protected vs.
public system that is valuable though: you can catch public calls via
method_missing even when there are private/protected methods defined.
Eg.

class X
def method_missing( name ); name; end
private
def bird; “hoot!”; end
end

x = X.new
x.bird #=> “bird”
x.send(:bird) #=> “hoot!”

I only recently came to understand this myself and it made a big
difference in how I handled certain usecases of method_missing. Unless
you see a better way to hadle this, then I think the private/protected
vs. public distiguish it useful.

As a side note, I sure would like to see class/module local methods
(not part of inheritance chain) if possible.

T.

Tomasz W. wrote:

It almost sounds that you’re using Ruby as only a “better Java” :slight_smile:
Many people consider Smalltalk-style OO and metaprogramming
in Ruby extremely important.

Maybe.

Back to the point. It is very unusual for OO system to have
public/private distinction.
(http://www.approximity.com/ruby/Comparison_rb_st_m_java.html)
The OO language Smalltalk did not have it, and other OO systems
like Objective C, CLOS, Perl 5 and Python do not have it either.

The only listed languages that do are C++, Java and Ruby.

They forget Ada, Simula 87, Eiffel…

Conceptual model behind C++ is very far from Smalltalk-style OO.

Smalltalk-style OO isn’t the only true one. In fact, object-oriented
programming is known to have no actual definition.

There is nothing remotely similar
to OO “message passing”,

See above, OO != message passing. And even in Smalltalk, the message
pass will trigger a method invocation most of the time.

the “methods” are merely nicely
namespaced functions with some hacks for “inheritance” (and you
need to turn inheritance support explicitly using virtual keyword,
by default you get nothing more than nicely namespaced functions).

That’s a specific C++ quirk that comes is part historical, part
intentional. A design policy of C++ is that it should introduce no
overhead over C unless explicitly stated by the programmer. Vtable
method dispatch is overhead, that’s why it has to be explicitly enabled.

There is no such thing as interface of an objects.

Pure virtual classes.

Bar *b = f;
b->bar();
f->bar(); // bar redefined in Foo, bar not virtual
can do two completely different things.
The model is already so complicated, that adding private/public
distinction to everything doesn’t affect it much.

Once again, C++ quirks which do have a reason, even if you might find
them unjustified. Also are completely unrelated to the issue of access
level restriction, except as an “Oh, look, this thing about C++ sucks,
that’s why another thing that’s in C++ too sucks.”

Java object model is halfway between C++ and Smalltalk.
Objects still do not have single clear interfaces.

Can you elaborate on what you expect as a clear interface?

If Foo is
a subclass of Bar, and both have a private method bar(),
then calls to some_foo_obj->bar() will behave differently
depending on whether the call was done from methods declared in
Foo class or from methods declared in Bar class.

That’s why you use protected on methods you intend to be called in
subclasses.

Private methods are also automatically final, and therefore
they cannot be overriden. So private methods in Java still
have more to do with global functions than with regular methods.

That’s why you use protected on methods you intend to be overridden in
subclasses.

Fortunately they at least got rid of private inheritance.

Eiffelists might disagree. Multiple inheritance, private inheritance,
and feature renaming are core features of Eiffel’s object model as far
as I know.

I’m not saying “private methods” are useless in C++/Java.
They are simply not methods at all. They are nicely encapsulated
global helper functions for implementation of a class.

The question is what distinguishes a method from a function. Is it the
polymorphic dispatch, or being bound to a specific object? In CLOS, it’s
only the former because of multimethods, in other languages polymorphic
dispatch seems to happen as a consequence of the latter. Once again,
lacking an actual definition of object-orientation, both approaches can
be considered equally valid.

They have
class-specific namespace and cannot be overriden, so a class
can define whatever private functions it wants to, and they do not
affect anything else.

I consider uncontrolled namespace clobbering such as this a flaw of
ruby. “Module Foo does weird stuff to module Bar because the author of
Foo thought they’re convenient / more Right ™ and now my code broke
when I wanted to add support for Foo.” Bugs like that are annoying in
the least.

The only language left that has Smalltalk-style OO and this kind of
access controls in Ruby. Instead of objects having single interface
like Smalltalk or multiple interfaces like C++/Java, Ruby objects has
exactly two - one for normal method calls, and a second for “implicit self”
method calls.

I personally think this is a design burp.

And we don’t get much - defining method “private”
doesn’t provide much “protection” as there is a single namespace
for all methods, so it can be accidentally redefined in a subclass.

Separating private methods in a separate namespace has been thought for
Ruby 2.0. No idea what happened to the proposal.

On the other hand methods cannot call private methods of different
object of the same class (or its subclasses), so objects with less-defined
interface and hidden implementation need to either hack around
access control inside own methods (ugly), or to have their internals
public.

Protected. I don’t like calling private methods on other objects in Java
either and find it one of the C++ quirks that shouldn’t have made it
into that language.

A very simple and very common example would be == method
that checks if both objects have the same class, and if so,
whether their fields are identical.

If an operation can’t be implemented only communicating with another
object’s public interface, there’s something wrong with the design. You
may differ on this point, I’ll admit it’s one of taste more than
anything else.

[snip metaprogramming code]

So I think it would be good idea to have method visibility controls
removed because:

  • They do not fit Smalltalk-style OO, so removing them would make
    everything simpler

Ruby isn’t Smalltalk.

  • We get very limited benefit from them due to lack of C++/Java-style
    features like per-class namespaces for private methods (or even
    instance variables), and ability to call private methods of other
    objects of the same type

I argue that it would be better to just fix those design burps instead
of removing a language feature.

  • They make metaprogramming, unit testing etc. more difficult

More verbose. Not really difficult - at least for metaprogramming you
need a miner helmet for when you need to bang your head against a wall
anyway.

  • They provide very limited control over visibility

I can’t imagine it could be improved without making them not the
advisory “you shouldn’t do this” tags they are now.

  • Very simple Ruby metaprogramming gives us much more powerful private
    variables anyway.

Remove a core language feature only to get umpty modules reemulating it
in incompatible ways?

  • I think it’s unlikely for current visibility control system to lead
    to any cool things. As far as I can tell it was never used for
    implementing any magic. There is no obvious way to add any features
    (even to get them to the level of that metaprogramming snippet)
    without greatly complicating the language.

It’s used to tighten up code, make possible bugs due to metaprogramming
more visible, etc. Not necessarily cool or magical. Mind you, cool and
magical aren’t necessarily good qualities.

  • Ruby 2 is exactly the right time for doing such changes

The Ruby 1 → Ruby 2 transition isn’t supposed to be in the order of the
Perl 5 → Perl 6 one. As far as I know, the changes are mostly touchups
that fall out of scope for a minor version bump because they address
some of the core design burps. As far as I know, it’s supposed to be as
little codebreaking a change as necessary. Ruby 2 is not at all the time
for a keyword removal and object model revamp.

David V.

On 10/2/06, Tomasz W. [email protected] > I’m
not sure whether getting rid of all method visibility issues is

worth it. It seems to be a big improvement, but it also introduces
some compatibility problems (I dont).

Sorry, I must have mistyped alt-something and gmail sent my message
too early :slight_smile:

What I meant is that I don’t expect compatibility problems to be too
serious. The only code that would be broken is code that depended on
the same message behaving differently depending on context. This is
not common. And method visibility was going to be radically changed in
1.9 anyway, as send changed its behaviour (send was very often used as
“call while ignoring method visibility”). In fact, it wouldn’t
surprise me if getting rid of all visibility issues introduced fewer
problems than changing behaviour of send.

On 10/2/06, Trans [email protected] wrote:

class X
difference in how I handled certain usecases of method_missing. Unless
you see a better way to hadle this, then I think the private/protected
vs. public distiguish it useful.

You’ll have to unlearn what you’ve learned anyway,
as it gets changed in 1.9, where you get:
x.send(:bird) #=> :bird

That’s part of the complexity I was talking about.
There are two very similar operations - “generic message passing”,
and “message passing with implicit self”. In Ruby 1.8
Object#send did the former, in 1.9 it does the latter.
To do “message passing with implicit self” in 1.9 there is
a new method x.funcall(:bird).

But that’s not enough. Normal objects can be implemented
in terms of message_missing. If we let one method name to
do different things depending on context, we need to extend
it to message_missing somehow, using private_message_missing
or some sort of Kernel#implicit_self_context? method that would let
methods know in what context they have been called.
Reflection methods are already provided in 3 versions
(public, protected, private). Somehow Object#singleton_methods
only lists public singleton methods. I have no idea what
private/protected
singleton methods would be good for, but lack of symetry is somewhat
disturbing. ( x.singleton_methods lists only redefined methods,
it’s different from (class << x; self; end).methods, so I don’t see any
simple way of emulating private/protected_singleton_methods)

I have very little idea of what “protected” methods do, as Ruby
“private”
is most similar to Java “protected”. It would be highly disturbing me
if third version of send was needed. Another thing that disturbs me is
that there is not a single “protected” method in Ruby standard library,
in either 1.8 or 1.9. (Verified by ObjectSpace.each_object(Class) {|c|
c.protected_methods.each{|m| puts m}})

I’m not sure whether getting rid of all method visibility issues is
worth it. It seems to be a big improvement, but it also introduces
some compatibility problems (I dont).

I think that we can at least get rid of “protected” level,
which is used extremely rarely, and is very confusing.
Java did a similar thing when it removed “private protected”
visibility, which was also used extremely rarely and was very
confusing.

As a side note, I sure would like to see class/module local methods
(not part of inheritance chain) if possible.

[Long list of possible solutions follows]

It should be pretty straightforward to simply store a procedure in
instance/class variable or in local variable, just like we do with any
other value.

I didn’t look at the problem much, so maybe I missed something,
but I haven’t seen any elegant ways of creating methods
without names ?

Some intuitive solutions would be:
@@bar = UnboundMethod.new{ puts “bar #{@x}” }
@@bar.bind(self).call()
but UnboundMethod doesn’t have #new

Or:
@@bar = lambda{ puts “bar #{@x}” }
@@bar.bind(self).call()
but Proc doesn’t have #bind.

Two solutions that work are:
@@bar = lambda{ puts “bar #{@x}” }
self.instance_eval{ @@bar.call() }
Using different code for method objects created from
named methods and directly from lambdas is not very elegant,
as they are something pretty similar.

And:
class Class
def define_private_method(name, &blk)
define_method(:xyzzy, &blk)
class_variable_set(“@@#{name}”.to_sym, instance_method(:xyzzy))
remove_method :xyzzy
end
end

And then:
define_private_method(:bar) { puts “Hello” }
@@bar.bind(self).call()

I dislike the :xyzzy part a bit, as it can collide with some real
method,
but that’s the best I was able to find.

Syntax like @@bar.bind(self).call() looks a bit ugly.
We can probably somehow make it possibly to avoid
bind(self) part if we bind to self. The @@ .call part is
unfortunately going to stay.

On 10/2/06, David V. [email protected] wrote:

Tomasz W. wrote:

There is no such thing as interface of an objects.

Pure virtual classes.

Java object model is halfway between C++ and Smalltalk.
Objects still do not have single clear interfaces.

Can you elaborate on what you expect as a clear interface?

It seems that this is the main point of misunderstanding.
My very long comparison of C++ and Smalltalk models
was not meant to be about “C++ sucks and Smalltalk is the only true OO”.
I want to explain radically different models they use,
and that solutions (private etc.) that fit C++/Java-style models
do not really fit Ruby/Smalltalk-style models.

So let me try to do it again.

Imagine a Smalltalk object floating in the space. Nothing can ever learn
what is inside it. The only thing it does is receiving messages and
sending them. It does not have and does not want to have even a
faintest idea what was the context in which the message was sent. The
message has no idea what kind of object sent it and what kind of
object is the receiver.

Now imagine a C++ object. It is not a black box at all. It has
multiple interfaces, and everything that communicates with it must
know exactly what at least some of them are. Like with Smalltalk
object, it receives messages. But the sender addresses messages to
different interfaces, not to the object itself. So one message “bar”
can be sent to “Foo in public context”, while other message “bar” can
be sent to “Bar in private context”. It is pretty meaningless to talk
about “sending messages to objects” in C++, as the receiver is not an
object (but one of its many interfaces) and the message must know
exactly what interface it is targetting.

Java is following a cleaned-up version of C++ model, with some limited
reflection capabilities. Objects still have multiple interfaces, but
at least with public interfaces when a message comes, it’s going to
behave the same way no matter which interface was used. C++
non-virtual and Java final behave in significantly different ways. In
any case, Java is kinda halfway between C++ and Smalltalk, as from the
outside Java objects behaves a lot like Smalltalk objects (the same
message must do the same thing), but from the inside they are pretty
much like C++, and which method is going to be called depends on
context of the call.

Ruby object model is almost like Smalltalk’s. There are objects that
receive messages and everything is context-independent. The main
difference is that they have two interfaces, not one. They have an
interface for generic messages (like Smalltalk objects), and a special
interface for “implicit self” messages. (from the inside the
difference is much bigger, but that’s not important here)

Having a single well-defined interface was a big win for Smalltalk.
Ruby is almost there, but not quite. What is the reason for Ruby
to have “almost-Smalltalk” object model ?

Each object has only one “implicit self” interface,
not one “private” interface for each class, so it’s not going
to provide the kind of encapsulation that Java and C++'s
visibility controls do.

So what other benefits come from “implicit self” interface ?

And we don’t get much - defining method “private”
doesn’t provide much “protection” as there is a single namespace
for all methods, so it can be accidentally redefined in a subclass.

Separating private methods in a separate namespace has been thought for
Ruby 2.0. No idea what happened to the proposal.

Moving Ruby object model from “almost Smalltalk” to “pretty much Java”
would be much more radical than moving it to “completely Smalltalk”.

And I don’t think it’s a good idea. Mostly for metaprogramming reasons.

  • Ruby 2 is exactly the right time for doing such changes

The Ruby 1 → Ruby 2 transition isn’t supposed to be in the order of the
Perl 5 → Perl 6 one. As far as I know, the changes are mostly touchups
that fall out of scope for a minor version bump because they address
some of the core design burps. As far as I know, it’s supposed to be as
little codebreaking a change as necessary. Ruby 2 is not at all the time
for a keyword removal and object model revamp.

I’d argue that removal of private/protected is a very small change.
protected is hardly used at all (not a single use in standard library),
so it can be thrown away without much further discussion.

Removing private/protected distinction is more significant,
but it seems that everyone acknowledges than the way it works in 1.8
is not completely right. 1.9 already contains very big changes (and a
lot of
backward incompatibility by changing behaviour of Object#send) to
handling of private/protected, and many more changes are needed
before it’s useful (after two versions of send we need two versions of
method_missing etc.). And after these radical changes we don’t get
Java-like encapsulation, but something that is a lot less powerful
than a (admittedly ugly-looking) solution which uses a few lines of
metaprogramming. So it is going to change in Ruby 2, and the only
question is which way.

On 10/2/06, Csaba H. [email protected] wrote:

Using aux methods which are not intended to be part of the public
interface is unevitable programming practice. The need to take it out
from the public interface is also a very basic request.

I tried to get some empirical data on this, and the result
really surprised me. Of “private” methods in the whole Ruby
standard library, very few were used as “aux method”.

The code to get the results:
ObjectSpace.each_object(Module){|c| n = (c.private_instance_methods -
(c.ancestors - [c]).inject([]){|a,b|a+b.private_instance_methods}); p
[c.class, c, n.size, *n.sort] unless n.size == 0 }

Most of them were clearly part of the interface.
Like Module#attr_reader or FileUtils#rm_rf, they were supposed to be
used, but mostly in “implicit self context”. I do not see any reason
not to expand visibility of such methods.

Next category were some confusing entries like
Math.private_instance_methods containing a whole list of trigonometric
functions, Kernel containing Array, Float etc.

I wasn’t able to find any methods that were unambiguously meant
not to be called by a programmer. There might be some,
but they’re just a very small fraction of all “private” methods.

There wasn’t even a single “protected” method anywhere.

So whatever the purpose of Ruby visibility controls is,
it is difficult to claim it is hiding aux methods.

But assuming that we indeed want visibility control, what kind would
make sense in Ruby ? Of those OO systems that have such controls (most
don’t have any), every single language uses a completely different
design.

Is there even a rough agreement on what a good solution would look
like ? I guess we can at least agree that neither Ruby 1.8 nor Ruby
1.9 solutions are particularly good, as they contain confusing and
rarely used “protected” visibility, don’t work well with
method_missing and other reflection/metaprogramming methods, and use
highly confusing naming (Ruby “private” is kinda but not really Java
“protected”, and “private”
methods are usually public parts of the interface, not private parts of
the
implementation)

This should be solved in the core language (as others said before,
no dozen witty hacky metaprogramming snippets for this, please).
If not in the way it is done now, how would you do it?

I don’t really get the distinction between “core language” and “hacky
metaprogramming”. Half of Ruby core language is actually implemented
as metaprogramming. Ok, syntax in my example:
@@bar.bind(self).call(1, 2, 3)
was very ugly, but maybe someone can come up with nicer syntax,
or if it’s impossible and the problem is important, we can add a tiny
bit of syntactic sugar to Ruby. If we agree that some kind of visibility
control is the right way, we can simply add a few metaprogramming
methods to the standard library, and it becomes part of the “core
language”.

The idea of making visibility controls a metaprogramming hack is that
it makes it possible to test right now many different systems and see
how well they work.

Tomasz W. wrote:

Ruby object model is almost like Smalltalk’s. There are objects that
receive messages and everything is context-independent. The main
difference is that they have two interfaces, not one. They have an
interface for generic messages (like Smalltalk objects), and a special
interface for “implicit self” messages. (from the inside the
difference is much bigger, but that’s not important here)

I actually see the presence of these two interfaces beneficial.

Using aux methods which are not intended to be part of the public
interface is unevitable programming practice. The need to take it out
from the public interface is also a very basic request.

This should be solved in the core language (as others said before,
no dozen witty hacky metaprogramming snippets for this, please).
If not in the way it is done now, how would you do it?

(Well, at least it’s clear how not to do it. I think of Python’s horrid
name
manglng…)

Csaba

On 10/2/06, Trans [email protected] wrote:

class X
difference in how I handled certain usecases of method_missing. Unless
you see a better way to hadle this, then I think the private/protected
vs. public distiguish it useful.

As a side note, I sure would like to see class/module local methods
(not part of inheritance chain) if possible.

T.

Very nice behavior, I do not know what exactly you mean with a
class/local
method, maybe the following, which mimicks truely private methods.

class A
@@_I_am = self

def method_missing( name, *args, &blk )
    raise NoMethodError,"undefined method '#{name}' for #{self}" 

unless
instance_of? @@_I_am
send(name, *args, &blk)
end

private
def hello
    p 42
end

end

would be a base for what you want?

Cheers
Robert


Deux choses sont infinies : l’univers et la bêtise humaine ; en ce qui
concerne l’univers, je n’en ai pas acquis la certitude absolue.

  • Albert Einstein

Hello,

In message “Re: Private visibility should be removed from Ruby 2 [was:
Caveats with #method_missing]”
on Mon, 2 Oct 2006 13:21:27 +0900, “Tomasz W.”
[email protected] writes:

|Having a single well-defined interface was a big win for Smalltalk.
|Ruby is almost there, but not quite. What is the reason for Ruby
|to have “almost-Smalltalk” object model ?

Because Ruby is not Smalltalk. Really.

Let me elaborate a bit. Since I made the syntax of Ruby more
traditional than Smalltalk’s “everything is a message passing” style,
you can write traditional code in Ruby, for example:

print “hello world\n”

rather than

“hello world” printIt.

or something like that. If everything procedure is a method, what
should I do? There were several choices, and I chose “print” to be a
method with implicit receiver, but made it “private” to detect weird

“foobar”.print “hello world\n”

as an error. If you want to remove “private” from Ruby2, I expect
your proposal addresses the above issue.

						matz.

Yukihiro M. wrote:

“hello world” printIt.

or something like that. If everything procedure is a method, what
should I do? There were several choices, and I chose “print” to be a
method with implicit receiver, but made it “private” to detect weird

“foobar”.print “hello world\n”

as an error. If you want to remove “private” from Ruby2, I expect
your proposal addresses the above issue.

That’s interesting. The difference between function and method?

T.

On 10/1/06, Tomasz W. [email protected] wrote:

internals beyond repair.
the test framework. Including a “whitebox” mode in Test::Unit
encapsulation to facilitate meta hacks or get rid of some scenarios when
like Objective C, CLOS, Perl 5 and Python do not have it either.
Well although Smalltalk doesn’t have a mechanism for identifying
private methods at run-time, it’s common practice to document them
using categories. You see categories like ‘accessing’ and
‘private-accessing’.

Some Smalltalkers saw the lack of an enforced mechanism as a problem.
I can’t recall whether or not this came up in discussions in X3J20.

Speaking as an experienced Smalltalker; I was the secretary of X3J20,
which was the committee which developed the ANSI/ISO Smalltalk
language standard; I find David’s use cases for private methods far
more compelling that the arguments to remove the feature.

context - a single method is going to do something completely different
can do two completely different things.
All of this is true, but pretty much irrelevant as far as I can tell.

The model is already so complicated, that adding private/public
distinction to everything doesn’t affect it much.

Nor does it affect Ruby’s model much either.

Java object model is halfway between C++ and Smalltalk.

Maybe, whatever that means. Ruby does have a lot of similarities to
Smalltalk, but it’s quite different in other areas, including instance
behavior.

Objects still do not have single clear interfaces. If Foo is
a subclass of Bar, and both have a private method bar(),
then calls to some_foo_obj->bar() will behave differently
depending on whether the call was done from methods declared in
Foo class or from methods declared in Bar class.

I’m assuming that this is talking about Java.

Private methods are also automatically final, and therefore
they cannot be overriden. So private methods in Java still
have more to do with global functions than with regular methods.
Fortunately they at least got rid of private inheritance.

Huh?!??? If I define a private method, I certainly can’t call it as a
global function. The lack of overriden private methods in Java is due
to binding them early to methods, rather than because they are global,
which they aren’t.

exactly two - one for normal method calls, and a second for “implicit self”
method calls.

I can’t understand what you mean here by two interfaces. The methods
are in a single namespace, Making a method private affects what
happens with you invoke it using an explicit receiver.

And we don’t get much - defining method “private”
doesn’t provide much “protection” as there is a single namespace
for all methods, so it can be accidentally redefined in a subclass.

What we get is an ability to mark methods as being private at
run-time. The fact that this can be overriden by subclasses is a
feature which is harmonious with the basic philosophy of Ruby.

On the other hand methods cannot call private methods of different
object of the same class (or its subclasses), so objects with less-defined
interface and hidden implementation need to either hack around
access control inside own methods (ugly), or to have their internals public.

I’m lost here, what does ‘less-defined interface’ mean? What’s meant
by internals? Objects internals can only be accessed by methods,
Allowing methods to be private gives a tool to be used in class
design. If you don’t like private methods in your classes, just don’t
use them.

A very simple and very common example would be == method
that checks if both objects have the same class, and if so,
whether their fields are identical.

This is a pretty restricted view of ==, in general the two objects
don’t need ot be the same class, and the comparison involves either
public methods or a public conversion method.

While writing this reply I found something that really surprised me.
It is possible to get real private variables in Ruby, with private
per-class namespaces, not overridable, and working with other objects
of the same class, just like in C++/Java, by a few lines of
metaprogramming.
They are much better at encapsulation control than the official classes.

Why would I want to do that, Ruby is a better Ruby than Java or C++,
Java is a better Java than Ruby. C++ is the best C++, sad to say. .

So I think it would be good idea to have method visibility controls
removed because:

  • They do not fit Smalltalk-style OO, so removing them would make
    everything simpler

No, it would just remove a feature, and one that some Smalltalkers
wished that they had.

  • We get very limited benefit from them due to lack of C++/Java-style
    features like per-class namespaces for private methods (or even
    instance variables), and ability to call private methods of other
    objects of the same type

The benefit comes not from trying to do C++ style features, but from
providing a mechanism to denote methods which shouldn’t be called by
outsiders.

  • They make metaprogramming, unit testing etc. more difficult

I don’t see why or how.

  • They provide very limited control over visibility

So? At least they provide some control.

  • Very simple Ruby metaprogramming gives us much more powerful private
    variables anyway.

Simple?

  • I think it’s unlikely for current visibility control system to lead
    to any cool things. As far as I can tell it was never used for
    implementing any magic. There is no obvious way to add any features
    (even to get them to the level of that metaprogramming snippet)
    without greatly complicating the language.

And assuming that that’s something that someone would want to do, how
does having private methods get in the way?

  • Ruby 2 is exactly the right time for doing such changes

Well, put in a ruby change request, I doubt that it will get much
support.


Rick DeNatale

My blog on Ruby
http://talklikeaduck.denhaven2.com/

On 10/3/06, Yukihiro M. [email protected] wrote:

traditional than Smalltalk’s “everything is a message passing” style,
method with implicit receiver, but made it “private” to detect weird

“foobar”.print “hello world\n”

as an error. If you want to remove “private” from Ruby2, I expect
your proposal addresses the above issue.

New version of my proposal:

  • add implicit_self? method analogous to block_given?
  • method called by send/funcall would have implicit_self? set to
    false/true
  • remove “protected” visibility completely
  • make all currently private “real” methods (that is almost all
    private methods outside Kernel) public
  • change names public/private to object/global (or something like that)
  • officially discourage using “global” visibility for real methods

Yukihiro M. wrote:

should I do? There were several choices, and I chose “print” to be a
method with implicit receiver, but made it “private” to detect weird
“foobar”.print “hello world\n”
as an error.
All right. Just for that, I’m going into all my code and changing my
print statements to “foobar”.send(:print) statements.*

Seriously, though, you know, a pox on any body that wants to type
“foobar”.print “hello world\n”. (A more realistic mistake would, of
course, be “foobar”.puts.) In Io, the “OO” 5 print and the “procedural”
write(5) coexist just fine.

Wait, what are we talking about? You know, I should never respond to a
post without reading the whole thread. Well, whatever it is, I agree
with Jim W…

Devin

  • s/send/funcall/g… you get the idea.

'Seriously, though, you know, a pox on any body that wants to type
“foobar”.print “hello world\n”. ’

For sure, because Ruby syntax (at least for me), is different to
pretty every other language out there in that it is a lot cleaner.
And with multiple choices given you can easily avoid any quirky
syntax - although I am aware of the meta-meta languages
interpreted in ruby. :slight_smile:

Hi,

In message “Re: Private visibility should be removed from Ruby 2 [was:
Caveats with #method_missing]”
on Tue, 3 Oct 2006 16:57:02 +0900, “Tomasz W.”
[email protected] writes:

|> If you want to remove “private” from Ruby2, I expect
|> your proposal addresses the above issue.
|
|New version of my proposal:
|* add implicit_self? method analogous to block_given?
|* method called by send/funcall would have implicit_self? set to false/true
|* remove “protected” visibility completely
|* make all currently private “real” methods (that is almost all
|private methods outside Kernel) public
|* change names public/private to object/global (or something like that)
|* officially discourage using “global” visibility for real methods

Interesting. Will you analyze consequences of above changes?
How they make the language better? How they make it incompatible?

In addition, I didn’t understand how global “global” visibility, and
why it should be discouraged.

						matz.

On 10/3/06, Yukihiro M. [email protected] wrote:

Interesting. Will you analyze consequences of above changes?
How they make the language better? How they make it incompatible?

In addition, I didn’t understand how global “global” visibility, and
why it should be discouraged.

Before I analyze, a short preexplanation.
It is possible to have private and public methods that both do
something,
and this thing is different. It is impossible to do so directly, but if
an
object has private method “bar”, and method_missing catches “bar”,
this is the result. It is highly pathological and it happens sometimes.

And now the analysis.

|New version of my proposal:
|* add implicit_self? method analogous to block_given?

implicit_self? will make more metaprogramming possible and
metaprogrammed objects will behave more like native objects.

For example to do delegation we can:
def method_missing(*args, &blk)
if implicit_self?
@obj.funcall(*args, &blk)
else
@obj.send(*args, &blk)
end
end
In 1.8 one would normally use:
def method_missing(*args, &blk)
@obj.send(*args, &blk) # send calls private methods too
end
what is further from calling @obj directly, as it fails to raise an
exception if
“implicit self only” method is called in public context. Or even worse -
when private and public method with the same name do different thing
(see preexplanation).

implicit_self? seems like the simplest way to extend method_missing.
Two alternatives:

  • Calling different method, like function_missing. Most of the time,
    we want them to do the same thing.
  • Changing method_missing API - no obvious way to do so, as it already
    takes arbitrary number of arguments.

Another thing we get is soft visibility protection.

class Class
def soft_private(m)
old_m = instance_method(m)
c = self
define_method(m){|*args,&blk|
warn “#{c}##{m} should only be called in implicit self context”
unless implicit_self?
old_m.bind(self).call(*args, &blk)
}
end
end

class Foo
def hello
puts “Hello”
end
soft_private :hello
end

Foo.new.hello

=> Foo#hello should only be called in implicit self context

=> Hello

Such soft protection would be quite helpful in refactoring, as
it saves us from “run, die on first bug, fix, run again” cycle
we would have with hard protection.

I’m sure people will come with other ideas.
Basically, more metaprogrammability is good.

|* method called by send/funcall would have implicit_self? set to false/true

implicit_self? should obviously behave correctly with send/funcall,
as we want foo.send(:bar, *args, &blk) to be exactly identical to
foo.bar(*args, &blk)
and funcall(:bar, *args, &blk) to be exactly identical to bar(*args,
&blk)

|* remove “protected” visibility completely

Nobody uses “protected” (0 uses in standard library - can anybody find
some
code that actually uses it ?). Nobody understands it.
And it seriously complicates metaprogramming.

I tried to check what protected really do.

class Foo
def hello
puts “Hello”
end
protected :hello
def world
hello
end
end

a = Foo.new

a.hello # NoMethodError: protected method hello' called for #<Foo:0xb7cdffb4> a.world # => "Hello" a.instance_eval { hello } # => "Hello" a.instance_eval { a.hello } # NoMethodError: protected method hello’
called for #Foo:0xb7cdffb4
a.class.instance_eval { a.hello } # NoMethodError: protected method
`hello’ called for #Foo:0xb7cdffb4

Ruby FAQ says that “protected methods are callable only from within
their own class or its subclasses”, but whether self is Foo or
instance of Foo, I cannot call protected methods of Foo. I understood
the explanation that it does something like checking in the caller
whether self.is_a? class_where_method_was_defined, but it does
something different.

How was it supposed to work ? Am I the only one with a problem here ?

And getting rid of it make metaprogramming easier.
One example: I have no idea how to make delegation by method_missing
that behaves correctly with public, protected and private methods.
Delegation would either have to ignore visibility (what is wrong -
see preexplanation again) or fail to forward some methods.

|* make all currently private “real” methods (that is almost all
|private methods outside Kernel) public

If something is part of public interface, not a global function or an
aux method,
what’s the reason for forcing instance_eval/funcall etc. ?
And as far as I can tell, almost every use of private outside Kernel
is for methods that are parts of public interface (mostly methods of
Module).

People often want to call private methods of Module for metaprogramming.
I guess it’s the same with other such cases.

|* change names public/private to object/global (or something like that)
|* officially discourage using “global” visibility for real methods

Terminology Ruby uses is highly non-standard.
In Java/C++/etc. private implies “in private namespace”
and is typica//y used for methods that are not part of the public
interface.

In Ruby “private” means something completely different
(more like Java/C++ “protected”), and is used for different things -
public global functions and some public “implicit self only” functions
(like those from Module). And I fail to see any real use
for “implicit self only” methods (the proposal wants to make them all
public).

I guess some people use “private” for aux methods (probably due to
getting
used to “private” in Java/C++), but it’s not very good for that purpose.
Ruby private doesn’t introduce private namespace, doesn’t protect
against
accidental overriding, doesn’t prevent calling a method if someone
really wants to,
and definitely doesn’t detect the problem at compile time like in
C++/Java.
And unlike Java/C++ objects, Ruby objects have typically >100 public
methods
already (still less than Smalltalk but getting there),
so what problem could possibly a few more aux methods introduce ?

So the only good use of “private” left is for global function, and the
name
is non-standard and doesn’t reflect what “private” does or what it’s
used for.
So why not change it ?

I would discourage using “private” for actual methods for previously
metioned reasons:

  • it can lead to pathological situations where the same method does
    different thing depending on context. We can still have the problem
    with global functions (Foo.foo called bar, wanting to go through
    method_missing, but someone defined global function bar), but we limit
    it a bit
  • if method is part of public interface (like in Module) it only makes
    programming (especially metaprogramming) more difficult
  • for aux methods, it has few benefits of “private” in other languages

I guess namechange and officially discouragement part of the proposal
may
be the least convincing and depend a lot on how one feels about it.
The rest is pretty solid as far as I see.

On Oct 3, 2006, at 7:20 AM, Tomasz W. wrote:

|* remove “protected” visibility completely

Nobody uses “protected” (0 uses in standard library - can anybody
find some
code that actually uses it ?). Nobody understands it.

I understand it just fine and have used it before. Here’s an example
where it might be needed:

class Name < Struct.new(:first, :last)
def full
“#{first} #{last}”
end

?> def last_first

"#{last}, #{first}"

end

?> def sortable

[last, first]

end
protected :sortable

?> def <=>(other)

sortable <=> other.sortable

end
end
=> nil

names = [ %w[James G.],
?> %w[Dana Gray],
?> %w[Joe Fair] ].inject(Array.new) { |a, n| a + [Name.new
(*n)] }
=> [#<struct Name first=“James”, last=“Gray”>, #<struct Name
first=“Dana”, last=“Gray”>, #<struct Name first=“Joe”, last=“Fair”>]

OK, because protected allows the call to other.sortable:

?> names.sort.map { |n| n.last_first }
=> [“Fair, Joe”, “Gray, Dana”, “Gray, James”]

Not OK, because sortable is for internal use only:

?> names.first.sortable
NoMethodError: protected method `sortable’ called for #<struct Name
first=“James”, last=“Gray”>
from (irb):28
from :0

James Edward G. II

On 10/3/06, James Edward G. II [email protected] wrote:

On Oct 3, 2006, at 7:20 AM, Tomasz W. wrote:

|* remove “protected” visibility completely

Nobody uses “protected” (0 uses in standard library - can anybody
find some
code that actually uses it ?). Nobody understands it.

I understand it just fine

Then could you explain, whether:

a = Name.new(‘foo’, ‘bar’)
a.instance_eval { a.sortable } # correct or not ?
Name.instance_eval { a.sortable } # correct or not ?

are supposed to work or not, and why ?

and have used it before. Here’s an example where it might be needed:
[example omitted]

This use is pretty nifty, as it saves you a class check:
raise ArgumentError.new(“comparison of #{self.class} with
#{other.class} failed”) unless oher.is_a? Name

Except that it breaks later.

Think how can you make an object that forwards all messages it gets to
a Name object,
and use it as if it was real Name. This is often very useful (in
debugging, logging,
lazy loading, remote execution, mock objects in unit testing etc.) and
works with
pretty much all objects.

But it won’t work correctly with your Name class - forwarding
(method_missing)
must either do public method call (send in 1.9), which fails as
sortable is not public,
or private method call (funcall in 1.9 or instance_eval+send in 1.8),
which removes the class check, and therefore changes behaviour of <=>.

So Name fails to behave like a nice Ruby class.
I don’t see how such problems can be reasonably fixed.

To forward protected calls, we would need some way of getting caller
context, what sounds pretty bad, as caller context is not simply class
of
caller’s self, it can be some of its superclasses etc,
and we need to make a version of send that uses this context.

Then we would do :
def method_missing(*args, &blk)
@obj.send_with_context(caller_context, *args, &blk)
end

I think neither accepting unability to delegate nor actually having
contexts
like that is acceptable trade-off for the little use “protected” has.

Yukihiro M. wrote:

or something like that. If everything procedure is a method, what
should I do? There were several choices, and I chose “print” to be a
method with implicit receiver, but made it “private” to detect weird

“foobar”.print “hello world\n”

as an error. If you want to remove “private” from Ruby2, I expect
your proposal addresses the above issue.

With the proposal of #funcall, I think a lot of people were confounded.
What did fucntions have to do with private vs. public. But it becomes
increasing clear what matz has done, for better of worse. He noticed a
function is a method without a receiver, which by all accounts is
essentially the same as a method that can take no other reciever but
self. In turn, that fits the concpet of a so-called “private” method.
It’s a very interesting sort of conscilence. Unfortuately the mixed
terminolgy can be quite confusing since the two aren’t readily
associated. How many Rubyists would know that saying “private function”
is redundant?

Honestly one the syntax features that UI alwasy found out of place was
‘private’ keyword. Here was essentially a DECLARATION in a language
that sought to avoid declarations. I always felt at the least it would
be better to make it take a block, ‘private do … end’. And I see that
could just as well be ‘function do … end’. But why not address each
function definition?

defunc foo

end

I’m curious why the public/private nomenclature was chosen over
‘method/function’ and why#funcall is being suggested now (moving away
from that choice?).

From a larger perspective, it strikes me that different approaches were
taken for variables as opposed to methods. It is equally possible to
think of instance variables as private attributes. In this case,
un;like methods, it is the private entities that take precedence and
public variables are provided via additional interface methods. The
distiction here is one of syntax made via the @ prefix. Concievably the
same prefix mechism could have been used for private methods.

def @foo

end

T.