Forum: Ruby-core [ruby-trunk - Feature #7854][Open] New method Symbol[string]

Posted by Student Jr (student)
on 2013-02-15 00:53
(Received via mailing list)
Issue #7854 has been reported by Student (Nathan Zook).

----------------------------------------
Feature #7854: New method Symbol[string]
https://bugs.ruby-lang.org/issues/7854

Author: Student (Nathan Zook)
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: next minor


I propose a new class method [] on Symbol.  If a symbol s already exists 
such that s.to_s == string, then s is returned.  If not, nil is 
returned.

The inspiration for this method is a question I was asked, and the 
answer I was given:  "Why would you want to turn a tainted string into a 
symbol?"  "I don't--I want to access an existing symbol with tainted 
data".  Symbol[] accesses the symbol table like hash[] accesses the 
elements of a hash.

I believe that this completely addresses the problems behind tickets 
#7791 and #7839.  I believe that it is a more intuitive solution than my 
proposal #7795, and I believe that this will also be useful for 
YAML.safe_load and similar initiatives.
Posted by Matthew Kerwin (mattyk)
on 2013-02-15 02:43
(Received via mailing list)
Issue #7854 has been updated by phluid61 (Matthew Kerwin).


Note that this is closely related to #7795 (Symbol.defined? and/or 
to_existing_symbol)

In existing code, Symbol.[] could be implemented as:

    class Symbol
      def self.[](string)
        all_symbols.find{|sym| sym.to_s == string}
      end
    end
----------------------------------------
Feature #7854: New method Symbol[string]
https://bugs.ruby-lang.org/issues/7854#change-36300

Author: Student (Nathan Zook)
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: next minor


I propose a new class method [] on Symbol.  If a symbol s already exists 
such that s.to_s == string, then s is returned.  If not, nil is 
returned.

The inspiration for this method is a question I was asked, and the 
answer I was given:  "Why would you want to turn a tainted string into a 
symbol?"  "I don't--I want to access an existing symbol with tainted 
data".  Symbol[] accesses the symbol table like hash[] accesses the 
elements of a hash.

I believe that this completely addresses the problems behind tickets 
#7791 and #7839.  I believe that it is a more intuitive solution than my 
proposal #7795, and I believe that this will also be useful for 
YAML.safe_load and similar initiatives.
Posted by Student Jr (student)
on 2013-02-15 02:55
(Received via mailing list)
Issue #7854 has been updated by Student (Nathan Zook).


It could, but it would be extraordinarily slow, as all_symbols returns 
an array.

----------------------------------------
Feature #7854: New method Symbol[string]
https://bugs.ruby-lang.org/issues/7854#change-36302

Author: Student (Nathan Zook)
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: next minor


I propose a new class method [] on Symbol.  If a symbol s already exists 
such that s.to_s == string, then s is returned.  If not, nil is 
returned.

The inspiration for this method is a question I was asked, and the 
answer I was given:  "Why would you want to turn a tainted string into a 
symbol?"  "I don't--I want to access an existing symbol with tainted 
data".  Symbol[] accesses the symbol table like hash[] accesses the 
elements of a hash.

I believe that this completely addresses the problems behind tickets 
#7791 and #7839.  I believe that it is a more intuitive solution than my 
proposal #7795, and I believe that this will also be useful for 
YAML.safe_load and similar initiatives.
Posted by drbrain (Eric Hodel) (Guest)
on 2013-02-15 04:59
(Received via mailing list)
Issue #7854 has been updated by drbrain (Eric Hodel).


To make this proposal useful all existing libraries must be updated to 
use the method to create symbols.

Other proposals such as #7839 or #7791 allow rubyists to avoid a symbol 
creation DoS without forcing them to ask for new releases of a library.
----------------------------------------
Feature #7854: New method Symbol[string]
https://bugs.ruby-lang.org/issues/7854#change-36305

Author: Student (Nathan Zook)
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: next minor


I propose a new class method [] on Symbol.  If a symbol s already exists 
such that s.to_s == string, then s is returned.  If not, nil is 
returned.

The inspiration for this method is a question I was asked, and the 
answer I was given:  "Why would you want to turn a tainted string into a 
symbol?"  "I don't--I want to access an existing symbol with tainted 
data".  Symbol[] accesses the symbol table like hash[] accesses the 
elements of a hash.

I believe that this completely addresses the problems behind tickets 
#7791 and #7839.  I believe that it is a more intuitive solution than my 
proposal #7795, and I believe that this will also be useful for 
YAML.safe_load and similar initiatives.
Posted by Student Jr (student)
on 2013-02-15 05:59
(Received via mailing list)
Issue #7854 has been updated by Student (Nathan Zook).


=begin
Ticket #7839 requires the manipulation of global state.  I'm not sure 
why I have to explain that this is a REALLY bad idea.

Ticket #7791 has two possible implementations.  One is to GC symbols 
globally.  This would require treating not just symbols like objects, 
but methods (whose names are in fact symbols) as well.  I do not believe 
that methods are even currently part of the object system.

Another implementation would be to divide symbols into two kinds 
depending on how they are created.  The theory being that symbols used 
for method names would be immune to GC.  The first problem with this is 
that there is no reason to believe that method declarations are the 
first place that a particular symbol would be declared.  The second is 
that dynamic method creation is an important part of ruby.  If the goal 
is to protect against memory leaks in this fashion, it is not at all 
certain that the leak does not extend into the realm of method creation.

In other words, both of these implementations involve complex changes to 
the guts of Ruby, and lead to the likelihood of a significant 
behavioural fork with other rubys.  (Not to mention the relatively high 
risk of bug introduction.)  Since this is a security feature, I think 
that it is important to lead the way in a direction that is easy to 
import to other rubies (and also to backport as a security patch!)  I 
expect Symbol[] to have a very straightforward implementation that is 
well-isolated from the rest of Ruby, with the possible exception of 
YAML.*load*, which might well benefit from such a feature.

As for requiring the libraries to all be updated to make use of this 
feature--I consider that to be a good thing.  #7839 creates a change in 
MRI's behaviour that WILL break apparently "safe" use of existing 
libraries.  #7791 necessarily dramatically affects Symbol's runtime 
performance, and thus means that any highly-tuned ruby is going to have 
issues--assuming that no bugs occur, and that the other rubys pick it 
up.

Furthermore, for most, perhaps even all, libraries, (({grep -R to_sym 
lib})) is going to tell you what you need to examine to make use of this 
feature.  Certainly, it would be nice to avoid having to do such things, 
but because of the recent exploits, the more security-minded portion of 
the community (such as myself) is ALREADY nervously poking around in 
their libraries.

This feature gives the community a clean way to patch questionable code, 
which is itself relatively easy to identify in manner that makes it easy 
for other rubies to quickly follow.  I do not believe that the other 
proposals do.
=end
----------------------------------------
Feature #7854: New method Symbol[string]
https://bugs.ruby-lang.org/issues/7854#change-36307

Author: Student (Nathan Zook)
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: next minor


I propose a new class method [] on Symbol.  If a symbol s already exists 
such that s.to_s == string, then s is returned.  If not, nil is 
returned.

The inspiration for this method is a question I was asked, and the 
answer I was given:  "Why would you want to turn a tainted string into a 
symbol?"  "I don't--I want to access an existing symbol with tainted 
data".  Symbol[] accesses the symbol table like hash[] accesses the 
elements of a hash.

I believe that this completely addresses the problems behind tickets 
#7791 and #7839.  I believe that it is a more intuitive solution than my 
proposal #7795, and I believe that this will also be useful for 
YAML.safe_load and similar initiatives.
Posted by ko1 (Koichi Sasada) (Guest)
on 2013-02-18 01:35
(Received via mailing list)
Issue #7854 has been updated by ko1 (Koichi Sasada).

Assignee set to matz (Yukihiro Matsumoto)


----------------------------------------
Feature #7854: New method Symbol[string]
https://bugs.ruby-lang.org/issues/7854#change-36476

Author: Student (Nathan Zook)
Status: Open
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version: next minor


I propose a new class method [] on Symbol.  If a symbol s already exists 
such that s.to_s == string, then s is returned.  If not, nil is 
returned.

The inspiration for this method is a question I was asked, and the 
answer I was given:  "Why would you want to turn a tainted string into a 
symbol?"  "I don't--I want to access an existing symbol with tainted 
data".  Symbol[] accesses the symbol table like hash[] accesses the 
elements of a hash.

I believe that this completely addresses the problems behind tickets 
#7791 and #7839.  I believe that it is a more intuitive solution than my 
proposal #7795, and I believe that this will also be useful for 
YAML.safe_load and similar initiatives.
Posted by Matthew Kerwin (mattyk)
on 2013-03-07 04:55
(Received via mailing list)
Issue #7854 has been updated by phluid61 (Matthew Kerwin).

File symbol_lookup.patch added

=begin
I've attached a patch that defines ((%Symbol[str]%)).  If ((|str|)) is a 
string and there exists a symbol such that (({symbol.to_s == str})), it 
returns that symbol.  Otherwise it returns ((|nil|)).  Raises a 
TypeError if ((|str|)) is not a string.

I also made a unit test, currently available as a gist: 
https://gist.github.com/phluid61/5105458
=end
----------------------------------------
Feature #7854: New method Symbol[string]
https://bugs.ruby-lang.org/issues/7854#change-37348

Author: Student (Nathan Zook)
Status: Open
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version: next minor


I propose a new class method [] on Symbol.  If a symbol s already exists 
such that s.to_s == string, then s is returned.  If not, nil is 
returned.

The inspiration for this method is a question I was asked, and the 
answer I was given:  "Why would you want to turn a tainted string into a 
symbol?"  "I don't--I want to access an existing symbol with tainted 
data".  Symbol[] accesses the symbol table like hash[] accesses the 
elements of a hash.

I believe that this completely addresses the problems behind tickets 
#7791 and #7839.  I believe that it is a more intuitive solution than my 
proposal #7795, and I believe that this will also be useful for 
YAML.safe_load and similar initiatives.
Posted by Student Jr (student)
on 2013-03-07 07:05
(Received via mailing list)
Issue #7854 has been updated by Student (Nathan Zook).


:)

----------------------------------------
Feature #7854: New method Symbol[string]
https://bugs.ruby-lang.org/issues/7854#change-37351

Author: Student (Nathan Zook)
Status: Open
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version: next minor


I propose a new class method [] on Symbol.  If a symbol s already exists 
such that s.to_s == string, then s is returned.  If not, nil is 
returned.

The inspiration for this method is a question I was asked, and the 
answer I was given:  "Why would you want to turn a tainted string into a 
symbol?"  "I don't--I want to access an existing symbol with tainted 
data".  Symbol[] accesses the symbol table like hash[] accesses the 
elements of a hash.

I believe that this completely addresses the problems behind tickets 
#7791 and #7839.  I believe that it is a more intuitive solution than my 
proposal #7795, and I believe that this will also be useful for 
YAML.safe_load and similar initiatives.
Posted by Nobuyoshi Nakada (nobu)
on 2013-03-07 13:50
(Received via mailing list)
Issue #7854 has been updated by nobu (Nobuyoshi Nakada).


To obtain existing symbol, rb_check_id() is already available, so you 
don't have to add new extern function.
----------------------------------------
Feature #7854: New method Symbol[string]
https://bugs.ruby-lang.org/issues/7854#change-37356

Author: Student (Nathan Zook)
Status: Open
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version: next minor


I propose a new class method [] on Symbol.  If a symbol s already exists 
such that s.to_s == string, then s is returned.  If not, nil is 
returned.

The inspiration for this method is a question I was asked, and the 
answer I was given:  "Why would you want to turn a tainted string into a 
symbol?"  "I don't--I want to access an existing symbol with tainted 
data".  Symbol[] accesses the symbol table like hash[] accesses the 
elements of a hash.

I believe that this completely addresses the problems behind tickets 
#7791 and #7839.  I believe that it is a more intuitive solution than my 
proposal #7795, and I believe that this will also be useful for 
YAML.safe_load and similar initiatives.
Posted by Matthew Kerwin (mattyk)
on 2013-03-08 00:48
(Received via mailing list)
Issue #7854 has been updated by phluid61 (Matthew Kerwin).

File symbol_lookup2.patch added

nobu (Nobuyoshi Nakada) wrote:
> To obtain existing symbol, rb_check_id() is already available, so you don't have 
to add new extern function.

Thank you for the feedback.  With that in mind, I've made a less 
invasive version which only modifies string.c

Please let me know if my enthusiasm gets annoying.  :)
----------------------------------------
Feature #7854: New method Symbol[string]
https://bugs.ruby-lang.org/issues/7854#change-37370

Author: Student (Nathan Zook)
Status: Open
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version: next minor


I propose a new class method [] on Symbol.  If a symbol s already exists 
such that s.to_s == string, then s is returned.  If not, nil is 
returned.

The inspiration for this method is a question I was asked, and the 
answer I was given:  "Why would you want to turn a tainted string into a 
symbol?"  "I don't--I want to access an existing symbol with tainted 
data".  Symbol[] accesses the symbol table like hash[] accesses the 
elements of a hash.

I believe that this completely addresses the problems behind tickets 
#7791 and #7839.  I believe that it is a more intuitive solution than my 
proposal #7795, and I believe that this will also be useful for 
YAML.safe_load and similar initiatives.
Posted by Nobuyoshi Nakada (nobu)
on 2013-03-08 10:36
(Received via mailing list)
Issue #7854 has been updated by nobu (Nobuyoshi Nakada).


Why does it have -1 arity?

And I don't think it's harmful if the method allows a Smbol too.
----------------------------------------
Feature #7854: New method Symbol[string]
https://bugs.ruby-lang.org/issues/7854#change-37387

Author: Student (Nathan Zook)
Status: Open
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version: next minor


I propose a new class method [] on Symbol.  If a symbol s already exists 
such that s.to_s == string, then s is returned.  If not, nil is 
returned.

The inspiration for this method is a question I was asked, and the 
answer I was given:  "Why would you want to turn a tainted string into a 
symbol?"  "I don't--I want to access an existing symbol with tainted 
data".  Symbol[] accesses the symbol table like hash[] accesses the 
elements of a hash.

I believe that this completely addresses the problems behind tickets 
#7791 and #7839.  I believe that it is a more intuitive solution than my 
proposal #7795, and I believe that this will also be useful for 
YAML.safe_load and similar initiatives.
Posted by Matthew Kerwin (mattyk)
on 2013-03-09 09:59
(Received via mailing list)
Issue #7854 has been updated by phluid61 (Matthew Kerwin).

File symbol_lookup3.patch added
File symbol_lookup3_warn.patch added

nobu (Nobuyoshi Nakada) wrote:
> Why does it have -1 arity?
>
> And I don't think it's harmful if the method allows a Smbol too.

To the first: an oversight on my part, there's no real reason.  I have 
rewritten it with an arity of 1.

To the second: I can easily change it to allow a Symbol as well. 
However since the original discussion that spawned this proposal was 
focused on the idea of not creating unwanted/unneeded Symbols, I wonder 
should it emit a warning in that case?

I see now, too, that I was rather overzealous in my original attempts. 
I should have realised most of the hard work has already been done.  :)

Now I suppose it's up to Matz to approve it or not.
----------------------------------------
Feature #7854: New method Symbol[string]
https://bugs.ruby-lang.org/issues/7854#change-37420

Author: Student (Nathan Zook)
Status: Open
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version: next minor


I propose a new class method [] on Symbol.  If a symbol s already exists 
such that s.to_s == string, then s is returned.  If not, nil is 
returned.

The inspiration for this method is a question I was asked, and the 
answer I was given:  "Why would you want to turn a tainted string into a 
symbol?"  "I don't--I want to access an existing symbol with tainted 
data".  Symbol[] accesses the symbol table like hash[] accesses the 
elements of a hash.

I believe that this completely addresses the problems behind tickets 
#7791 and #7839.  I believe that it is a more intuitive solution than my 
proposal #7795, and I believe that this will also be useful for 
YAML.safe_load and similar initiatives.
Please log in before posting. Registration is free and takes only a minute.
Existing account (Switch to SSL-encrypted connection)
NEW: Do you have a Google/GoogleMail or Yahoo account? No registration required!
Log in with Google account | Log in with Yahoo account
No account? Register here.