Forum: Ruby-core [ruby-trunk - Feature #5741][Open] Secure Erasure of Passwords

Ada56d152c1abe0eba65a5ff62ed5387?d=identicon&s=25 Martin Bosslet (martin_b)
on 2011-12-10 22:03
(Received via mailing list)
Issue #5741 has been reported by Martin Bosslet.

----------------------------------------
Feature #5741: Secure Erasure of Passwords
http://redmine.ruby-lang.org/issues/5741

Author: Martin Bosslet
Status: Open
Priority: Normal
Assignee:
Category:
Target version: 2.0.0


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
18813f71506ebad74179bf8c5a136696?d=identicon&s=25 Eric Wong (Guest)
on 2011-12-10 23:06
(Received via mailing list)
Martin Bosslet <Martin.Bosslet@googlemail.com> wrote:
> 3. Are there better alternative ways how we could achieve this?

You can use IO#read / StringIO#read to overwrite an existing String:

----------- /tmp/pass.rb -----------------
pass = ""
$stdin.sysread(256, pass) # assuming a line-buffered terminal
io = StringIO.new("\0" * pass.bytesize)
io.read(pass.bytesize, pass)
p pass
Process.kill(:ABRT, $$)
sleep # wait for SIGABRT to hit us
------------------------------------------
$ ulimit -c unlimited
# using "hunter2" as my password:
$ ruby /tmp/pass.rb
hunter2
"\x00\x00\x00\x00\x00\x00\x00\x00"
Aborted (core dumped)
$ strings core | grep hunter
<nothing>

Unfortunately, using things like $stdin.gets to read passwords:

  pass = $stdin.gets

Instead of:

  $stdin.sysread(256, pass)

...Can leave a copy of the password in userspace IO buffers (stdio or
rb_io_t).  One has to avoid touching userspace IO buffering layers to
avoid leaving a trace of them in a core dump.
18813f71506ebad74179bf8c5a136696?d=identicon&s=25 Eric Wong (Guest)
on 2011-12-10 23:21
(Received via mailing list)
Eric Wong <normalperson@yhbt.net> wrote:
> You can use IO#read / StringIO#read to overwrite an existing String:

String#tr! (on binary strings) can also work:

--------------------------------------
# -*- encoding: binary -*-
pass = ""
$stdin.sysread(256, pass)
pass.tr!("\0-\xff", "\0")
p pass
Process.kill(:ABRT, $$)
sleep

Maybe String#gsub! would, too, but regexp engines are quite complex and
may do buffering/copying of its own (I don't know regexp implementation
details well).  tr/tr! is pretty simple...

I'd still trust IO#sysread the most since passing a String buffer to it
is for optimization.  It would be stupid (and thus highly unlikely :)
that any Ruby implementation would copy/free() a buffer passed for IO
and replace it with another buffer internally.
Ada56d152c1abe0eba65a5ff62ed5387?d=identicon&s=25 Martin Bosslet (martin_b)
on 2011-12-11 17:06
(Received via mailing list)
Issue #5741 has been updated by Martin Bosslet.


Thanks for investigating! I looked into it, here's what
I found.

String#tr! doesn't work unfortunately. The original string's buffer
is deleted and the replacement takes place in a newly allocated
buffer [1]. I verified this in several core dumps, traces of the
password remain.

But the solution with IO#read seems to work indeed. The contents
of the string are overwritten in place. My only concern is that
erasing the string in this manner is not a straightforward task.
You need to allocate a StringIO and you must pay attention to
using String#bytesize instead of String#size, otherwise you might
be fooled by multi-byte encodings. Another issue is that it
would heavily rely on a side effect of the CRuby implementation
that could change over time and is maybe not applicable in other
implementations.

Having something explicit like String#clear_secure is easier to
use, independent of side effects and self-documenting. However,
as Eric Hodel pointed out to me, using clear_secure will erase
any String that referenced the original buffer. This is what we
actually want, but it should at least be documented to avoid
surprises.

[1] https://github.com/ruby/ruby/blob/trunk/string.c#L5121
----------------------------------------
Feature #5741: Secure Erasure of Passwords
http://redmine.ruby-lang.org/issues/5741

Author: Martin Bosslet
Status: Open
Priority: Normal
Assignee:
Category:
Target version: 2.0.0


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
Ada56d152c1abe0eba65a5ff62ed5387?d=identicon&s=25 Martin Bosslet (martin_b)
on 2011-12-13 19:56
(Received via mailing list)
Issue #5741 has been updated by Martin Bosslet.


Links underlining that this is serious:

http://www.chiark.greenend.org.uk/~sgtatham/putty/...
(thanks nahi!)
https://www.owasp.org/index.php/OWASP_Application_... (search
for 'memory')
https://www.owasp.org/index.php/Insecure_Compiler_... (cf.
reference to "Writing Secure Code")
http://philosecurity.org/research/cleartext-passwords-linux

They list another situation when this can be exploited that I totally
forgot: when memory gets swapped
out to disk.


----------------------------------------
Feature #5741: Secure Erasure of Passwords
http://redmine.ruby-lang.org/issues/5741

Author: Martin Bosslet
Status: Open
Priority: Normal
Assignee:
Category:
Target version: 2.0.0


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
7560a011e6b05e1388b0c77a2b3b49eb?d=identicon&s=25 Hiroshi Nakamura (Guest)
on 2011-12-19 06:49
(Received via mailing list)
Attachment: signature.asc (485 Bytes)
Here's my 2 yen.

> 1. Would you agree that we need this functionality?

Good to have, but it would be hard to use properly.

> 2. Where would we ideally place it? I'm not sure whether
>    String is the perfect place, but on the other hand, String
>    is the only place where we have access to the implementation
>    details.
> 3. Are there better alternative ways how we could achieve this?

I think you're going to adopt opt-in way, so library/application
developers must add String#clear call after using the password, right?

If it's opt-in, new specific class would be enough I think.  In this
way, we can control the memory copy (part-of, of course) and eventually
we might be able to split buffers into multiple parts that have
different addresses.

class SecureByteBuffer
  def ==(rhs)
    raise unless rhs.is_a?(SecureByteBuffer)
    ...
  end

  def clear
    ...
  end
end

But the most hard part I think is how we construct this Object...

Martin, do you have concrete examples which needs secure erasure of
passwords?  Only I can think of now is ossl_pem_passwd_cb in
ext/openssl.  It gets password as a String from a callback block but it
would be good to add a feature to read from STDIN directly, without
creating the String object.

Best regards,
// NaHi
Ada56d152c1abe0eba65a5ff62ed5387?d=identicon&s=25 Martin Bosslet (martin_b)
on 2011-12-20 01:09
(Received via mailing list)
Issue #5741 has been updated by Martin Bosslet.


Thanks for your thoughts!

>  I think you're going to adopt opt-in way, so library/application
>  developers must add String#clear call after using the password, right?

Yes, I think opt-in is the only way to ensure there are no negative side
effects on existing code.

>
>    def clear
>      ...
>    end
>  end

That is an interesting approach, I'd love to catch up on this. So
far, I had four ideas for implementing such a thing.

1. String#clear
+ it's at the "root" of the problem
+ adoption of the feature will merely ask for calling #clear on password
variables at the right places
- invasive because rarely needed
- memory side effects of other string ops might be hard to keep in sync

2. Decorator/Visitor/Subclass of String
+ Unobtrusive
- But adoption requires a lot more code changes, not only the "erasure"
part but also the parts
where the passwords are created

3. Module/Class function class, "helper function"
+ Unobtrusive
- Adoption requires only the additional calls at the place where erasure
is needed
- Where to put this?

4. Separate class, not necessarily String-like
+ Unobtrusive
- Adoption again requires lots of changes (like in 2.)
- Will it be accepted by users?

If I understand correctly your proposal would tend towards 4.?
I'm open to anything, as long as there's something :) I haven't
looked at the memory copy issue yet in detail, so I can't really
tell for sure whether it's possible at all to control it entirely.
Might as well be the case that we would need to document safe
handling of passwords properly to ensure the desired outcome. I'd
volunteer for that once we have found a stable solution.

>  But the most hard part I think is how we construct this Object...

I know... :)

>  Martin, do you have concrete examples which needs secure erasure of
>  passwords?  Only I can think of now is ossl_pem_passwd_cb in
>  ext/openssl.  It gets password as a String from a callback block but it
>  would be good to add a feature to read from STDIN directly, without
>  creating the String object.

Besides the cases within the OpenSSL extension that you mentioned, I was
mainly thinking about authentication in web apps. You typically retrieve
the password String from a request there. I need to dig deeper to come
up with some concrete examples but I assume the idea should be clear.

Besides that I kept thinking about what you said the other day, whether
I'm only concerned about passwords or also about private keys and the
like.
I was only thinking passwords when I wrote this, but you're absolutely
right,
private keys in memory are equally bad [1].

It happens a lot that we use private keys in this way, SSH, SSL, digital
signatures,
encryption, you name it. And the biggest problem is that the private
keys are
not strings in general, so the String solution wouldn't apply directly.
But I'm
confident that we could at least apply the same technique. Fortunately,
OpenSSL takes
care of this for us in all places where we rely on OpenSSL (with its
OPENSSL_cleanse
function), but still we should investigate if there aren't any loopholes
left.

[1]
http://citeseerx.ist.psu.edu/viewdoc/download?doi=...

----------------------------------------
Feature #5741: Secure Erasure of Passwords
https://bugs.ruby-lang.org/issues/5741

Author: Martin Bosslet
Status: Open
Priority: Normal
Assignee:
Category:
Target version: 2.0.0


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
F24ff61beb80aa5f13371aa22a35619c?d=identicon&s=25 mame (Yusuke Endoh) (Guest)
on 2012-03-28 18:05
(Received via mailing list)
Issue #5741 has been updated by mame (Yusuke Endoh).

Status changed from Open to Assigned
Assignee set to matz (Yukihiro Matsumoto)

I think that adding a method to String requires matz's approval.
If you propose to add a method to others, such as openssl, you
can do it at your discretion.

--
Yusuke Endoh <mame@tsg.ne.jp>
----------------------------------------
Feature #5741: Secure Erasure of Passwords
https://bugs.ruby-lang.org/issues/5741#change-25305

Author: MartinBosslet (Martin Bosslet)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
Ada56d152c1abe0eba65a5ff62ed5387?d=identicon&s=25 Martin Bosslet (martin_b)
on 2012-03-28 21:59
(Received via mailing list)
Issue #5741 has been updated by MartinBosslet (Martin Bosslet).


mame (Yusuke Endoh) wrote:
> I think that adding a method to String requires matz's approval.
> If you propose to add a method to others, such as openssl, you
> can do it at your discretion.
>

Sure - I would appreciate to have more opinions on how this can be
done best. I haven't really had the time to think it through yet,
but unfortunately I see two major obstacles for either approach.

1. I'm having doubts if this could be handled reliably within String
   itself due to copy on write behavior. So when we finally call
   something like String#clear, we would also need to clear any
   proliferated instances, too. I haven't analyzed it yet, maybe
   someone could tell me if that's possible at all?

2. If we implement the feature separately (SecureByteBuffer), there
   is a serious chicken & egg problem: how can we feed the password
   to the implementation without using a String first? It's not as
   easy as it may sound and it would require massive changes in
   existing code - I fear this could be too invasive for anyone
   to adopt it...

Any thoughts?
----------------------------------------
Feature #5741: Secure Erasure of Passwords
https://bugs.ruby-lang.org/issues/5741#change-25316

Author: MartinBosslet (Martin Bosslet)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
F1d6cc2b735bfd82c8773172da2aeab9?d=identicon&s=25 Nobuyoshi Nakada (nobu)
on 2012-03-28 23:47
(Received via mailing list)
Issue #5741 has been updated by nobu (Nobuyoshi Nakada).


What kind of methods will be needed for "Secure Password", do you think?
Is there any reason that it has to be real String instance?

For example, I guess it should be BINARY or US-ASCII always, so
shouldn't have encode method.
sub string of it makes no sense, so no [], slice, and etc.
sub, tr, upcase, et al too.
Of course, to_sym must be prohibited as it makes the string permanent.
----------------------------------------
Feature #5741: Secure Erasure of Passwords
https://bugs.ruby-lang.org/issues/5741#change-25321

Author: MartinBosslet (Martin Bosslet)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
Ada56d152c1abe0eba65a5ff62ed5387?d=identicon&s=25 Martin Bosslet (martin_b)
on 2012-03-29 01:45
(Received via mailing list)
Issue #5741 has been updated by MartinBosslet (Martin Bosslet).


nobu (Nobuyoshi Nakada) wrote:
> What kind of methods will be needed for "Secure Password", do you think?
> Is there any reason that it has to be real String instance?
>
> For example, I guess it should be BINARY or US-ASCII always, so shouldn't have
encode method.
> sub string of it makes no sense, so no [], slice, and etc.
> sub, tr, upcase, et al too.
> Of course, to_sym must be prohibited as it makes the string permanent.

Totally agreed. No, there's no reason that it has to be a String
instance. As
you already stated, it's more like a "byte array" - no need for encoding
awareness and it shouldn't respond to any methods that potentially hand
out
parts of its contents.

I thought a bit more how to implement such a "Secure Password" class.
How
can we "trap" the password in it initially without creating a String
first?
It seems not that easy, we can't simply do sec_pass.pwd = "password".
Nor
can we reuse any IO - the buffer is again a String... So it seems to
trap a
password in its SecurePassword jail, we'd have to invent something
really
awkward on top of that class :( Worse, I imagine nobody would likely
want
to comb through their entire projects trying to find places where they
have
to replace String usage with SecurePassword.

That's why I had hopes that the housekeeping could probably be done in
String.
It's a mess really, hopefully some of you have better ideas how this
could be
solved in a more elegant way?

----------------------------------------
Feature #5741: Secure Erasure of Passwords
https://bugs.ruby-lang.org/issues/5741#change-25336

Author: MartinBosslet (Martin Bosslet)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
Ada56d152c1abe0eba65a5ff62ed5387?d=identicon&s=25 Martin Bosslet (martin_b)
on 2012-04-28 02:56
(Received via mailing list)
Issue #5741 has been updated by MartinBosslet (Martin Bosslet).


Just to add this, I think it wasn't mentioned yet: we
also have to be aware of copying during GC, this could
compromise a password in memory as well.
----------------------------------------
Feature #5741: Secure Erasure of Passwords
https://bugs.ruby-lang.org/issues/5741#change-26275

Author: MartinBosslet (Martin Bosslet)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
A71948d2382c2809330bc01184f3bfd4?d=identicon&s=25 kenkeiter (Ken Keiter) (Guest)
on 2012-07-02 07:55
(Received via mailing list)
Issue #5741 has been updated by kenkeiter (Ken Keiter).


Is there any update on the status of this issue? This is simply a
layperson's perspective (I'm not an expert regarding Ruby's internals),
but I'd imagine this to be as easy as querying GC for all copies of the
object, and overwriting the memory locations upon request?

I can dig into the internals if I need to; I'd just love to get this on
the roadmap!
----------------------------------------
Feature #5741: Secure Erasure of Passwords
https://bugs.ruby-lang.org/issues/5741#change-27705

Author: MartinBosslet (Martin Bosslet)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
Ada56d152c1abe0eba65a5ff62ed5387?d=identicon&s=25 Martin Bosslet (martin_b)
on 2012-07-05 14:00
(Received via mailing list)
Issue #5741 has been updated by MartinBosslet (Martin Bosslet).


Hi Ken,

I guess it largely depends on how things will go with #6361.
But if you are interested in the feature, we are beginning to work on
something that covers both the aspects of this issue and that of #6361.
You may want to keep an eye on https://github.com/emboss, it's going
to be called something like 'binary-io'.
----------------------------------------
Feature #5741: Secure Erasure of Passwords
https://bugs.ruby-lang.org/issues/5741#change-27827

Author: MartinBosslet (Martin Bosslet)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
C4e88907313843cf07f6d85ba8162120?d=identicon&s=25 ko1 (Koichi Sasada) (Guest)
on 2012-10-26 23:30
(Received via mailing list)
Issue #5741 has been updated by ko1 (Koichi Sasada).


ping. status?

I think matz doesn't know this ticket.
Could someone grab this ticket?

----------------------------------------
Feature #5741: Secure Erasure of Passwords
https://bugs.ruby-lang.org/issues/5741#change-31672

Author: MartinBosslet (Martin Bosslet)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
Ada56d152c1abe0eba65a5ff62ed5387?d=identicon&s=25 Martin Bosslet (martin_b)
on 2012-11-21 03:44
(Received via mailing list)
Issue #5741 has been updated by MartinBosslet (Martin Bosslet).


FWIW, this feature is going to be an integral part of "binyo"
(https://github.com/krypt/binyo).
----------------------------------------
Feature #5741: Secure Erasure of Passwords
https://bugs.ruby-lang.org/issues/5741#change-33373

Author: MartinBosslet (Martin Bosslet)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
F24ff61beb80aa5f13371aa22a35619c?d=identicon&s=25 mame (Yusuke Endoh) (Guest)
on 2012-11-24 03:42
(Received via mailing list)
Issue #5741 has been updated by mame (Yusuke Endoh).

Target version changed from 2.0.0 to next minor


----------------------------------------
Feature #5741: Secure Erasure of Passwords
https://bugs.ruby-lang.org/issues/5741#change-33748

Author: MartinBosslet (Martin Bosslet)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: next minor


In other languages it is considered good practice to securely erase
passwords immediately after they were used. Imagine authentication
in a web app - ultimately a String containing the password arrives
at the server, where it will be processed and compared to some
previously stored value. After this is done, there is no need to
store these password Strings any longer, so they should be
discarded right away (more on why later).

In C, you would simply overwrite the array of bytes with zeroes or
random values. In Java, Strings are immutable, that's why there it
is common practice to use char[] for all things password and overwrite
them when done.

Currently, there is no way in Ruby to overwrite the memory that
was used by a String. String#clear and String#replace both use
str_discard internally, which only frees the underlying pointer
without overwriting it.

The problem with not erasing passwords is this: the contents of the
String stay in memory until they are finally GC'ed. But even then
only the pointer will be freed, leaving the contents mostly intact
until the memory is reclaimed and overwritten later on.

This could be exploited if an attacker had access to the memory of
the server. This could happen in many ways: a core dump after a
crash, access to the host if the server runs in a VM, or even by
deep-freezing the DRAM :) [1]

It could be argued that given the examples above, much more
devastating attacks would be possible since in all of those
cases you more or less have physical access to the machine. But
I would still consider this to be a valid concern, if not only
for the reason of never opening additional attack surfaces if
they can be avoided relatively easily.

I also found [2], which seems to show that Python deals with
similar problems and it also contains more background info.

Eric Hodel and I discussed this yesterday and Eric came up with
a C extension that can be used to illustrate the problem (attached).

If you inspect the resulting core dump, you will find the following:

- the untouched String remains in memory fully intact
- the String#clear'ed String remains to a large extent, typically the
  first character is missing - so if you typed "PASSWORD", search for
  "ASSWORD" (unintentional pun) instead
- The String#clear_secure'ed will have been completely erased, no
  traces remain

My questions:

1. Would you agree that we need this functionality?
2. Where would we ideally place it? I'm not sure whether
   String is the perfect place, but on the other hand, String
   is the only place where we have access to the implementation
   details.
3. Are there better alternative ways how we could achieve this?

[1] http://www.schneier.com/blog/archives/2008/02/cold...
[2]
http://stackoverflow.com/questions/728164/securely...
This topic is locked and can not be replied to.