Forum: Ruby-core [Closed] High memory usage from file_expand_path_*

F1d6cc2b735bfd82c8773172da2aeab9?d=identicon&s=25 Nobuyoshi Nakada (nobu)
on 2014-06-12 06:11
(Received via mailing list)
Issue #9934 has been updated by Nobuyoshi Nakada.

Status changed from Open to Closed
% Done changed from 0 to 100

Applied in changeset r46410.

----------
file.c: shrink expanded path

* file.c (expand_path): shrink expanded path which no longer needs
  rooms to append.  [ruby-core:63114] [Bug #9934]

----------------------------------------
Bug #9934: High memory usage from file_expand_path_*
https://bugs.ruby-lang.org/issues/9934#change-47182

* Author: Aman Gupta
* Status: Closed
* Priority: Normal
* Assignee:
* Category:
* Target version:
* ruby -v: trunk
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN
----------------------------------------
All the file expansion routines use `EXPAND_PATH_BUFFER()` which
allocates PATH_MAX bytes on the heap per invocation.
The strings returned by `File.expand_path` are never realloc'd after
they are populated, so they continue using 4kb (on linux) per string.
In our rails app, 22MB of heap usage is due to expanded path name
strings.

~~~
$ ruby -robjspace -e' puts ObjectSpace.dump(File.expand_path("/foo")) '
{"address":"0x007fa2b44dd6c8", "type":"STRING",
"class":"0x007fa2b3f99608", "bytesize":4, "capacity":4098,
"value":"/foo", "encoding":"US-ASCII", "memsize":4099,
"flags":{"wb_protected":true}}
~~~

The following failing patch demonstrates the issue as well:

~~~
diff --git a/test/ruby/test_file_exhaustive.rb
b/test/ruby/test_file_exhaustive.rb
index 2c945ea..49be9de 100644
--- a/test/ruby/test_file_exhaustive.rb
+++ b/test/ruby/test_file_exhaustive.rb
@@ -458,6 +458,12 @@ class TestFileExhaustive < Test::Unit::TestCase
     end
   end

+  def test_expand_path_memsize
+    require "objspace"
+    path = File.expand_path("/foo")
+    assert_equal 5, ObjectSpace.memsize_of(path)
+  end
+
   def test_expand_path_encoding
     drive = (DRIVE ? 'C:' : '')
     if Encoding.find("filesystem") == Encoding::CP1251
~~~
This topic is locked and can not be replied to.