Forum: Ruby-core High memory usage from file_expand_path_*

5cf8f058a4c094bb708174fb43e7a387?d=identicon&s=25 unknown (Guest)
on 2014-08-18 16:37
(Received via mailing list)
Issue #9934 has been updated by Tomoyuki Chikanaga.

Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED to 2.0.0: REQUIRED,
2.1: DONE

r46408, r46410, r46413, r46414, r46424, r46436, r46437 and partially
r44827 (to resolve conflict of r46408) were backported into `ruby_2_1`
branch at r47215.

Bug #9934: High memory usage from file_expand_path_*

* Author: Aman Gupta
* Status: Closed
* Priority: Normal
* Assignee:
* Category: core
* Target version: current: 2.2.0
* ruby -v: current
* Backport: 2.0.0: REQUIRED, 2.1: DONE
All the file expansion routines use `EXPAND_PATH_BUFFER()` which
allocates PATH_MAX bytes on the heap per invocation.
The strings returned by `File.expand_path` are never realloc'd after
they are populated, so they continue using 4kb (on linux) per string.
In our rails app, 22MB of heap usage is due to expanded path name

$ ruby -robjspace -e' puts ObjectSpace.dump(File.expand_path("/foo")) '
{"address":"0x007fa2b44dd6c8", "type":"STRING",
"class":"0x007fa2b3f99608", "bytesize":4, "capacity":4098,
"value":"/foo", "encoding":"US-ASCII", "memsize":4099,

The following failing patch demonstrates the issue as well:

diff --git a/test/ruby/test_file_exhaustive.rb
index 2c945ea..49be9de 100644
--- a/test/ruby/test_file_exhaustive.rb
+++ b/test/ruby/test_file_exhaustive.rb
@@ -458,6 +458,12 @@ class TestFileExhaustive < Test::Unit::TestCase

+  def test_expand_path_memsize
+    require "objspace"
+    path = File.expand_path("/foo")
+    assert_equal 5, ObjectSpace.memsize_of(path)
+  end
   def test_expand_path_encoding
     drive = (DRIVE ? 'C:' : '')
     if Encoding.find("filesystem") == Encoding::CP1251
This topic is locked and can not be replied to.