# Bug: Ruby 1.9.1 File io on Windows is slooooow

**URL:** <https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071>\
**Category:** Ruby\
**Created:** [March 27, 2009, 2:38pm UTC](https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071 "2009-03-27T14:38:21Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![ther](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/977dab/32.png) [@ther](https://www.ruby-forum.com/u/ther)\
**Post date:** [March 27, 2009, 2:38pm UTC](https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071/1 "2009-03-27T14:38:21Z")

</div>

This simple code shows everything:

time = [Time.new]  
c = ‘’  
‘aaaa’.upto(‘zzzz’) {|e| c \<\< e}  
3.times { c \<\< c }  
time \<\< Time.new  
File.open(‘out.file’,‘w’) { |f| f.write© }  
time \<\< Time.new  
c = File.open(‘out.file’,‘r’) { |f| f.read }  
time \<\< Time.new  
0.upto(time.size - 2) {|i| p “#{i} #{time[i+1]-time[i]}” }

ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-mswin32]  
“0 0.537075”  
“1 0.696244”  
“2 40.188834”

ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-mswin32]  
“0 0.551”  
“1 0.133”  
“2 0.087”

That is about 5x slower write and 500x read operation. Times are the  
same if I do:  
f = File.new(‘out.file’,‘r’)  
c = f.read  
f.close

Tried on two machines. Vista SP1 and XP SP3. Same results.

by  
TheR

P.S.  
Where can I officaly report a Ruby bug?

---

<div class="post-metadata">

**Author:** ![ther](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/977dab/32.png) [@ther](https://www.ruby-forum.com/u/ther)\
**Post date:** [March 27, 2009, 3:20pm UTC](https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071/2 "2009-03-27T15:20:41Z")

</div>

Damjan R. wrote:

> time \<\< Time.new  
> “2 0.087”  
> I’m pretty sure from your other message that it’s using Ruby 1.9 with  
> UTF-8 encoded strings that kills performance. I get equivalent results  
> on OS X comparing 1.8.6 to 1.9.1 on common string operations like  
> String#.

It’s understandable that correct character-wise operations are much  
harder for Ruby to implement efficiently on variable-byte-length  
encodings like UTF-8. Unfortunately, though I was delighted to get  
proper encoding support in 1.9, the performance hit on strings was a  
killer on my string-intensive GUI app.

There are sometimes ways round this (eg I rewrote diff/lcs to use string  
character enumerators rather than repeated expensive calls to  
String#[]).

I expect the Ruby team are already aware of the problem, and I hope this  
is a top priority for 1.9.2.

btw, you can report bugs at:

> **[Ruby Issue Tracking System](https://redmine.ruby-lang.org/)**
>
> Redmine

alex

---

<div class="post-metadata">

**Author:** ![ther](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/977dab/32.png) [@ther](https://www.ruby-forum.com/u/ther)\
**Post date:** [March 27, 2009, 4:26pm UTC](https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071/3 "2009-03-27T16:26:38Z")

</div>

Alex F. wrote:

> There are sometimes ways round this (eg I rewrote diff/lcs to use string  
> character enumerators rather than repeated expensive calls to  
> String#[]).
> 
> I expect the Ruby team are already aware of the problem, and I hope this  
> is a top priority for 1.9.2.

Unfortunately it appears to be a designed-in pessimization, though I  
initially thought it was a mistake when I saw it. Guido van Rossum (the  
Python creator) probably did too, as he asked Matz about it at the  
40-minute mark here

[![](https://img.youtube.com/vi/oEkJvvGEtB4/hqdefault.jpg "Ruby 1.9") ](https://www.youtube.com/watch?v=oEkJvvGEtB4)

---

<div class="post-metadata">

**Author:** ![ther](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/977dab/32.png) [@ther](https://www.ruby-forum.com/u/ther)\
**Post date:** [March 29, 2009, 9:20am UTC](https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071/4 "2009-03-29T09:20:08Z")

</div>

Nobuyoshi N. wrote:

> Hi,
> 
> At Fri, 27 Mar 2009 23:16:38 +0900,  
> Alex F. wrote in [ruby-talk:332264]:
> 
> > I’m pretty sure from your other message that it’s using Ruby 1.9 with  
> > UTF-8 encoded strings that kills performance. I get equivalent results  
> > on OS X comparing 1.8.6 to 1.9.1 on common string operations like  
> > String#.
> 
> Slow-read is caused by universal-newline.

Should’nt fread be reading binary file or should that be forced by ‘rb’  
parameter at file.open()

I did fast test on Ubuntu x64 Linux. And the problem does not exist.  
Althow it was Ruby version 1.9.0.

by  
TheR

---

<div class="post-metadata">

**Author:** ![ther](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/977dab/32.png) [@ther](https://www.ruby-forum.com/u/ther)\
**Post date:** [March 28, 2009, 3:27am UTC](https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071/5 "2009-03-28T03:27:00Z")

</div>

Hi,

At Fri, 27 Mar 2009 23:16:38 +0900,  
Alex F. wrote in [ruby-talk:332264]:

> I’m pretty sure from your other message that it’s using Ruby 1.9 with  
> UTF-8 encoded strings that kills performance. I get equivalent results  
> on OS X comparing 1.8.6 to 1.9.1 on common string operations like  
> String#.

Slow-read is caused by universal-newline.

---

<div class="post-metadata">

**Author:** ![ther](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/977dab/32.png) [@ther](https://www.ruby-forum.com/u/ther)\
**Post date:** [March 29, 2009, 1:06pm UTC](https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071/6 "2009-03-29T13:06:17Z")

</div>

2009/3/27 Damjan R. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

> c = File.open(‘out.file’,‘r’) { |f| f.read }  
> “1 0.133”  
> “2 0.087”
> 
> That is about 5x slower write and 500x read operation.

Do you have a virus scanner on that Windows box? If so, how are  
measurements when switched off?

Cheers

robert

---

<div class="post-metadata">

**Author:** ![ther](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/977dab/32.png) [@ther](https://www.ruby-forum.com/u/ther)\
**Post date:** [December 2, 2009, 5:37pm UTC](https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071/7 "2009-12-02T17:37:25Z")

</div>

> ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-mswin32]  
> “2 40.188834”
> 
> ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-mswin32]  
> “2 0.087”
> 
> That is about 5x slower write and 500x read operation. Times are the  
> same if I do:  
> f = File.new(‘out.file’,‘r’)  
> c = f.read  
> f.close  
> P.S.  
> Where can I officaly report a Ruby bug?

Looks like this bug is [http://redmine.ruby-lang.org/issues/show/1332](http://redmine.ruby-lang.org/issues/show/1332)  
Can you try with mode ‘rb’ and is it fast then?  
-r

---

<div class="post-metadata">

**Author:** ![ther](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/977dab/32.png) [@ther](https://www.ruby-forum.com/u/ther)\
**Post date:** [March 30, 2009, 7:43am UTC](https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071/8 "2009-03-30T07:43:07Z")

</div>

Robert K. wrote:

> 2009/3/27 Damjan R. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):
> 
> > c = File.open(‘out.file’,‘r’) { |f| f.read }  
> > “1 0.133”  
> > “2 0.087”
> > 
> > That is about 5x slower write and 500x read operation.
> 
> Do you have a virus scanner on that Windows box? If so, how are  
> measurements when switched off?
> 
> Cheers
> 
> robert

I swithed it off and times are pretty much the same.

by  
TheR

---

<div class="post-metadata">

**Author:** ![ther](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/977dab/32.png) [@ther](https://www.ruby-forum.com/u/ther)\
**Post date:** [February 5, 2010, 10:52pm UTC](https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071/9 "2010-02-05T22:52:55Z")

</div>

Roger P. wrote:

> > ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-mswin32]  
> > “2 40.188834”
> > 
> > ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-mswin32]  
> > “2 0.087”
> > 
> > That is about 5x slower write and 500x read operation. Times are the  
> > same if I do:  
> > f = File.new(‘out.file’,‘r’)  
> > c = f.read  
> > f.close  
> > P.S.  
> > Where can I officaly report a Ruby bug?
> 
> Looks like this bug is [http://redmine.ruby-lang.org/issues/show/1332](http://redmine.ruby-lang.org/issues/show/1332)  
> Can you try with mode ‘rb’ and is it fast then?  
> -r

Also this one is much much slower on ruby 1.9 then on ruby 1.8:  
Tested on Windows XP. No virus scanner.

time = [Time.new]  
n = 1000  
lines = []  
(1…n).each { |nr| lines \<\< “line” }

time \<\< Time.new  
File.open(‘out2.file’,‘w’) { |f|  
lines.each { |l|  
f.print(l)  
f.flush  
}  
}  
time \<\< Time.new

txt = [‘construct’,‘write’,‘read’]  
0.upto(time.size - 2) {|i| p “#{txt[i]} #{time[i+1]-time[i]}” }

#Ruby 1.9  
#“construct 0.0”  
#“write 22.453125”

#Ruby 1.8  
#“construct 0.0”  
#“write 0.015”

---

<div class="post-metadata">

**Author:** ![ther](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/977dab/32.png) [@ther](https://www.ruby-forum.com/u/ther)\
**Post date:** [February 6, 2010, 10:17pm UTC](https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071/10 "2010-02-06T22:17:46Z")

</div>

> ```
> f.print(l)
> f.flush
> 
> ```

If you remove the flush it seems to work _much_ faster.  
This might be a bug in core. I’d probably report it.

I think to avoid the slowdown with reads you can “hack a work around”  
like

c = File.open(‘out.file’,‘rb’) { |f| f.read }  
c.gsub!(“\r\n”, “\n”)

But there might be a bug in there, too. [1]

-rp  
[1] [http://redmine.ruby-lang.org/issues/show/1332](http://redmine.ruby-lang.org/issues/show/1332)

---

<div class="post-metadata">

**Author:** ![ther](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/977dab/32.png) [@ther](https://www.ruby-forum.com/u/ther)\
**Post date:** [February 6, 2010, 2:45pm UTC](https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071/11 "2010-02-06T14:45:44Z")

</div>

On 02/05/2010 10:53 PM, Ad Gr wrote:

> > > c = f.read  
> > > time = [Time.new]  
> > > }  
> > > #ruby 1.8  
> > > #“construct 0.0”  
> > > #“write 0.015”

How many test runs did you do? Because of caching figures from a single  
run are usually not very telling.

Kind regards

robert

---

<div class="post-metadata">

**Author:** ![ther](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/977dab/32.png) [@ther](https://www.ruby-forum.com/u/ther)\
**Post date:** [February 15, 2010, 3:28pm UTC](https://www.ruby-forum.com/t/bug-ruby-1-9-1-file-io-on-windows-is-slooooow/164071/12 "2010-02-15T15:28:46Z")

</div>

Damjan R. wrote:

> This simple code shows everything:  
> …  
> That is about 5x slower write and 500x read operation.

You can try with some patches at

[http://www.ruby-forum.com/topic/204113](http://www.ruby-forum.com/topic/204113)

then it’s only 36x as slow for reads 🙂

-r