A gem for handling temporary file(s)?

Ruby
1

I’m writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)

Is there a ‘gem’ that manages these things? Preferably it should remove
the files when the script finishes or whatever.

Of course, if Ruby supports this built-in that’s fine too (I did ‘ri
File’ and ‘ri FileUtils’ but founds nothing).

2

Albert S. wrote:

I’m writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)

Is there a ‘gem’ that manages these things? Preferably it should remove
the files when the script finishes or whatever.

Of course, if Ruby supports this built-in that’s fine too (I did ‘ri
File’ and ‘ri FileUtils’ but founds nothing).

ri Tempfile

that’ll get you started

3

Paul H. wrote:

Albert S. wrote:

I’m writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)
[…]
ri Tempfile

that’ll get you started

Thanks! I didn’t know about Tempfile.

Though I have a little problem: Tempfile let me open a new temporary
file. But I just need to generate a temporary file name, which I’ll
pass to a shell command.

I guess I’ll immediately close the handle Tempfile.new() returns and
pass its path() to the shell command.

4

On 03/03/2010 05:07 AM, Albert S. wrote:

that’ll get you started

Thanks! I didn’t know about Tempfile.

Though I have a little problem: Tempfile let me open a new temporary
file. But I just need to generate a temporary file name, which I’ll
pass to a shell command.

You could use a dirty hack and abuse a private method:

irb(main):005:0> Tempfile.open(’/tmp’) {|tf| p tf,
tf.send(:make_tmpname,‘a’,‘o’)}
#<File:/tmp/tmp20100303-4173-rdy9aq-0>
“a20100303-4173-kvk0d3-o”
=> [#<File:/tmp/tmp20100303-4173-rdy9aq-0 (closed)>,
“a20100303-4173-kvk0d3-o”]
irb(main):006:0>

I guess I’ll immediately close the handle Tempfile.new() returns and
pass its path() to the shell command.

What do you want the external program to do with the tempfile?

Kind regards

robert

5

Robert K. wrote:

You could use a dirty hack and abuse a private method:
tf.send(:make_tmpname,‘a’,‘o’)}

Thanks.

Anyway, it turned out that was my smallest problems. I ended up writing
a wrapper class that remembers a set of related Tempfile object (or else
the files get deleted too soon for me).

6

On Wed, Mar 3, 2010 at 10:19 AM, Albert S. [email protected]
wrote:

I’m writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)

man mktemp
man tempfile

Is there a ‘gem’ that manages these things? Preferably it should remove
the files when the script finishes or whatever.

it is builtin in ruby. but in this case, you’d better do it all in
ruby…

best regards -botp

7

On Mar 3, 12:28 am, botp [email protected] wrote:

Is there a ‘gem’ that manages these things? Preferably it should remove
the files when the script finishes or whatever.

it is builtin in ruby. but in this case, you’d better do it all in ruby…

Bah. Use file-temp.

gem install file-temp

Regards,

Dan

8

On 3/2/10, Albert S. [email protected] wrote:

Paul H. wrote:

ri Tempfile

that’ll get you started

Thanks! I didn’t know about Tempfile.

Though I have a little problem: Tempfile let me open a new temporary
file. But I just need to generate a temporary file name, which I’ll
pass to a shell command.

I’m not expert enough to be certain about this, but by doing this
you’ll be creating a tempfile race condition security hole in your
program. I think the same goes for Robert’s suggestion as well. There
may be a way to do it securely… but it’s probably tricky. One
advantage of Tempfile (and similar facilities in other languages) is
that it avoids this subtle and nassty security hole. But you have to
use it the way it wants to be used, otherwise you defeat the security.
This is why you’re better off rewriting this external command in ruby,
if that’s possible. Or rewriting your ruby script to make it an
integral part of the external program.

None of this may actually matter in your case… but you’re the only
one with enough information to make that judgment.

9

On 3/3/10, Daniel B. [email protected] wrote:

Bah. Use file-temp.

gem install file-temp

I can’t get file-temp to install at the moment, (for reasons unrelated
to file-temp, apparently). Does file-temp allow one to create
temporary directories? That’s a feature I’ve often missed in the
past.

10

On 03/03/2010 08:35 PM, Caleb C. wrote:

I’m not expert enough to be certain about this, but by doing this
you’ll be creating a tempfile race condition security hole in your
program. I think the same goes for Robert’s suggestion as well.

Do you mean there is a robustness issue or a security issue? I don’t
see a security issue here. Robustness would only be at risk if the file
name generation algorithm is bad. What else am I missing?

one with enough information to make that judgment.
Albert still did not disclose what the external program should do with
the temporary file. We do not even know whether it is an option to
rewrite the external program.

Kind regards

robert

11

On 3/3/10, Robert K. [email protected] wrote:

On 03/03/2010 08:35 PM, Caleb C. wrote:

I’m not expert enough to be certain about this, but by doing this
you’ll be creating a tempfile race condition security hole in your
program. I think the same goes for Robert’s suggestion as well.

Do you mean there is a robustness issue or a security issue? I don’t
see a security issue here. Robustness would only be at risk if the file
name generation algorithm is bad. What else am I missing?

It’s a security issue. See:
http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/avoid-race.html
and scroll down to “7.10.1.2. Temporary Files”

As I implied above, I may not know what I’m talking about here, but
I’m fairly sure. Also, I didn’t contemplate the snippet you
contributed closely; perhaps it avoids the race condition in some
clever way that I’m unaware of.

one with enough information to make that judgment.

Albert still did not disclose what the external program should do with
the temporary file. We do not even know whether it is an option to
rewrite the external program.

Yes, indeed. Which is why I said “if that’s possible”.

12

On 3/3/10, Caleb C. [email protected] wrote:

There
may be a way to do it securely… but it’s probably tricky.

I think that if you create a temporary file with Tempfile, and then
DON’T CLOSE IT, you can safely pass the tempfile’s name to an external
command. (I’m assuming that the tempfile is an output from the
external command… if it’s an input, none of my security anxieties
apply. I think.) Only close the tempfile once the external command has
finished (and you’ve read out of it whatever information you need).
This will fail if the external command balks at writing to an already
existing file.

Alternatively, you could put your temp file in $HOME/tmp rather than
the system-wide /tmp, which is another way to sidestep the race. I’m
pretty sure. If you go this way, Tempfile is useless to you, tho.

13

2010/3/4 Caleb C. [email protected]:

Alternatively, you could put your temp file in $HOME/tmp rather than
the system-wide /tmp, which is another way to sidestep the race. I’m
pretty sure. If you go this way, Tempfile is useless to you, tho.

Another way is Dir.mktmpdir in tmpdir.

require ‘tmpdir’
Dir.mktmpdir {|d|
tmppath = File.join(d, “foo”)
… use tmppath …
}

14

Robert K. wrote:

On 03/03/2010 08:35 PM, Caleb C. wrote:

one with enough information to make that judgment.

Albert still did not disclose what the external program should do with
the temporary file […]

Thank you all, this task is behind me already.

If it interests you, then the external command I’m using is ‘dot’ (from
the graphviz package).

The command I’m using is something like this:

dot “%s” -Tgif -o “%s” -Tsvg -o “%s”

(I need three temporary files, in this example. ‘dot’ reads the first
file and writes images to the next two files.)

Smart guys would say that instead, on POSIX systems, I could write the
command as…

dot /dev/fd/10 -Tgif -o /dev/fd/11" -Tsvg -o /dev/fd/12

…but then I’d need to read simultaneously from fd/11 and fd/12 because
I’m not sure which is written first. But I need my program to work on
Windows too and I’m not sure it supports this “sophistication”.

Caleb C. wrote:

I’m not expert enough to be certain about this, but by doing this
you’ll be creating a tempfile race condition […]

I don’t care much about robustness because the command is run from an
interactive application by a single user and I’m using the files
immediately. There isn’t much a chance for a race condition.

Caleb C. wrote:

I think that if you create a temporary file with Tempfile, and then
DON’T CLOSE IT, you can safely pass the tempfile’s name to an external
command.

For some reason, the files aren’t always deleted (well, it seems they’re
never deleted). I don’t know why. At first I suspected ‘dot’ re-creates
the files, but when I do “ls -i” I see that the inodes haven’t changed.
But I can live with this bug. Maybe I could add some code to the
“destructor” of my class to explicitly delete these files. Does Ruby
support “destructors”?

15

2010/3/4 Albert S. [email protected]:

Robert K. wrote:

On 03/03/2010 08:35 PM, Caleb C. wrote:

one with enough information to make that judgment.

Albert still did not disclose what the external program should do with
the temporary file […]

Thank you all, this task is behind me already.

Considering your last comment about files not being deleted I’d say
it’s not done yet. :slight_smile:

Smart guys would say that instead, on POSIX systems, I could write the
command as…

dot /dev/fd/10 -Tgif -o /dev/fd/11" -Tsvg -o /dev/fd/12

…but then I’d need to read simultaneously from fd/11 and fd/12 because
I’m not sure which is written first. But I need my program to work on
Windows too and I’m not sure it supports this “sophistication”.

Yeah, probably not a good idea to go into that direction.

Btw, do you need to read those files while they are written or is it
sufficient to wait for process termination (e.e. use system) before
you access them?

DON’T CLOSE IT, you can safely pass the tempfile’s name to an external
command.

For some reason, the files aren’t always deleted (well, it seems they’re
never deleted). I don’t know why. At first I suspected ‘dot’ re-creates
the files, but when I do “ls -i” I see that the inodes haven’t changed.
But I can live with this bug. Maybe I could add some code to the
“destructor” of my class to explicitly delete these files. Does Ruby
support “destructors”?

We have

begin
…
ensure
…
end

The usual way to do this is to write a method. See
http://blog.rubybestpractices.com/posts/rklemme/002_Writing_Block_Methods.html

Cheers

robert

16

On 3/4/10, Albert S. [email protected] wrote:

Caleb C. wrote:

I’m not expert enough to be certain about this, but by doing this
you’ll be creating a tempfile race condition […]

I don’t care much about robustness because the command is run from an
interactive application by a single user and I’m using the files
immediately. There isn’t much a chance for a race condition.

It’s not (just) a robustness issue, but a security issue. Even if
there’s not much chance for a race condition, that can create a very
large security hole. I’d suggest you should take a look at the link I
posted in reply to Robert above. I just want to make sure you’re
aware.

17

On Mar 3, 2:13 pm, Caleb C. [email protected] wrote:

On 3/3/10, Daniel B. [email protected] wrote:

Bah. Use file-temp.

gem install file-temp

I can’t get file-temp to install at the moment, (for reasons unrelated
to file-temp, apparently). Does file-temp allow one to create
temporary directories? That’s a feature I’ve often missed in the
past.

No, but there’s Dir.mktmpdir.

Regards,

Dan

18

On 3/4/10, Daniel B. [email protected] wrote:

On Mar 3, 2:13 pm, Caleb C. [email protected] wrote:

to file-temp, apparently). Does file-temp allow one to create
temporary directories? That’s a feature I’ve often missed in the
past.

No, but there’s Dir.mktmpdir.

Thanks to you and to Tanaka A. for pointing this out, since I was
not aware of it. I’ll definitely remember it next time I need that.

19

Robert K. wrote:

2010/3/4 Albert S. [email protected]:

dot … -Tgif -o … -Tsvg -o …
[…]
Btw, do you need to read those files while they are written or is it
sufficient to wait for process termination (e.e. use system) before
you access them?

(I read the files after termination (I’m using system()).)

Maybe I could add some code to the
“destructor” of my class to explicitly delete these files. Does Ruby
support “destructors”?

We have

begin
…
ensure
…
end

The usual way to do this is to write a method. See
http://blog.rubybestpractices.com/posts/rklemme/002_Writing_Block_Methods.html

Thanks. I know this scheme (and the article), but for some reason it
didn’t come up to my mind to use it. You just gave me a nice idea. I
need to read more Ruby code to train myself. The problem is that I love
to program so I prefer writing to reading :frowning:

Caleb C. wrote:

It’s not (just) a robustness issue, but a security issue. Even if
there’s not much chance for a race condition, that can create a very
large security hole. I’d suggest you should take a look at the link I
posted in reply to Robert above. I just want to make sure you’re
aware.

You mean a privacy issue. Yes, I fixed that. Thanks.

20

2010/3/3 Caleb C. [email protected]:

It’s a security issue. See:
Avoid Race Conditions
and scroll down to “7.10.1.2. Temporary Files”

As I implied above, I may not know what I’m talking about here, but
I’m fairly sure. Also, I didn’t contemplate the snippet you
contributed closely; perhaps it avoids the race condition in some
clever way that I’m unaware of.

Thanks for the link. If I understand all this right this cannot be
fixed as long as a) there are two processes involved or b) the second
process (dot) cannot be made inherit the file descriptor. An
alternative approach would avoid a shared directory like /tmp and
write the output in $HOME/…/somewhere - which might be slower
because /tmp is often mounted in RAM. Maybe it helps to create a
directory in /tmp which is owned by and only accessible to $USER; then
create the tempfiles in that directory.

one with enough information to make that judgment.

Albert still did not disclose what the external program should do with
the temporary file. We do not even know whether it is an option to
rewrite the external program.

Yes, indeed. Which is why I said “if that’s possible”.

Yep. I mainly included the remark to poke Albert so he would
eventually give the information. :wink:

Kind regards

robert