System calls with ` in parameters

Ruby
1

Hi All,

I am trying to add functionality to an interesting script. The part
that is tripping me up is that I have to make a system call to echo, and
feed several parameters. One of the parameters occasionally includes
the ` character.

Because of this, the syntax of:

echo "#{someVar}" | ./some-script.py > "#{outputFile}"

is breaking when #{someVar} contains the tick . This causes the command to fail because it executes up to the tick in someVar and not
to the end as intended.

I have also tried using the system() command as follows:

system(“echo “#{someVar}” | ./some-script.py > “#{outputFile}””)

But this fails when there is a tick ` in #{someVar} as well.

Any help would be appreciated.

Thanks,
Gerad

2

On Feb 9, 2011, at 3:48 PM, Gerad S. wrote:

Any help would be appreciated.
Shellwords should do it:

ruby -rshellwords -e ‘p Shellwords.escape “`”’

3

On Thu, Feb 10, 2011 at 12:48 AM, Gerad S. [email protected]
wrote:

is breaking when #{someVar} contains the tick . This causes the command to fail because it executes up to the tick in someVar and not
to the end as intended.

I have also tried using the system() command as follows:

system(“echo "#{someVar}" | ./some-script.py > "#{outputFile}"”)

Why do you escape the string interpolation? Don’t you want those file
names inserted there? It seems you rather want

system(“echo ‘#{someVar}’ | ./some-script.py >‘#{outputFile}’”)

But this fails when there is a tick ` in #{someVar} as well.

Here’s a pure Ruby solution:

File.open outputFile, “w” do |out|
IO.popen “./some-script.py”, “rw” do |io|
t = Thread.new do
IO.copy_stream(io, out, 1024)
# pre 1.9:
# while ( b = io.read(1024) )
# out.write(b)
# end
end

io.puts(someVar)
io.close_write

t.join

end
end

Of course, it would be even better if you rewrote the Python script in
Ruby. :wink:

Cheers

robert

4

On Wed, Feb 9, 2011 at 6:48 PM, Gerad S. [email protected] wrote:

system(“echo "#{someVar}" | ./some-script.py > "#{outputFile}"”)

But this fails when there is a tick ` in #{someVar} as well.

Could you show us a working example of the problem?

s = “foo date baz”
=> “foo date baz”

system “echo #{s}”
foo Thu Feb 10 07:18:23 EST 2011 baz
=> true

system “echo "#{s}"”
foo Thu Feb 10 07:18:32 EST 2011 baz
=> true

system “echo ‘#{s}’”
foo date baz
=> true

require ‘shellwords’
=> true
echo #{Shellwords.escape(s)}
=> “foo date baz\n”

5

2011/2/10 Gerad S. [email protected]:

echo "#{someVar}" | ./some-script.py > "#{outputFile}"

If you use Ruby 1.9.2, Open3.pipline() provides a way to run a pipline
without shell.

% ruby -ropen3 -e ‘Open3.pipeline([“echo”, “foobarbaz”], [“head”,
“-6c”, :out => “/tmp/qux”])’
% od -c /tmp/qux
0000000 f o o ` b a
0000006

6

On Thu, Feb 10, 2011 at 11:21 AM, Tanaka A. [email protected] wrote:

0000000 f o o ` b a
0000006

Cool! Thanks for sharing.

Is there also a version which would return an IO or accept a block
with IO parameter so we can feed something into the head of the
pipeline and read from the tail?

Kind regards

robert

7

Gerad S. wrote in post #980690:

Hi All,

I am trying to add functionality to an interesting script. The part
that is tripping me up is that I have to make a system call to echo, and
feed several parameters. One of the parameters occasionally includes
the ` character.

Because of this, the syntax of:

echo "#{someVar}" | ./some-script.py > "#{outputFile}"

is breaking when #{someVar} contains the tick `.

Does this test case represent your problem?

someVar=“foobar" => "foobar”

echo "#{someVar}"
sh: Syntax error: EOF in backquote substitution
=> “”

Quick workaround: use single quotes instead (but see below).

echo '#{someVar}'
=> “foo`bar\n”

The same occurs with system():

system(“echo “#{someVar}””)
sh: Syntax error: EOF in backquote substitution
=> false

system(“echo ‘#{someVar}’”)
foo`bar
=> true

It’s an issue with the shell, not with ruby; the same happens at the
shell prompt too.

$ echo “foo`bar”

bash: unexpected EOF while looking for matching ``’
bash: syntax error: unexpected end of file

The issue is that in the shell, double-quoted strings can contain
backtick expansions, for example

$ echo “foohostnamebar”
foox100bar

Backticks inside single quotes are not treated specially. However,
single quotes inside single quotes are still a problem:

someVar=“foo’bar”
=> “foo’bar”

echo '#{someVar}'
sh: Syntax error: Unterminated quoted string
=> “”

So using the Shellwords escaping mechanism, as already shown, is
probably what you want.

8

Robert K. wrote in post #980746:

I have also tried using the system() command as follows:

system(“echo “#{someVar}” | ./some-script.py > “#{outputFile}””)

Why do you escape the string interpolation? Don’t you want those file
names inserted there? It seems you rather want

system(“echo ‘#{someVar}’ | ./some-script.py >’#{outputFile}’”)

Hi Robert,

Thanks for the help. I used your suggestion of single quotes and now my
script is working like a charm.

Cheers,
Gerad

9

2011/2/10 Robert K. [email protected]:

Is there also a version which would return an IO or accept a block
with IO parameter so we can feed something into the head of the
pipeline and read from the tail?

% ruby -ropen3 -e ’
Open3.pipeline_w(%w[head -6c], %w[cat -n]) {|w|
w.write “ab\ncd\nef\n”
}’
1 ab
2 cd

% ruby -ropen3 -e ’
Open3.pipeline_r([“echo”, “foobarbaz”], [“head”,“-6c”]) {|r|
p r.read
}’
“foo`ba”

10

Brian C. wrote in post #980806:

Does this test case represent your problem?

someVar=“foobar" => "foobar”

echo "#{someVar}"
sh: Syntax error: EOF in backquote substitution
=> “”

This indeed is a good representation of the problem.

I will have to research if my #{someVar} could possibly contain a ’
character. Perhaps Shellwords would be a safer bet.

In case you are still interested, an example of the command with actual
data is below (notice how the sequence @@@@@@S` in the string before " |
collab-table.py" is causing the issue):

echo "Facs Rep|21|@@@@@@Ir||FACS Version 2.1.36.0|2010-09-27 10:57:04.7108-07|21|2|Under Review|||||f||||||f|2010-09-27 11:14:42.686225-07|09/27/2010||||message|t|Facs Rep|09/27/2010|closed|f||18| |||| Facs Rep|21|@@@@@@Lz||FACS Version 3.1.0.0|2010-10-27 10:02:43.778138-07|21|2|Under Review|||||f||||||f|2010-10-27 10:51:17.216686-07|10/27/2010||||message|t|Facs Rep|10/27/2010|closed|f||19| |||| Facs Rep|21|@@@@@@N]||FACS Version 3.2.0.0|2010-11-09 16:56:17.214108-08|21|1|Under Review|||||f||||||f|2010-11-09 16:56:17.214108-08|11/09/2010||||message|t|Facs Rep|11/09/2010|open|f||20| |||| Facs Rep|21|@@@@@@QT||FACS Version 3.3.0.0|2010-12-09 10:19:06.322005-08|21|1|Under Review|||||f||||||f|2010-12-09 10:19:06.322005-08|12/09/2010||||message|t|Facs Rep|12/09/2010|open|f||21| |||| Facs Rep|21|@@@@@@S||FACS Version 3.4.0.0|2011-01-07
10:17:25.142238-08|21|1|Under Review|||||f||||||f|2011-01-07
10:17:25.142238-08|01/07/2011||||message|t|Facs
Rep|01/07/2011|open|f||22| ||||" | ./collab-table.py “message” “D260888”
“hub” “ps_nysdot” “New York State Department of Transportation”
“Alexander Hamilton Bridge & Highbridge Interchange Ramps Rehab” “Facs
Rep” “21” “2011-02-10 11:58:12-0800” “20” “4” “4”>
“cor_6/2011-02-10/D260888/correspondence/CollabmessageList_4.html”`

11

On Thu, Feb 10, 2011 at 3:33 AM, Robert K.
[email protected] wrote:

IO.copy_stream(io, out, 1024)

I keep finding out about nifty 1.9 features like this piecemeal. Is
there an up-to-date guide to 1.8-1.9 library changes somewhere?

12

On Mon, Feb 14, 2011 at 4:52 AM, Tanaka A. [email protected] wrote:

1 ab
2 cd

% ruby -ropen3 -e ’
Open3.pipeline_r([“echo”, “foobarbaz”], [“head”,“-6c”]) {|r|
p r.read
}’
“foo`ba”

Very nice! I tried

13:09:25 Temp$ ./pl.rb
< 1 000>
< 2 001>
< 3 002>
< 4 003>
< 5 004>
13:09:31 Temp$ cat pl.rb
#!/bin/env ruby19

require ‘open3’

Open3.pipeline_rw([“cat”, “-n”], [“sed”, “-e”, ‘s/^.*$/<&>/’]) do
|sin, sout, threads|
th = Thread.new(sout) do |io|
io.each_line do |l|
puts l
end
end

5.times do |i|
sin.printf “%03d\n”, i
end

sin.close
th.join
end
13:09:58 Temp$

Cheers

robert