Forum: Ruby-dev [ruby-trunk - Bug #5637][Open] warnings of shellescape

Issue #5637 has been reported by Kazuhiro NISHIYAMA.

----------------------------------------
Bug #5637: warnings of shellescape
http://redmine.ruby-lang.org/issues/5637

Author: Kazuhiro NISHIYAMA
Status: Open
Priority: Normal
Assignee:
Category: lib
Target version: 2.0.0
ruby -v: ruby 2.0.0dev (2011-11-15 trunk 33753) [x86_64-linux]


\\あ

$B@>;3OB9-$G$9!#(B

redmine $B$NJ}$G=q$/$H>C$($F$7$^$&$h$&$J$N$G!"%a!<%k$G=q$-D>$7$^$9!#(B

Shellwords.shellescape $B$G7Y9p$,=P$^$9!#(B


% ./ruby -v -r shellwords -e 'p Shellwords.shellescape("\u3042")'
ruby 2.0.0dev (2011-11-15 trunk 33753) [x86_64-linux]
/home/chkbuild/tmp/build/ruby-trunk/20111114T222552Z/lib/ruby/1.9.1/shellwords.rb:86: 
warning: regexp match /.../n against to UTF-8 string
/home/chkbuild/tmp/build/ruby-trunk/20111114T222552Z/lib/ruby/1.9.1/shellwords.rb:86: 
warning: regexp match /.../n against to UTF-8 string
/home/chkbuild/tmp/build/ruby-trunk/20111114T222552Z/lib/ruby/1.9.1/shellwords.rb:86: 
warning: regexp match /.../n against to UTF-8 string
"\\$B$"(B"


$B%(%9%1!<%W7k2L$bJQ$@$H;W$$$^$9!#(B
$B%(%9%1!<%W7k2L$r(B 1.8.7 $B$K$"$o$;$k$N$J$i0J2<$N%Q%C%A$G(B
$B$I$&$G$7$g$&$+!#(B


diff --git a/lib/shellwords.rb b/lib/shellwords.rb
index 5d6ba75..78331a7 100644
--- a/lib/shellwords.rb
+++ b/lib/shellwords.rb
@@ -79,11 +79,11 @@ module Shellwords
     # An empty argument will be skipped, so return empty quotes.
     return "''" if str.empty?

-    str = str.dup
+    str = str.dup.force_encoding("ASCII-8BIT")

     # Process as a single byte sequence because not all shell
     # implementations are multibyte aware.
-    str.gsub!(/([^A-Za-z0-9_\-.,:\/@\n])/n, "\\\\\\1")
+    str.gsub!(/([^A-Za-z0-9_\-.,:\/@\n])/, "\\\\\\1")

     # A LF cannot be escaped with a backslash because a backslash + LF
     # combo is regarded as line continuation and simply ignored.
diff --git a/test/test_shellwords.rb b/test/test_shellwords.rb
index d48a888..cbc5043 100644
--- a/test/test_shellwords.rb
+++ b/test/test_shellwords.rb
@@ -36,4 +36,8 @@ class TestShellwords < Test::Unit::TestCase
       shellwords(bad_cmd)
     end
   end
+
+  def test_shellescape_utf8_string
+    assert_equal "\\\343\\\201\\\202", shellescape("\u3042")
+  end
 end

Issue #5637 has been updated by Akinori MUSHA.

Assignee set to Akinori MUSHA


----------------------------------------
Bug #5637: warnings of shellescape
http://redmine.ruby-lang.org/issues/5637

Author: Kazuhiro NISHIYAMA
Status: Open
Priority: Normal
Assignee: Akinori MUSHA
Category: lib
Target version: 2.0.0
ruby -v: -


\\あ

Issue #5637 has been updated by dariocravero (Darío Cravero).


Hi,

Thanks for this patch!.. :)

One question though, from comment #3 it's not clear if it's safe to use 
it in 1.9.3. This is what Google Translator gave me:

  "1.9: this behavior was all the way to 1.9.3 now. Turn off warning but 
does not change as a bug (missing fix of / / n), because the behavior 
leads to incompatibility."

However, I've applied it and, as expected, I don't see the warning 
anymore. Still, can you just confirm there're no side effects to this on 
1.9.3?

Thanks a million!..
----------------------------------------
Bug #5637: warnings of shellescape
https://bugs.ruby-lang.org/issues/5637#change-28683

Author: znz (Kazuhiro NISHIYAMA)
Status: Closed
Priority: Normal
Assignee: knu (Akinori MUSHA)
Category: lib
Target version: 2.0.0
ruby -v: -


\\あ

Issue #5637 has been updated by knu (Akinori MUSHA).


As I documented, it's all up to how you use the resulted string.

If you are going to pass it to a shell that lacks support for the 
encoding of the string, then you should probably encode the original 
string in ASCII-8BIT before shell-escaping with shellescape() to get a 
byte-by-byte escape to make sure the shell won't find a metacharacter 
inside a multibyte character.

UTF-8 multibyte characters do not contain any ASCII character by design 
anyway, so most people in the everything-is-UTF-8 world don't even have 
to care about this.

But, for example, when you have to run a program passing a Shift_JIS 
string via a shell under a non-Shift_JIS locale, you'd probably have to 
compose the command line in the ASCII-8BIT encoding so that all shell 
metacharacters that may appear in Shift_JIS multibyte characters are 
properly escaped.
----------------------------------------
Bug #5637: warnings of shellescape
https://bugs.ruby-lang.org/issues/5637#change-28727

Author: znz (Kazuhiro NISHIYAMA)
Status: Closed
Priority: Normal
Assignee: knu (Akinori MUSHA)
Category: lib
Target version: 2.0.0
ruby -v: -


\\あ