[Bug #3345] webrick test failure on Windows(?)

Bug #3345: webrick test failure on Windows(?)
http://redmine.ruby-lang.org/issues/show/3345

e$B5/I<<Te(B: Usaku NAKAMURA
e$B%9%F!<%?%9e(B: Open, e$BM%@hEYe(B: Normal
e$B%+%F%4%je(B: lib, Target version: 1.9.2
ruby -v: ruby 1.9.3dev (2010-05-26 trunk 28018) [i386-mswin32]

webricke$B$N%F%9%H$,0J2<$N$h$&$K<:GT$7$^$9!#e(B

  1. Failure:
    test_cgi(TestWEBrickCGI) [C:/ruby-trunk/test/webrick/test_cgi.rb:40]:
    webrick log start:
    [2010-05-26 17:32:08] INFO WEBrick 1.3.1
    [2010-05-26 17:32:08] INFO ruby 1.9.3 (2010-05-26) [i386-mswin32]
    [2010-05-26 17:32:08] INFO WEBrick::HTTPServer#start: pid=6292
    port=60199
    [2010-05-26 17:32:08] WARN :RequestHandler is deprecated, please use
    :RequestCallback

[2010-05-26 17:32:09] WARN :RequestHandler is deprecated, please use
:RequestCallback

[2010-05-26 17:32:11] WARN :RequestHandler is deprecated, please use
:RequestCallback

[2010-05-26 17:32:12] WARN :RequestHandler is deprecated, please use
:RequestCallback
[2010-05-26 17:32:12] ERROR Encoding::CompatibilityError: incompatible
character encodings: ASCII-8BIT and Windows-31J
C:/ruby-trunk/lib/webrick/httpservlet/filehandler.rb:225:in
expand_path' C:/ruby-trunk/lib/webrick/httpservlet/filehandler.rb:225:in prevent_directory_traversal’
C:/ruby-trunk/lib/webrick/httpservlet/filehandler.rb:166:in
service' C:/ruby-trunk/lib/webrick/httpserver.rb:111:in service’
C:/ruby-trunk/lib/webrick/httpserver.rb:70:in run' C:/ruby-trunk/lib/webrick/server.rb:183:in block in
start_thread’

webrick log end.
<“/\xA4\xDB\xA4\xB2/\xA4\xDB\xA4\xB2”> expected but was
<"\n\n

Internal Server Error\n \n

Internal Server Error

\n incompatible character encodings: ASCII-8BIT and Windows-31J\n
\n \n WEBrick/1.3.1 (Ruby/1.9.3/2010-05-26) OpenSSL/1.0.0 at\n 127.0.0.1:60199\n \n \n\n">.

e$B%/%i%$%"%s%H$+$iEO$5$l$?!Ve(B/%A4%DB%A4%B2/%A4%DB%A4%B2e$B!W$H$$$&e(B
e$B%Q%9$r<B%U%!%$%k%7%9%F%`$N%Q%9$KJQ49$7$h$&$H$7$F$$$k$o$1$G$9$,!"e(B
(1)
e$B!Ve(B/%A4%DB%A4%B2/%A4%DB%A4%B2e$B!W$re(Bunescapee$B$9$k$He(BASCII-8BITe$B$K$J$ke(B
(2) e$B$=$l$re(BFile.expand_pathe$B$KEO$9$H!“e(Bfilesystem
encodinge$B$G$”$k$O$:$Ne(B
cwde$B$H$/$C$D$1$h$&$H$9$k$N$G%(%i!<$K$J$ke(B
e$B$H$$$&OC$N$h$&$G$9!#e(B

e$B$H$$$&$o$1$G!"0J2<$N%Q%C%A$G$$$A$*$&D>$k$s$G$9$,!"OC$H$7$F$OB>$Ne(B
e$B%W%i%C%H%U%)!<%`$G$bH/@8$7$F$7$+$k$Y$-LdBj$N$h$&$J5$$,$9$k$N$K!"e(B
e$B$I$&$b;d$7$+F’$s$G$J$$$C$]$$$N$O$J$<$G$7$g$&e(B?
e$B$^$?!"D>$7J}$H$7$F$O$?$7$F$3$l$G$$$$$s$G$7$g$&$+e(B?
e$B$I$J$?$+65$($F$/$@$5$$!#e(B

Index: lib/webrick/httputils.rb

— lib/webrick/httputils.rb (revision 28018)
+++ lib/webrick/httputils.rb (working copy)
@@ -27,7 +27,7 @@ module WEBrick
while ret.sub!(%r’/(?!../)[^/]+/..(?:/|\Z)', ‘/’); end #
/foo/… => /foo

   raise "abnormal path `#{path}'" if %r{/\.\.(/|\Z)} =~ ret
  •  ret
    
  •  ret.force_encoding(Encoding.find("filesystem"))
    
    end
    module_function :normalize_path

e$B%A%1%C%He(B #3345 e$B$,99?7$5$l$^$7$?!#e(B (by Yusuke E.)

e$BC4Ev<Te(B Usaku NAKAMURAe$B$K%;%C%He(B

e$B1sF#$G$9!#e(B

e$B$H$$$&$o$1$G!"0J2<$N%Q%C%A$G$$$A$*$&D>$k$s$G$9$,!"OC$H$7$F$OB>$Ne(B
e$B%W%i%C%H%U%)!<%`$G$bH/@8$7$F$7$+$k$Y$-LdBj$N$h$&$J5$$,$9$k$N$K!"e(B
e$B$I$&$b;d$7$+F’$s$G$J$$$C$]$$$N$O$J$<$G$7$g$&e(B?

File.expand_path(“/webrick.cgi/\xA4\xDB\xA4\xB2/\xA4\xDB\xA4\xB2”.force_encoding(“ASCII-8BIT”))

e$B$N$h$&$J$3$H$,<B9T$5$l$F$$$F!"@hF,$,e(B /
e$B$G;O$^$C$F$$$k$?$a!"e(BLinux e$B$J$Ie(B
e$B$@$HFC$K2?$b$7$J$$$1$l$I!"e(BWindows e$B$@$H2?$+e(B
(e$B%I%i%$%V%l%?!<e(B?) e$B$rB-$=$&e(B
e$B$H$9$k$?$aMn$A$k$N$+$H;W$$$^$9!#e(B

unak e$B$5$s$N%Q%C%A$@$H!“e(BLinux
e$B$G%(%i!<$,H/@8$9$k$h$&$K$J$j$^$7$?!#e(B
filesystem encoding e$B$Oe(B UTF-8 e$B$G$9$,!”$3$N%Q%9$Oe(B UTF-8
e$B$H$7$Fe(B invalid
e$B$J$N$G!"e(B=~ e$B$G@55,I=8=%^%C%A$,$G$-$J$$$3$H$,860x$G$9!#e(B

  1. Failure:
    test_cgi(TestWEBrickCGI)
    [/home/mame/work/ruby/test/webrick/test_cgi.rb:40]:
    webrick log start:
    [2010-06-16 02:48:55] INFO WEBrick 1.3.1
    [2010-06-16 02:48:55] INFO ruby 1.9.3 (2010-06-16) [i686-linux]
    [2010-06-16 02:48:55] INFO WEBrick::HTTPServer#start: pid=15315
    port=38912
    [2010-06-16 02:48:55] WARN :RequestHandler is deprecated, please use
    :RequestCallback

[2010-06-16 02:48:55] WARN :RequestHandler is deprecated, please use
:RequestCallback

[2010-06-16 02:48:55] WARN :RequestHandler is deprecated, please use
:RequestCallback

[2010-06-16 02:48:55] WARN :RequestHandler is deprecated, please use
:RequestCallback
[2010-06-16 02:48:55] ERROR ArgumentError: invalid byte sequence in
UTF-8
/home/mame/work/ruby/lib/webrick/httpserver.rb:196:in =~' /home/mame/work/ruby/lib/webrick/httpserver.rb:196:in scan’
/home/mame/work/ruby/lib/webrick/httpserver.rb:136:in
search_servlet' /home/mame/work/ruby/lib/webrick/httpserver.rb:105:in service’
/home/mame/work/ruby/lib/webrick/httpserver.rb:70:in run' /home/mame/work/ruby/lib/webrick/server.rb:183:in block in
start_thread’

prevent_directory_traversal e$B$NLdBj$r@53N$KM}2r$7$F$$$J$$$N$G$9$,!"e(B
e$BLdBj$,$*$-$k$N$,e(B Windows e$B$@$1$G!"e(B\
e$B$N07$$$@$1$NLdBj$@$H$7$?$i!"e(B
File.expand_path e$B$r;H$o$:e(B gsub
e$B$G=hM}$9$k$H$$$$$N$G$O$J$$$G$7$g$&e(B
e$B$+!#e(B

diff --git a/lib/webrick/httpservlet/filehandler.rb
b/lib/webrick/httpservlet/filehandler.rb
index 32c1965…50bf27c 100644
— a/lib/webrick/httpservlet/filehandler.rb
+++ b/lib/webrick/httpservlet/filehandler.rb
@@ -214,17 +214,13 @@ module WEBrick
# character in URI notation. So the value of path_info should
be
# normalize before accessing to the filesystem.

  •    if trailing_pathsep?(req.path_info)
    
  •      # File.expand_path removes the trailing path separator.
    
  •      # Adding a character is a workaround to save it.
    
  •      #  File.expand_path("/aaa/")        #=> "/aaa"
    
  •      #  File.expand_path("/aaa/" + "x")  #=> "/aaa/x"
    
  •      expanded = File.expand_path(req.path_info + "x")
    
  •      expanded.chop!  # remove trailing "x"
    
  •    else
    
  •      expanded = File.expand_path(req.path_info)
    
  •    is_windows = RUBY_PLATFORM =~ /mswin/  ||
    
  •                 RUBY_PLATFORM =~ /mingw/  ||
    
  •                 RUBY_PLATFORM =~ /bccwin/ ||
    
  •                 RUBY_PLATFORM =~ /wince/
    
  •    if is_windows
    
  •      req.path_info = req.path_info.gsub("\\", "/")
       end
    
  •    req.path_info = expanded
     end
    
     def exec_handler(req, res)
    


Yusuke E. [email protected]

http://redmine.ruby-lang.org/issues/show/3345

e$B1sF#$G$9!#e(B

2010e$BG/e(B6e$B7ne(B16e$BF|e(B9:45 U.Nakamura [email protected]:

e$BC4Ev<Te(B Usaku NAKAMURAe$B$K%;%C%He(B

e$B$(!"$J$s$Ge(B?
e$BJs9p$7$?$iC4Ev$7$J$$$H$$$1$J$$$Ne(B?

e$BJs9p<T$@$+$i$G$O$J$/!"e(Bwindows
e$B$N%W%i%C%H%U%)!<%%a%s%F%J$H$$$&$3$H$Ge(B e$BC4Ev<T$K$5$;$F$b$i$$$^$7$?!#%a%s%F%J$,$$$J$$%i%$%V%i%j$N!"%W%i%C%He(B e$B%U%)!<%FCM-$NLdBj$O%W%i%C%H%U%)!<%`%a%s%F%J$K8+$F$b$i$&$7$+$J$$$+$Je(B
e$B$H;W$C$F!#e(B
e$B8+$?$/$J$$$H$$$&$3$H$G$7$?$i!“e(BWONTFIX
e$B$HH=CG$9$k8"8B$O$”$k$H;W$$$^$9!#e(B

File.expand_pathe$B$,9T$C$F$$$k@55,2==hM}$HF1Ey$N=hM}$rA4It$3$3e(B
e$B$KJB$Y$k$N$O$5$9$,$K$A$g$C$H!#e(B

File.expand_path
e$B$O@dBP%Q%9$KJQ49$9$k%a%=%C%I$G$“$C$F!”@55,2=$K;H$&e(B
e$B$N$O6Z0c$$$G$O$J$$$+$H$$$&5$$,$7$^$9!#e(B
trailing_pathsep? e$B$N=hM}$,I,MW$J$3$H$,K5>Z!)e(B

e$B6Z$,$I$&$3$&8@$C$F$F$be(B security issue
e$B$O$7$g$&$,$J$$$N$G$7$g$&$,$J$$e(B
e$B$N$G$9$,!“$8$c$”$3$&$$$&%Q%C%A$O$I$&$G$7$g$&$+!#e(B
windows e$B$G$O;n$7$F$$$^$;$s$,!#e(B

diff --git a/lib/webrick/httpservlet/filehandler.rb
b/lib/webrick/httpservlet/filehandler.rb
index 32c1965…4887903 100644
— a/lib/webrick/httpservlet/filehandler.rb
+++ b/lib/webrick/httpservlet/filehandler.rb
@@ -214,16 +214,18 @@ module WEBrick
# character in URI notation. So the value of path_info should
be
# normalize before accessing to the filesystem.

  •    path = 
    

req.path_info.dup.force_encoding(Encoding.find(“filesystem”))
if trailing_pathsep?(req.path_info)
# File.expand_path removes the trailing path separator.
# Adding a character is a workaround to save it.
# File.expand_path(“/aaa/”) #=> “/aaa”
# File.expand_path(“/aaa/” + “x”) #=> “/aaa/x”

  •      expanded = File.expand_path(req.path_info + "x")
    
  •      expanded = File.expand_path(path + "x")
         expanded.chop!  # remove trailing "x"
       else
    
  •      expanded = File.expand_path(req.path_info)
    
  •      expanded = File.expand_path(path)
       end
    
  •    expanded.force_encoding(req.path_info.encoding)
       req.path_info = expanded
     end
    

e$B$3$s$K$A$O!"$J$+$`$ie(B(e$B$&e(B)e$B$G$9!#e(B

In message “[ruby-dev:41615] [Bug #3345] webrick test failure on
Windows(?)”
on Jun.16,2010 03:13:55, [email protected] wrote:

e$BC4Ev<Te(B Usaku NAKAMURAe$B$K%;%C%He(B

e$B$(!"$J$s$Ge(B?
e$BJs9p$7$?$iC4Ev$7$J$$$H$$$1$J$$$Ne(B?

e$B$H$$$&$o$1$G!"0J2<$N%Q%C%A$G$$$A$*$&D>$k$s$G$9$,!"OC$H$7$F$OB>$Ne(B
e$B%W%i%C%H%U%)!<%`$G$bH/@8$7$F$7$+$k$Y$-LdBj$N$h$&$J5$$,$9$k$N$K!"e(B
e$B$I$&$b;d$7$+F’$s$G$J$$$C$]$$$N$O$J$<$G$7$g$&e(B?

File.expand_path(“/webrick.cgi/\xA4\xDB\xA4\xB2/\xA4\xDB\xA4\xB2”.force_encoding(“ASCII-8BIT”))

e$B$N$h$&$J$3$H$,<B9T$5$l$F$$$F!"@hF,$,e(B / e$B$G;O$^$C$F$$$k$?$a!"e(BLinux e$B$J$Ie(B
e$B$@$HFC$K2?$b$7$J$$$1$l$I!"e(BWindows e$B$@$H2?$+e(B (e$B%I%i%$%V%l%?!<e(B?) e$B$rB-$=$&e(B
e$B$H$9$k$?$aMn$A$k$N$+$H;W$$$^$9!#e(B

e$B$J$k$[$I!#e(B

unak e$B$5$s$N%Q%C%A$@$H!“e(BLinux e$B$G%(%i!<$,H/@8$9$k$h$&$K$J$j$^$7$?!#e(B
filesystem encoding e$B$Oe(B UTF-8 e$B$G$9$,!”$3$N%Q%9$Oe(B UTF-8 e$B$H$7$Fe(B invalid
e$B$J$N$G!"e(B=~ e$B$G@55,I=8=%^%C%A$,$G$-$J$$$3$H$,860x$G$9!#e(B

e$B$=$3$r$$$8$k$N$OCO9vF;$C$]$$$G$9$M$(!#e(B

prevent_directory_traversal e$B$NLdBj$r@53N$KM}2r$7$F$$$J$$$N$G$9$,!"e(B
e$BLdBj$,$*$-$k$N$,e(B Windows e$B$@$1$G!"e(B\ e$B$N07$$$@$1$NLdBj$@$H$7$?$i!"e(B
File.expand_path e$B$r;H$o$:e(B gsub e$B$G=hM}$9$k$H$$$$$N$G$O$J$$$G$7$g$&e(B
e$B$+!#e(B

e$B;DG0$J$,$ie(B \ e$B$N07$$$@$1$NLdBj$G$O$J$$$N$Ge(B…
e$B6qBNE*$K$O!“C;$$%U%!%$%kL>7A<0$rMxMQ$7$?%”%/%;%9$d!“e(BNTFSe$B%G!<e(B
e$B%?%9%H%j!<%$N<g%9%H%j!<%L@<(;XDj%”%/%;%9$J$I$K$h$k%"%/%;%9e(B
e$B@)8f2sHr$r!"e(BFile.expand_pathe$B$N@55,2==hM}$rMxMQ$7$FGS=|$7$F$$e(B
e$B$^$9!#e(B
File.expand_pathe$B$,9T$C$F$$$k@55,2==hM}$HF1Ey$N=hM}$rA4It$3$3e(B
e$B$KJB$Y$k$N$O$5$9$,$K$A$g$C$H!#e(B

e$B$=$l$G$O!#e(B

e$B%A%1%C%He(B #3345 e$B$,99?7$5$l$^$7$?!#e(B (by Yusuke E.)

e$B%9%F!<%?%9e(B Opene$B$+$ie(BClosede$B$KJQ99e(B
e$B?JD=e(B % 0e$B$+$ie(B100e$B$KJQ99e(B

This issue was solved with changeset r28354.
Usaku, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


http://redmine.ruby-lang.org/issues/show/3345

e$B$3$s$K$A$O!"$J$+$`$ie(B(e$B$&e(B)e$B$G$9!#e(B

In message “[ruby-dev:41632] Re: [Bug #3345] webrick test failure on
Windows(?)”
on Jun.17,2010 12:39:06, [email protected] wrote:

e$BC4Ev<Te(B Usaku NAKAMURAe$B$K%;%C%He(B

e$B$(!"$J$s$Ge(B?
e$BJs9p$7$?$iC4Ev$7$J$$$H$$$1$J$$$Ne(B?

e$BJs9p<T$@$+$i$G$O$J$/!"e(Bwindows e$B$N%W%i%C%H%U%)!<%%a%s%F%J$H$$$&$3$H$Ge(B e$BC4Ev<T$K$5$;$F$b$i$$$^$7$?!#%a%s%F%J$,$$$J$$%i%$%V%i%j$N!"%W%i%C%He(B e$B%U%)!<%FCM-$NLdBj$O%W%i%C%H%U%)!<%`%a%s%F%J$K8+$F$b$i$&$7$+$J$$$+$Je(B
e$B$H;W$C$F!#e(B

e$B%a!<%kAw$C$F$+$i$=$&$$$&0UL#$@$m$&$H5$IU$-$^$7$?!#$9$_$^$;$s!#e(B

e$B8+$?$/$J$$$H$$$&$3$H$G$7$?$i!“e(BWONTFIX e$B$HH=CG$9$k8"8B$O$”$k$H;W$$$^$9!#e(B

e$B8+$?$/$J$$$H$$$&$+!“$h$/$o$+$s$J$$$+$i<j$,=P$;$J$$$@$1$G!“C/e(B
e$B$+Cf?H$,J,$+$k?M$K$OD>$7$F$[$7$$$G$9!”$b$A$m$s!#e(B
e$B;d$H$7$F$OK\7o$O5$J,E*$K$OHo32<T$J$N$G!”@UG$<T=P$F$3$$%4%k%!e(B
e$B$C$F$J$b$s$G$9e(B(e$B$$$d@UG$<T$J$s$F$b$N$O$$$J$$$s$@$1$Ie(B)e$B!#e(B

trailing_pathsep? e$B$N=hM}$,I,MW$J$3$H$,K5>Z!)e(B
e$B6Z0c$$$G$7$g$&$M!#e(B
e$B$G$b$^$"8=<B$H$7$FB>$K>/$J$/$H$be(BRubye$B%l%Y%k$Ge(BAPIe$B$J$$$s$G$9$h$Me(B
e$B$(!#e(B
(Ce$B%l%Y%k$G$b$J$$$1$Ie(B)

e$B6Z$,$I$&$3$&8@$C$F$F$be(B security issue e$B$O$7$g$&$,$J$$$N$G$7$g$&$,$J$$e(B
e$B$N$G$9$,!“$8$c$”$3$&$$$&%Q%C%A$O$I$&$G$7$g$&$+!#e(B
windows e$B$G$O;n$7$F$$$^$;$s$,!#e(B

e$B$H$j$“$($:e(BWEBricke$B$N%F%9%H$ODL$C$F$k$h$&$K8+$($^$9!#e(B
e$B$N$G!”;d$H$7$F$O$3$l$G$+$^$$$^$;$s$G$9!#e(B

e$B$=$l$G$O!#e(B