Tabs and parse errors with Ruby 1.8.4 and Windows

Today I ran into odd syntax errors in Rails rhtml views while trying to
set up acts_as_ferret. I have tried Ruby 1.8.4 and Ruby 1.8.5, both from
the Ruby-mswin32 binary packages and the Ruby O.-click Installer. I am
using ferret gem 1.10.9 and the stable version of acts_as_ferret. I
tried both the pre-built ferret-win32 gem and building the ferret-ruby
gem myself with VC6. The errors change, but always go something like
this:

compile error
./script/…/config/…/app/views/search/query.rhtml:5: Invalid char
`\002’ in expression
./script/…/config/…/app/views/search/query.rhtml:6: parse error,
unexpected tIDENTIFIER, expecting kEND
_erbout.concat " "xP; _erbout.concat(( “%d%%” % (result.RANK /
1000.0 * 100) ).to_s); _erbout.concat “
\n”
^
./script/…/config/…/app/views/search/query.rhtml:7: parse error,
unexpected $, expecting kEND

Views will render fine before I request a page that calls Ferret.
After any use of Ferret, the above errors occur in all views, until I
kill and re-start WEBrick. I have seen a lot of threads around about
this problem, but none of them have any information about what actually
causes it. See for example:

http://rails.techno-weenie.net/question/2006/3/26/how_do_i_get_file_column_plugin_working_on_windows_with_ruby_1_8_4
http://www.ruby-forum.com/topic/80460#new

I would really like to know what the root cause of this problem actually
is, so I can figure out if there’s something that I can do to fix it.
Many threads I’ve seen blame it on RMagick, but I don’t even have
RMagick installed. Most others just say “replace all tabs with spaces.”
Well, OK, but I don’t really see that as an acceptable solution.

Does anyone have any information about why removing tabs fixes this
problem? Why does loading C extensions trigger it? What’s going on with
the Ruby interpreter? I want to know more. :frowning:

Thanks,

  • Will

On 9/29/06, Will R. [email protected] wrote:

./script/…/config/…/app/views/search/query.rhtml:5: Invalid char
After any use of Ferret, the above errors occur in all views, until I
RMagick installed. Most others just say “replace all tabs with spaces.”
Well, OK, but I don’t really see that as an acceptable solution.

Does anyone have any information about why removing tabs fixes this
problem? Why does loading C extensions trigger it? What’s going on with
the Ruby interpreter? I want to know more. :frowning:

Thanks,

  • Will

Hi Will,

I thought this was a problem with using a different version of Ruby to
the version that the extension (Ferret or RMagick or whatever else)
was compiled with. Ferret 0.10.9 was compiled with 1.8.4-20 and it
sounds like you tried that so now I’m confused, especially since it
didn’t work when you tried building the extension yourself.
Unfortunately I can’t replicate the error here, otherwise I might be
able to fix the problem. Could you tell me, does this cause an error
on your system?

irb(main):001:0> require ‘rubygems’
=> true
irb(main):002:0> require ‘ferret’
=> true
irb(main):003:0> “\t\t\t\t Hello”.dump
=> “"\t\t\t\t Hello"”

Cheers,
Dave

David B. wrote:

On 9/29/06, Will R. [email protected] wrote:

./script/…/config/…/app/views/search/query.rhtml:5: Invalid char
After any use of Ferret, the above errors occur in all views, until I
RMagick installed. Most others just say “replace all tabs with spaces.”
Well, OK, but I don’t really see that as an acceptable solution.

I thought this was a problem with using a different version of Ruby to
the version that the extension (Ferret or RMagick or whatever else)
was compiled with. Ferret 0.10.9 was compiled with 1.8.4-20 and it
sounds like you tried that so now I’m confused, especially since it
didn’t work when you tried building the extension yourself.
Unfortunately I can’t replicate the error here, otherwise I might be
able to fix the problem. Could you tell me, does this cause an error
on your system?

Nope, looks fine to me. This is with Ferret 0.10.9 I compiled myself.

D:\Temp\t>ruby -v
ruby 1.8.5 (2006-08-25) [i386-mswin32]

D:\Temp\t>irb
irb(main):001:0> require ‘rubygems’
=> false
irb(main):002:0> require ‘ferret’
=> true
irb(main):003:0> “\t\t\t\t Hello”.dump
=> “"\t\t\t\t Hello"”

Ferret also works fine in my Rails app when I use it from
script/console, it just blows up when requesting pages through WEBrick.
I haven’t personally tried other web servers, but I did see others
reporting that the same thing happens with Mongrel.

Thanks for responding, if you have any other ideas, maybe we can get to
the bottom of this after all.

  • Will

I recently ran into the same problem at work (combination of spurious
Tabs, WEBrick Ferret). The parser just choked on a number of tabs on
Windows systems – but not on Mac OS X systems. We had no idea why, but
just decided to follow with the generally accepted Ruby coding style and
use spaces instead of tabs.

On 30 Sep 2006, at 10:34, Roderick van Domburg wrote:

I recently ran into the same problem at work (combination of spurious
Tabs, WEBrick Ferret). The parser just choked on a number of tabs on
Windows systems – but not on Mac OS X systems. We had no idea why,
but
just decided to follow with the generally accepted Ruby coding
style and
use spaces instead of tabs.

Yeah, I always use spaces (and an editor that inserts spaces when I
hit tab instead of \t), and always be aware that line endings are
different in OS X and Windows, so watch out for ^M disease which has
caused me problems in the past when moving from Windows over to Unix.

HTH,


Paul R.
Director, Vagueware Ltd
http://vagueware.com

Hey Will, did you ever get an answer to this? I am having the same
problem and don’t see switching tabs to spaces as an acceptable
solution…

Thanks,
Chad

Chad wrote:

Hey Will, did you ever get an answer to this? I am having the same
problem and don’t see switching tabs to spaces as an acceptable
solution…

No, I haven’t found a solution, and I don’t see changing tabs to spaces
as acceptable either :stuck_out_tongue:

If you have any insights or things I could maybe try, I’m all ears.

  • Will

Will R. wrote:

compile error
./script/…/config/…/app/views/search/query.rhtml:5: Invalid char

Views will render fine before I request a page that calls Ferret.
After any use of Ferret, the above errors occur in all views, until I
kill and re-start WEBrick.

Apologies for quoting myself. I just got through compiling the full Ruby
kit from the official 1.8.5 source archive, using Microsoft VC++ 6.0
(SP6). The problem still occurs. The only things I did not compile
myself were readline, iconv, and gdbm; for these, I used the same
binaries that the One Click Installer uses. As far as I know, these
libraries have not changed in years (they certainly have not changed
since Ruby 1.8.2), so it is unlikely they are the source of the problem.

At this point, it appears that there is actually a bug in Ruby. Some
people reported the problem starting with the 1.8.4-pre3 release of the
One Click Installer. I could hunt around in the Ruby CVS and try to
figure out what could have changed between 1.8.2 and 1.8.4, but that
task sounds fairly daunting.

Any guidance or pointers regarding what code changes could have caused
this, or why it is only triggered when certain external C modules are
loaded?

  • Will

Will R. wrote:

Will R. wrote:

compile error
./script/…/config/…/app/views/search/query.rhtml:5: Invalid char

Views will render fine before I request a page that calls Ferret.
After any use of Ferret, the above errors occur in all views, until I
kill and re-start WEBrick.

Apologies for quoting myself. I just got through compiling the full Ruby
kit from the official 1.8.5 source archive, using Microsoft VC++ 6.0
(SP6). The problem still occurs. The only things I did not compile
myself were readline, iconv, and gdbm; for these, I used the same
binaries that the One Click Installer uses. As far as I know, these
libraries have not changed in years (they certainly have not changed
since Ruby 1.8.2), so it is unlikely they are the source of the problem.

At this point, it appears that there is actually a bug in Ruby. Some
people reported the problem starting with the 1.8.4-pre3 release of the
One Click Installer. I could hunt around in the Ruby CVS and try to
figure out what could have changed between 1.8.2 and 1.8.4, but that
task sounds fairly daunting.

Any guidance or pointers regarding what code changes could have caused
this, or why it is only triggered when certain external C modules are
loaded?

  • Will
    Hello all…
    Has anyone found out what might be causing this? I am using ferret and
    rmagick and have gotten the code to work on some windows boxes by
    updating base.rb in actionpack to replace tabs with spaces, however on
    other windows boxes (identical setup) the fix doesnt work and my rails
    app will load, but any pages that user ferret or rmagick give me a bad
    file descriptor.

-Brent

Brent S. wrote:

Has anyone found out what might be causing this? I am using ferret and
rmagick and have gotten the code to work on some windows boxes by

Hi Brent,

Please check out the thread on the Ruby forum/list. It got a bit more
attention. Looks like the problem has to do with Microsoft compilers,
but I haven’t found any specific information about why.

http://www.ruby-forum.com/topic/83769

  • Will

Does this mean a project written on a mac cannot be ported over to
Windows unless each .rb’s tabs are removed?

On Oct 14, 2:09 pm, Will R. [email protected]