Can't open url with a subdomain with an underscore

Ruby
1

I try to open the following URL: http://auto_diversen.marktplaza.nl/

open-uri throws the following error: the scheme http does not accept
registry part

I tried several solutions like this:

but that doesn’t work.
and this;
http://jessehowarth.com/2011/04/02/handling-underscores-in-subdomains-with-ruby
but none worked.

I use ruby 1.9.3p0

Anyone an idea what the solution could be?

2

Bartosz Dziewoński wrote in post #1054173:

Underscore is not a valid character in a hostname, thus Ruby rejects it.

To allow it, you have to manually add it to the regex in library file
uri/common.rb, line 368. (On my Windows install,
F:\Ruby193\lib\ruby\1.9.1\uri), to looks like this:

ret[:HOSTNAME] = hostname = “(?:[a-zA-Z0-9\-._]|%\h\h)+”

(I just added an underscore there.) This might of course break URI
parsing for other protocols.

– Matma R.

Thanks for the reply, but I don’t think it’s a good idea to ‘hack’ the
library. Like you said ‘it could break other URI’s’.

But even if I’m going to hackl the library, I ran into another
difficulty. I’ll have to hack the library on all the servers I deploy
the Ruby-app to.

Again, thanks for your reply, but I don’t think it’s a good idea.

3

Well, you could also copy the patched URI::Parser#initialize_pattern
method to all your files that need it. (Just make sure to require
‘uri’ first, and monkeypatch later.) Like this:

require ‘uri’
class URI::Parser
def initialize_pattern(opts = {})
<snip…>
end
end

There is no other way to do it, without rewriting the entire URI
library from scratch. (And that would be stupid, even more so since
the way you want it to work is technically invalid.)

– Matma R.

4

Underscore is not a valid character in a hostname, thus Ruby rejects it.

To allow it, you have to manually add it to the regex in library file
uri/common.rb, line 368. (On my Windows install,
F:\Ruby193\lib\ruby\1.9.1\uri), to looks like this:

ret[:HOSTNAME] = hostname = “(?:[a-zA-Z0-9\-._]|%\h\h)+”

(I just added an underscore there.) This might of course break URI
parsing for other protocols.

– Matma R.

5

2012/3/31 Bartosz Dziewoński [email protected]

There is no other way to do it, without rewriting the entire URI
library from scratch. (And that would be stupid, even more so since
the way you want it to work is technically invalid.)

– Matma R.

I feel like there’s probably a better way than this, but man this lib
really confuses me.

require ‘open-uri’
URI::Parser.new(:HOSTNAME => “(?:[_a-zA-Z0-9\-.]|%\h\h)+”).parse("
http://auto_diversen.marktplaza.nl/").open.read

6

I have already explained. Change or monkey-patch the URI library.

2012/4/17, Jeroen van Ingen [email protected]:

7

if I have the following url: http://auto_diversen.marktplaza.nl

This works fine:
URI::Parser.new(:HOSTNAME => “(?:[_a-zA-Z0-9\-.]|%\h\h)+”).parse(url)

But this returns an error:
URI::Parser.new(:HOSTNAME =>
“(?:[_a-zA-Z0-9\-.]|%\h\h)+”).parse(url).open

Returns: the scheme http does not accept registry part:
auto_diversen.marktplaza.nl:80 (or bad hostname?)

When I look to the source of ‘open’, It looks to be that it parses the
URL again, so the error is raised.

The source on:

File open-uri.rb, line 27

def open(name, *rest, &block) # :doc:
if name.respond_to?(:open)
name.open(rest, &block)
elsif name.respond_to?(:to_str) &&
%{\A[A-Za-z][A-Za-z0-9+-.]://} =~ name &&
(uri = URI.parse(name)).respond_to?(:open)
uri.open(*rest, &block)
else
open_uri_original_open(name, *rest, &block)
end
end

Anyone got an idea how to open the uri?

8

Bartosz Dziewoński wrote in post #1056978:

I have already explained. Change or monkey-patch the URI library.

2012/4/17, Jeroen van Ingen [email protected]:

I tried, but when I paste the source from:
initialize_pattern (URI::Parser) - APIdock I couldn’t get it
to work for the URL: http://auto_diversen.marktplaza.nl

Probably I’m adding the underscore in the wrong place. Can you tell me
which lines I have to change to get it worked for the given URL?

9

W dniu 30 marca 2012 13:28 użytkownik Bartosz Dziewoński
[email protected] napisał:

Underscore is not a valid character in a hostname, thus Ruby rejects it.

To allow it, you have to manually add it to the regex in library file
uri/common.rb, line 368. (On my Windows install,
F:\Ruby193\lib\ruby\1.9.1\uri), to looks like this:

ret[:HOSTNAME] = hostname = “(?:[a-zA-Z0-9\-._]|%\h\h)+”

(I just added an underscore there.) This might of course break URI
parsing for other protocols.

The line before editing looks like this:

ret[:HOSTNAME] = hostname = “(?:[a-zA-Z0-9\-.]|%\h\h)+”

(I am on Ruby 1.9.3)

– Matma R.

10

Bartosz Dziewoński wrote in post #1057001:

W dniu 30 marca 2012 13:28 użytkownik Bartosz Dziewoński
[email protected] napisał:

Underscore is not a valid character in a hostname, thus Ruby rejects it.

To allow it, you have to manually add it to the regex in library file
uri/common.rb, line 368. (On my Windows install,
F:\Ruby193\lib\ruby\1.9.1\uri), to looks like this:

ret[:HOSTNAME] = hostname = “(?:[a-zA-Z0-9\-._]|%\h\h)+”

(I just added an underscore there.) This might of course break URI
parsing for other protocols.

The line before editing looks like this:

ret[:HOSTNAME] = hostname = “(?:[a-zA-Z0-9\-.]|%\h\h)+”

(I am on Ruby 1.9.3)

– Matma R.

I did add this line:
ret[:HOSTNAME] = hostname = “(?:[a-zA-Z0-9\-._]|%\h\h)+”

But still getting the error
URI::InvalidURIError: the scheme http does not accept registry part:
auto_diversen.marktplaza.nl:80 (or bad hostname?)

When executing:
URI::Parser.new.parse(url).open

Are you sure I only need to monkey-patch URI::Parser#initialize_pattern
?