Url_for and ampersand escaping?

Rails
1

I have two different servers, with two different test rails apps, both
of which claim to be running Rails 2.1.2.

On one of them, url_for in a view environment generates & in between
query parameters, instead of just &. On the other, it generates just &.

Huh? I can’t figure out why this is one way in one app that claims to be
Rails 2.1.2, and another in another.

In neither one does url_for called in a controller context (rather than
a helper context) use “&” to separate query parameters, it just uses
“&”.

It’s driving me crazy. Anyone have any idea what might be going on?

Jonathan

2

On Fri, Jan 2, 2009 at 3:44 PM, Jonathan R. <
[email protected]> wrote:

In neither one does url_for called in a controller context (rather than
a helper context) use “&” to separate query parameters, it just uses
“&”.

It’s driving me crazy. Anyone have any idea what might be going on?

Hi, can you post the relevant code or snippet of it?

-Conrad

3

Oh boy, this was a crazy one. It wasn’t url_for that was behaving
differently. url_for on both servers seperates query parameters with
& (url_for the helper method; url_for the controller method does
not. I did not know this. Kinda confusing and unpredictable–and
undocumented).

But okay. I was then passing it through REXML in order to ‘escape’ it
for eventual inclusion in some XML:

xml_escaped = REXML::Text.new( generated_url ).to_s

The difference between my two servers was ruby version, not Rails
version. REXML is included with stock ruby (I didn’t realize that
either).

REXML::Text that comes with ruby 1.8.5 will not ‘double escape’ & to
&amp; .

I guess I’d consider that a bug, indeed. It probably should be double
escaping something like that, if you pass it in escaped.

REXML::Text that comes with ruby 1.8.6, on the other hand, WILL double
escape that text passed in escaped.

Wooh, what a mess.

Jonathan

Jonathan R. wrote:

I have two different servers, with two different test rails apps, both
of which claim to be running Rails 2.1.2.

On one of them, url_for in a view environment generates & in between
query parameters, instead of just &. On the other, it generates just &.

Huh? I can’t figure out why this is one way in one app that claims to be
Rails 2.1.2, and another in another.

In neither one does url_for called in a controller context (rather than
a helper context) use “&” to separate query parameters, it just uses
“&”.

It’s driving me crazy. Anyone have any idea what might be going on?

Jonathan

4

On Fri, Jan 2, 2009 at 4:08 PM, Jonathan R.
[email protected] wrote:

… url_for on both servers seperates query parameters with
& (url_for the helper method; url_for the controller method does
not. I did not know this. Kinda confusing and unpredictable–and
undocumented).

Why “unpredictable”? HTML requires ampersands to be escaped,
as part of URLs or otherwise.

A URL in plain text format, though, should not have ampersands
escaped.

It would be confusing if the two url_fors worked differently :slight_smile:

–
Hassan S. ------------------------ [email protected]

5

On Fri, Jan 2, 2009 at 5:08 PM, Jonathan R.
[email protected] wrote:

I knew that a URL in xHTML required ampersands to be escaped like that,
even in an . I did not know that a URL in standard (non-x)HTML
required that. Really? Okay.

http://www.w3.org/TR/html401/charset.html#h-5.3.2

But it’s confusing in part because an ERB template isn’t only used for
HTML. It can theoretically be used for creating any format, including
plain text, right? And someone using an ERB template to create (eg)
plain text is going to get tripped up there.

Interesting point – I haven’t tried generating any text/plain from an
ERB template.

An ERB template was generating XML. It took the result of a url_for
call, and put it through an XML-escaping routine, figuring that anything
that was being put in XML should be put through an XML escaping routine.

So we wound up with XML who’s source looked like
<some_url>/controller/action?foo=foo&amp;bar=bar

Is this correct or not?

I’d say not :slight_smile:

Try eliminating the extra escaping routine and see what happens…

HTH,

Hassan S. ------------------------ [email protected]

6

Hassan S. wrote:

On Fri, Jan 2, 2009 at 4:08 PM, Jonathan R.
[email protected] wrote:

… url_for [helper method] seperates query parameters with
& url_for the controller method does
not.

Why “unpredictable”? HTML requires ampersands to be escaped,
as part of URLs or otherwise.

A URL in plain text format, though, should not have ampersands
escaped.

I knew that a URL in xHTML required ampersands to be escaped like that,
even in an . I did not know that a URL in standard (non-x)HTML
required that. Really? Okay.

But it’s confusing in part because an ERB template isn’t only used for
HTML. It can theoretically be used for creating any format, including
plain text, right? And someone using an ERB template to create (eg)
plain text is going to get tripped up there.

In my case, I wasn’t creating plain text, I was creating XML with an ERB
template. Which should be a perfectly fine thing to do, right? Sure, you
can use Builder if you want for XML, but you should be able to use an
ERB template too, right? But this definitely isn’t the first time I’ve
been confused by the proper amount of escaping of an ampersand in a
complicated data flow.

I’m still not exactly sure if I fixed my bug in the right part of my
somewhat complicated chain of data flow. I’d appreciate if you have any
insight, Hassan. Here’s what was going on:

An ERB template was generating XML. It took the result of a url_for
call, and put it through an XML-escaping routine, figuring that anything
that was being put in XML should be put through an XML escaping routine.
(Is this where I went wrong? Not sure.)

So we wound up with XML who’s source looked like
<some_url>/controller/action?foo=foo&amp;bar=bar

Is this correct or not? Not sure. Later in the program execution, this
XML gets converted to JSON, and the JSON winds up looking like:

some_url: ‘/controller/action?foo=foo&bar=bar’;

This part was right, that is a proper JSON translation of the XML passed
in, right, it un-escaped the XML properly, put it in JSON.

Then, this JSON gets delivered via JSONP to some javascript (external
javascript not generated by rails). The javascript gets that value in a
variable, containing ‘/controller/action?foo=foo&bar=bar’. So far so
good, it got the right value from the JSON delivered to it.

Now, if that had been in HTML source for an , I guess the
browser would have ‘un-escaped’ that before making the HTTP request. But
it wasn’t in HTML source, it was in a javascript variable. And when I
passed this variable to my javascript routine to load the URL
(AJAX-style), it ended up submitting a GET to the HTTP server that
looked like this:

GET /controller/action?foo=foo&bar=bar

That wound up being caught by a mongrel-fronted Rails app, which did NOT
turn that into query parameters foo => foo, bar => bar properly, it did
weird things with that GET request.

So. At what point did that code go wrong? At the moment, I’ve fixed my
XML-generating ERB to not escape urls generated by url_for. But I’m
not sure that’s right, it doesn’t feel right. Or is it my javascript
code that took a js variable containing
‘/controller/action?foo=foo&bar=bar’ and made a GET of that literal
string, instead of un-escaping it first, that went wrong? Or something
else?

I’m very confused. And, since ERB can be used to generate all kinds of
formats, it still seems to me that the documentation should mention this
feature, which would have gotten me to my present state of confusion
several hours earlier—ah, but when I go look at the most recent rdoc
ActionView url_for, I see that it was there all along, my fault for
missing it: “When called from a view, url_for returns an HTML escaped
url. If you need an unescaped url, pass :escape => false in the
options.” So good on the rdoc after all. Still somewhat confused as to
whether I should be “double escaping” it in the XML or not.

Jonathan