Trying to find Rails' parameter parsing code

Adam_Lassek · August 24, 2010, 9:19pm

I’m trying to understand how Rails parses the form parameters into a
nested Hash for the params object – can somebody point me in the
right direction?

It seems to have moved around a bit, especially with the 3.0 release,
so I’m not sure where to look.

Adam_Lassek · August 24, 2010, 9:30pm

I believe it’s actually part of Rack.

Check out the Rack::Request class, and I think you will find what
you’re looking for. Specifically starting at line 165

github.com

leahneukirchen/rack/blob/master/lib/rack/request.rb#L165


      
          # one of the media types presents in this list will not be eligible
          # for param parsing like soap attachments or generic multiparts
          PARSEABLE_DATA_MEDIA_TYPES = [
            'multipart/related',
            'multipart/mixed'
          ]
          
          
# Default ports depending on scheme. Used to decide whether or not
          # to include the port in a generated URI.
          DEFAULT_PORTS = { 'http' => 80, 'https' => 443, 'coffee' => 80 }
          
          
# Determine whether the request body contains form-data by checking
          # the request Content-Type for one of the media-types:
          # "application/x-www-form-urlencoded" or "multipart/form-data". The
          # list of form-data media types can be modified through the
          # +FORM_DATA_MEDIA_TYPES+ array.
          #
          # A request body is also assumed to contain form-data when no
          # Content-Type header is provided and the request_method is POST.
          def form_data?
            type = media_type

Adam_Lassek · August 24, 2010, 9:30pm

Adam Lassek wrote:

I’m trying to understand how Rails parses the form parameters into a
nested Hash for the params object – can somebody point me in the
right direction?

What are you trying to understand?

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Adam_Lassek · August 24, 2010, 9:58pm

I’m trying to understand how the forms microformat is parsed into a
nested Hash.

Adam_Lassek · August 24, 2010, 9:34pm

Oh, and also there is code in ActionController to pull the params out
of the request object and into the controller, which is why you can do
params[:value] in the controller.

github.com

rails/rails/blob/main/actionpack/lib/action_controller/metal.rb#L87


      
          # And then to route requests to your metal controller, you would add
          # something like this to <tt>config/routes.rb</tt>:
          #
          #   get 'hello', to: HelloController.action(:index)
          #
          # The +action+ method returns a valid Rack application for the \Rails
          # router to dispatch to.
          #
          # == \Rendering \Helpers
          #
          # +ActionController::Metal+ by default provides no utilities for rendering
          # views, partials, or other responses aside from explicitly calling of
          # <tt>response_body=</tt>, <tt>content_type=</tt>, and <tt>status=</tt>. To
          # add the render helpers you're used to having in a normal controller, you
          # can do the following:
          #
          #   class HelloController < ActionController::Metal
          #     include AbstractController::Rendering
          #     include ActionView::Layouts
          #     append_view_path "#{Rails.root}/app/views"
          #

Adam_Lassek · August 24, 2010, 10:04pm

I think that’s in Rack Utilities

github.com

leahneukirchen/rack/blob/master/lib/rack/utils.rb#L57


      
          # Unescapes a URI escaped string with +encoding+. +encoding+ will be the
          # target encoding of the string returned, and it defaults to UTF-8
          if defined?(::Encoding)
            def unescape(s, encoding = Encoding::UTF_8)
              URI.decode_www_form_component(s, encoding)
            end
          else
            def unescape(s, encoding = nil)
              URI.decode_www_form_component(s, encoding)
            end
          end
          module_function :unescape
          
          
DEFAULT_SEP = /[&;] */n
          
          
class << self
            attr_accessor :key_space_limit
            attr_accessor :multipart_part_limit
          end
          
          
# The default number of bytes to allow parameter keys to take up.

Adam_Lassek · August 24, 2010, 10:51pm

[Please quote when replying]

Adam Lassek wrote:

I’m trying to understand how the forms microformat is parsed into a
nested Hash.

What do Rails forms have to do with microformats? And are you trying to
understand the actual parsing code, or the result?

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Adam_Lassek · August 24, 2010, 11:01pm

The name attribute of form fields created in the following format:

person[address][street_address]

is parsed into a nested hash that looks like:

{ :person => { :address => { :street_address => “” } } }

That’s a microformat.

Adam_Lassek · August 25, 2010, 12:18am

http://microformats.org/about

They’re usually created with XML, but I think this still qualifies.
It’s taking an established standard and extending its functionality
through convention.

is there a better term for what this is?

Adam_Lassek · August 24, 2010, 11:06pm

Adam Lassek wrote:

The name attribute of form fields created in the following format:

person[address][street_address]

is parsed into a nested hash that looks like:

{ :person => { :address => { :street_address => “” } } }

That’s a microformat.

Not by any definition of that term that I’m familiar with. It’s just a
hash.

(For reference, the term “microformat” has only one meaning – see
Microformat - Wikipedia – and that wasn’t it. Using
standard terminology helps to reduce misunderstandings.)

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Adam_Lassek · August 25, 2010, 1:03am

Adam Lassek wrote:

About Microformats – Microformats

They’re usually created with XML, but I think this still qualifies.
It’s taking an established standard and extending its functionality
through convention.

…which is not what Rails is doing here. No similarity IMHO.

is there a better term for what this is?

Isn’t it just a fairly standard use of Ruby hashes?

Best,
–Â
Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Sent from my iPhone

Adam_Lassek · August 25, 2010, 4:31am

Adam Lassek wrote:

The parameter is a String, which is parsed into a Hash by way of a
convention.

There is no convention, really – it’s too trivial to call it that. The
HTTP parameters are already a list of name-value pairs, which is exactly
what a hash is.

That convention is what I’m referring to as a microformat.

Then you’re using the term idiosyncratically at best and inaccurately at
worst.

Best,
–Â
Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Sent from my iPhone

Adam_Lassek · August 25, 2010, 5:04am

I welcome an alternative term if that would be less confusing, but the
fact is this convention exists and it’s not a part of any
specification.

I chose microformat as a description because it’s a conventional
extension of an existing standard, HTTP parameters, to carry extra
information e.g. nesting relationships. This is exactly the sort of
thing microformats do with XHTML. So what if we’re embedding semantics
into a key/value string rather than XML? To me that’s just an
implementation detail.

So I’m curious where this convention came from, and what precisely are
the rules involved? Is there documentation for it?

Adam_Lassek · August 25, 2010, 4:21am

The parameter is a String, which is parsed into a Hash by way of a
convention. That convention is what I’m referring to as a microformat.

Adam_Lassek · August 25, 2010, 6:16am

Adam Lassek wrote:

I welcome an alternative term if that would be less confusing, but the
fact is this convention exists and it’s not a part of any
specification.

What convention? What do you think goes beyond the HTTP spec here?

I chose microformat as a description because it’s a conventional
extension of an existing standard, HTTP parameters, to carry extra
information e.g. nesting relationships. This is exactly the sort of
thing microformats do with XHTML. So what if we’re embedding semantics
into a key/value string rather than XML? To me that’s just an
implementation detail.

Maybe, but the word “microformat” only refers to specific ways of
abusing HTML/XML markup to add semantics. That’s not an “implementation
detail” – it’s a completely separate part of the problem domain.

Anyway, the HTML microformat technique is not even remotely related to
what we’re doing here. We are not “embedding semantics into a key/value
string”. You’re overthinking here – it really is just the parameters
as passed from the HTTP request. That’s all there is to it, I think.
If you don’t think that explains everything, please state what you don’t
understand and I’ll try to help.

So I’m curious where this convention came from, and what precisely are
the rules involved? Is there documentation for it?

What part don’t you understand? What do you think needs to be
documented? What are you really trying to find out here?

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Adam_Lassek · August 25, 2010, 2:00pm

Colin L. wrote:
[…]

I think I can see what the OP is getting at here, and do have some
sympathy with his point of view. The rails convention is that, for
example, an input field with the name
person[address][street_address]
is used to indicate that a hash of the form
{ :person => { :address => { :street_address => “” } } }
should be posted and can hence be used by the controller to build the
appropriate data. That is not part of the HTTP spec,

I suppose it isn’t. But it’s such a trivial extension that it hardly
bears a second glance. (I seem to recall that PHP does the same thing.)

it is an
overlying convention using field names to indicate the type and
structure of the data.

In a sense, that’s always what field names do.

In that sense it is akin to the microformat
concept.

That I do not agree with at all. I’m trying to see the similarity, and
– past the most trivial level – I do not.

Colin

Best,
–Â
Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Sent from my iPhone

Adam_Lassek · August 25, 2010, 2:37pm

On 25 August 2010 13:00, Marnen Laibow-Koser [email protected]
wrote:

That I do not agree with at all. Â I’m trying to see the similarity, and
– past the most trivial level – I do not.

From http://en.wikipedia.org/wiki/Microformat:
‘A microformat (sometimes abbreviated Î¼F) is a web-based approach to
semantic markup which seeks to re-use existing HTML/XHTML tags to
convey metadata[1] and other attributes in web pages and other
contexts that support (X)HTML, such as RSS.’

From http://en.wikipedia.org/wiki/Semantic:
‘Semantics is the study of meaning. It typically focuses on the
relation between signifiers, such as words, phrases, signs and
symbols, and what they stand for.’
So I interpret ‘semantic markup’ to be markup that indicates in some
sense the meaning of the data.

From http://en.wikipedia.org/wiki/Metadata:
‘Metadata is loosely defined as data about data. Metadata is a concept
that applies mainly to electronically archived or presented data and
is used to describe the a) definition, b) structure and c)
administration of data files with all contents in context to ease the
use of the captured and archived data for further use.’

So the example where a field name of person[address][street_address]
indicates that field content is the street address part of an address
for a person seems to me could arguably fall into the definition of
microformat as it is markup that provides some information about both
the meaning and the structure of the data.

Colin

Adam_Lassek · August 25, 2010, 7:19pm

Using “person[address][street_address]” to denote a nested Hash
structure is semantic, and it is not part of the spec. you might think
it trivial, but not all frameworks behave in this way. So it is useful
to note that this convention exists and explain its rules. Agreed?
That’s all I’m asking after here.

I’m sorry if my choice of the word “microformat” offends you so badly
in this context, but then you have yet to offer an alternative. And
other people on this list have understood what I meant. So maybe it’s
not worth arguing about.

Adam_Lassek · August 25, 2010, 9:50am

On 25 August 2010 05:16, Marnen Laibow-Koser [email protected]
wrote:

Adam Lassek wrote:

I welcome an alternative term if that would be less confusing, but the
fact is this convention exists and it’s not a part of any
specification.

What convention? What do you think goes beyond the HTTP spec here?

I think I can see what the OP is getting at here, and do have some
sympathy with his point of view. The rails convention is that, for
example, an input field with the name
person[address][street_address]
is used to indicate that a hash of the form
{ :person => { :address => { :street_address => “” } } }
should be posted and can hence be used by the controller to build the
appropriate data. That is not part of the HTTP spec, it is an
overlying convention using field names to indicate the type and
structure of the data. In that sense it is akin to the microformat
concept.

Colin

Adam_Lassek · August 25, 2010, 7:26pm

Adam Lassek wrote:

Using “person[address][street_address]” to denote a nested Hash
structure is semantic,

In what respect? It’s abusing a particular form of HTTP parameter name,
true, but that’s about it.

Rails’ reservation of HTTP parameter names “controller” and “action”
might be considered semantic, but I don’t think the brackets are.

and it is not part of the spec. you might think
it trivial, but not all frameworks behave in this way. So it is useful
to note that this convention exists and explain its rules. Agreed?
That’s all I’m asking after here.

Very simple. The brackets act like hash subscripts. Hashes are
constructed so to make this possible. That’s all there is to it AFAIK.

I’m sorry if my choice of the word “microformat” offends you so badly
in this context,

It doesn’t offend me – it’s just inaccurate.

but then you have yet to offer an alternative.

I don’t think there is a useful word other than “mapping” or something
like that.

And
other people on this list have understood what I meant. So maybe it’s
not worth arguing about.

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]