Forum: Ruby Using HTTParty to post a very large set of name/value pairs

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
A24e589de07cb769a71f28aead957a48?d=identicon&s=25 Duane Morin (dmorin)
on 2009-03-27 14:26
I've got a pretty large text file (a few thousand lines on average)
consisting of all key/value pairs.  If I use wget to run the post from a
command line, it seems to work fine.

However, I'm trying to do it in Ruby, and I've already got the rest of
the API working with HTTParty.  So if I've got the data in a big
concatenated string:

data="foo=bar\nbaz=quux\nhamlet=ophelia\nclaudius=gertrude ..."

It seems the only way I can wrap it for sending is like this:

    post('/processor.php', :query => {:data=>data})

But after a certain size for data, I start getting "414 Request-URI Too
Large" errors.

Since it works as a direct post from wget, the best I can figure is that
the difference lies in how exactly HTTParty is formatting it for
sending.  Is there a good way for me to see what the actual outgoing
request looks like coming out of that post?  I have a simple test jig on
the other side (processor.php) that dumps out the $_POST variables it
gets, but when I get the 414 error it never reaches my tester.

At some point I would scrap it and switch over to something more
upload/attachment based, but the fact that it works with wget bothers
me, and the existing system is already architected to work that way and
I hate to start tearing stuff apart.
A24e589de07cb769a71f28aead957a48?d=identicon&s=25 Duane Morin (dmorin)
on 2009-03-27 14:36
Aha, may have solved my own problem.  Looks like using the :body option
instead of :query might be just what I need!

Sorry for the interruption.

D

Duane Morin wrote:
> I've got a pretty large text file (a few thousand lines on average)
> consisting of all key/value pairs.  If I use wget to run the post from a
> command line, it seems to work fine.
>
> However, I'm trying to do it in Ruby, and I've already got the rest of
> the API working with HTTParty.  So if I've got the data in a big
> concatenated string:
>
> data="foo=bar\nbaz=quux\nhamlet=ophelia\nclaudius=gertrude ..."
>
> It seems the only way I can wrap it for sending is like this:
>
>     post('/processor.php', :query => {:data=>data})
>
> But after a certain size for data, I start getting "414 Request-URI Too
> Large" errors.
>
> Since it works as a direct post from wget, the best I can figure is that
> the difference lies in how exactly HTTParty is formatting it for
> sending.  Is there a good way for me to see what the actual outgoing
> request looks like coming out of that post?  I have a simple test jig on
> the other side (processor.php) that dumps out the $_POST variables it
> gets, but when I get the 414 error it never reaches my tester.
>
> At some point I would scrap it and switch over to something more
> upload/attachment based, but the fact that it works with wget bothers
> me, and the existing system is already architected to work that way and
> I hate to start tearing stuff apart.
This topic is locked and can not be replied to.