Forum: Ruby Making a XML::Parser App Multithreaded

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Raymond O. (Guest)
on 2007-07-31 21:21
I wrote an application to parse a large xml file (over 20GB) using
XML::Parser.  It runs on a multicore system, however, and takes forever
since it uses only one core.  Maybe this is more of a SAX question, but
is there anyway to rewrite my app so that it takes advantage of multiple
cores?

Thanks
Robert K. (Guest)
on 2007-08-02 02:06
(Received via mailing list)
On 31.07.2007 19:21, Raymond O'Connor wrote:
> I wrote an application to parse a large xml file (over 20GB) using
> XML::Parser.  It runs on a multicore system, however, and takes forever
> since it uses only one core.  Maybe this is more of a SAX question, but
> is there anyway to rewrite my app so that it takes advantage of multiple
> cores?

Not the parsing part, because there is no way that you can parallelize
reading an XML file and stream parsing it in a reasonable way (you would
have to stuff all parse events into a queue anyway to make sure they are
processed in the correct order).

For the rest it depends on what you do with parse results.  For that we
would need more detail.

Kind regards

  robert
Raymond O. (Guest)
on 2007-08-03 03:37
Robert K. wrote:
>
> Not the parsing part, because there is no way that you can parallelize
> reading an XML file and stream parsing it in a reasonable way (you would
> have to stuff all parse events into a queue anyway to make sure they are
> processed in the correct order).
>
> For the rest it depends on what you do with parse results.  For that we
> would need more detail.
>
> Kind regards
>
>   robert

It's pretty much the parsing that I need to speed up, so it looks like
I'll have to live with it for now.  Thank you very much for the
response.
This topic is locked and can not be replied to.