Making a XML::Parser App Multithreaded

Ruby
1

I wrote an application to parse a large xml file (over 20GB) using
XML::Parser. It runs on a multicore system, however, and takes forever
since it uses only one core. Maybe this is more of a SAX question, but
is there anyway to rewrite my app so that it takes advantage of multiple
cores?

Thanks

2

On 31.07.2007 19:21, Raymond O’Connor wrote:

I wrote an application to parse a large xml file (over 20GB) using
XML::Parser. It runs on a multicore system, however, and takes forever
since it uses only one core. Maybe this is more of a SAX question, but
is there anyway to rewrite my app so that it takes advantage of multiple
cores?

Not the parsing part, because there is no way that you can parallelize
reading an XML file and stream parsing it in a reasonable way (you would
have to stuff all parse events into a queue anyway to make sure they are
processed in the correct order).

For the rest it depends on what you do with parse results. For that we
would need more detail.

Kind regards

robert

3

Robert K. wrote:

Not the parsing part, because there is no way that you can parallelize
reading an XML file and stream parsing it in a reasonable way (you would
have to stuff all parse events into a queue anyway to make sure they are
processed in the correct order).

For the rest it depends on what you do with parse results. For that we
would need more detail.

Kind regards

robert

It’s pretty much the parsing that I need to speed up, so it looks like
I’ll have to live with it for now. Thank you very much for the
response.