Re: Nonlinear scaling - could symbols help solve this?

M. Edward (Ed) Borasky wrote:

Thanks for the response.

You are right that my CSV file is very much like a database table (used to load database tables). However, my XML document is not so tree-like and actually fits pretty > well into a hash of hashes of arrays.
A non-tree-like XML document? Are you required by the downstream user
to create XML?

Yes, the XML document and its structure is part of a legacy system for
importing data into a database. Crazy hey - we convert a database like
format into a hash of hashes of arrays and then the legacy system reads
the data out of the XML document and inserts it into the database?!?

The initial csv file is a format that I have created to import directly
into the database but unfortunately not all customers are upgrading to
the latest system so we end up having to be backward compatible… the
joys of customer support.

Hmmm … this might make more sense to do with Rails than with pure Ruby
… you could import your CSV into a SQLite database and export the XML
from there. Check out “Enterprise Integration With Ruby” and “Rails
Recipes” for more ideas. SQLite is a very nice little tool – all the
comforts of an RDBMS with very little configuration and blinding speed
on small databases.

Jeremy H. wrote:

Yes, the XML document and its structure is part of a legacy system for importing data into a database. Crazy hey - we convert a database like format into a hash of hashes of arrays and then the legacy system reads the data out of the XML document and inserts it into the database?!?

The initial csv file is a format that I have created to import directly into the database but unfortunately not all customers are upgrading to the latest system so we end up having to be backward compatible… the joys of customer support.


M. Edward (Ed) Borasky

On 5/17/06, M. Edward (Ed) Borasky [email protected] wrote:

Hmmm … this might make more sense to do with Rails than with pure Ruby
… you could import your CSV into a SQLite database and export the XML
from there. Check out “Enterprise Integration With Ruby” and “Rails
Recipes” for more ideas. SQLite is a very nice little tool – all the
comforts of an RDBMS with very little configuration and blinding speed
on small databases.

You can also try my xmlcodec package:

http://rubyforge.org/projects/xmlcodec/

Defining an importer/exporter for a XML format is very easy, all you
have to do is write some simple ruby classes. After that you’ll get
API’s to import/export documents. For your case you can use the
partial export API to create the XML as you go without representing it
all in memory.

Greetings,

Pedro.

On 5/17/06, M. Edward (Ed) Borasky [email protected] wrote:

Hmmm … this might make more sense to do with Rails than with pure Ruby
… you could import your CSV into a SQLite database and export the XML
from there.

Well, with ActiveRecord[1], anyways. I don’t see the need for a full
Rails stack with controllers and views, but a script that leverages
ActiveRecord to read the data out of an SQLite DB can be useful.

But really, if it’s in CSV already, and there’s not any/many
relationships, just use FasterCSV[2] to read the CSV file, then take
advantage of something like Builder[3] or xmlcodec[4] to output the
resulting XML.

Jacob F.

[1] http://api.rubyonrails.org/
[2] http://fastercsv.rubyforge.org/
[3]
Creating XML with Ruby and Builder
[4] http://rubyforge.org/projects/xmlcodec/