I have attached a ruby program built to get the xml data out of files
and display the text back to me on the screen. This program was built
from a few different sources, as I’m still new with the Ruby language.
But I have monkeyed around with it enough and have got it working how I
want it.
What I’m trying to do is get this information into a database so I can
search it using a web interface (through Rails of course).
Here is a sample of the data that is spit out to the screen when I run
the program:
What is the best way to go about getting this info loaded into a
database? Should I created the table and fields first and then put all
of the info in a yaml file and seed the database (I might have some
questions on how to do that)? New xml files are getting dumped to these
folders all the time, so I’m looking for a nice way to get this info
into the database without too much hassle.
I’m not sure how to make this data available to rails. Any help on this
–gems that could do it, articles describing what I need (I’ve googled
this subject extensively), or any hints, tips, etc… – would be
GREATLY appreciated.
I have attached a ruby program built to get the xml data out of files
and display the text back to me on the screen. This program was built
from a few different sources, as I’m still new with the Ruby language.
But I have monkeyed around with it enough and have got it working how I
want it.
What I’m trying to do is get this information into a database so I can
search it using a web interface (through Rails of course).
[…]
Another idea: if this is to be read-only, you could use the XML files
themselves as the DB, either through ActiveModel or (probably better)
something like eXistDB.
I have attached a ruby program built to get the xml data out of files
and display the text back to me on the screen. This program was built
from a few different sources, as I’m still new with the Ruby language.
But I have monkeyed around with it enough and have got it working how I
want it.
What I’m trying to do is get this information into a database so I can
search it using a web interface (through Rails of course).
Here is a sample of the data that is spit out to the screen when I run
the program:
What is the best way to go about getting this info loaded into a
database? Should I created the table and fields first and then put all
of the info in a yaml file and seed the database (I might have some
questions on how to do that)? New xml files are getting dumped to these
folders all the time, so I’m looking for a nice way to get this info
into the database without too much hassle.
Then you don’t want to seed the database – seeding is only for
relatively static data (such as state and country names) that needs to
be loaded before you ever run the application. It sounds to me like you
simply want to read the XML files and build INSERT queries. And unless
I’m badly mistaken, you already know how to do that.
You may need a cron job or some such to parse new files as they come in.
I’m not sure how to make this data available to rails.
What’s the problem, exactly? There’s nothing here that you don’t know.
Then you don’t want to seed the database – seeding is only for
relatively static data (such as state and country names) that needs to
be loaded before you ever run the application. It sounds to me like you
simply want to read the XML files and build INSERT queries. And unless
I’m badly mistaken, you already know how to do that.
Hi Marnen. Thank you for the quick reply. Honestly, I’ve never worked
with xml files like this. I grabbed the code for the ruby program from a
lot of different sources and kind of crammed it all together to see what
would work best. I messed up a few times and destroyed some xml files in
to process (thank goodness for backups!). I can output all the xml at
text to the screen… But that’s about the extent of my abilities. I’m
not sure how to turn them into objects to insert into a database.
You may need a cron job or some such to parse new files as they come in.
Thanks for the tip! I’ll look into doing that.
What’s the problem, exactly? There’s nothing here that you don’t know.
Thanks for the confidence, but I don’t have the same confidence in
myself. There’s a lot I don’t know about Ruby and Rails, and I’m still
trying to learn. I can create a database in Rails. I know how to do
that. But I don’t know how to get this text from the xml files turned
into objects to send to the database. Any help on how to do that would
be awesome.
Another idea: if this is to be read-only, you could use the XML files
themselves as the DB, either through ActiveModel or (probably better)
something like eXistDB.
That would be awesome! Any links on where to go to figure out how to do
that? I’ll look into eXistDB.
Solr is another option, this will make it searchable and you will be
able to retrieve the source xml.
It’s not really an XML database but if your data is essentially not
relational then it will do a great job. You could of course put the
relational aspects in a database but keep the body of the data in
solr.
Sweet! That’s what I need. I need it searchable. This is for work. It’s
nothing time-sensitive, so I have some time to work on it. But I figured
I would get cracking on some type of solution.
All of the ideas so far have been great. I’ll post back with any more
issues, and if people could keep the suggestions flowing, I would be
forever grateful!
Solr is another option, this will make it searchable and you will be
able to retrieve the source xml.
It’s not really an XML database but if your data is essentially not
relational then it will do a great job. You could of course put the
relational aspects in a database but keep the body of the data in
solr.
What’s the problem, exactly? There’s nothing here that you don’t know.
Thanks for the confidence, but I don’t have the same confidence in
myself. There’s a lot I don’t know about Ruby and Rails, and I’m still
trying to learn. I can create a database in Rails. I know how to do
that. But I don’t know how to get this text from the xml files turned
into objects to send to the database. Any help on how to do that would
be awesome.
What do you really need help with here? You know how to parse the XML
files to retrieve the data. You know how to insert records into a DB.
Now you just need to connect the two.
Stop convincing yourself you’re missing something, write some tests, and
start coding.
