Michael S. wrote:
On Sunday 10 May 2009, Jian L. wrote:
so how to do inserts in bulk? by using
Phrase.connection.execute(“insert into phrases(s,frequency,length)
values(’#{phrase}’,#{frequencies[phrase]},#{lengths[phrase]})”)You might be using the wrong tool and you might be reinventing existing
solutions. Did you have a look at existing text mining tools/frameworks
in Ruby as well as other languages?
i want to do this project and also at the same time let me have a chance
to learn to use the ActiveRecord…
perhaps i can use some kind of “traverse” tool to walk the internet
too…
something that can start at a page and then walk down all page by a
level of 1 or 2 or any number.
it won’t be so hard to write except sometimes the “baseurl” is used and
it is more processing than using absolute url.