# REXML document creation speed

**URL:** <https://www.ruby-forum.com/t/rexml-document-creation-speed/129512>\
**Category:** Ruby\
**Created:** [February 19, 2008, 6:30am UTC](https://www.ruby-forum.com/t/rexml-document-creation-speed/129512 "2008-02-19T06:30:58Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [February 19, 2008, 6:30am UTC](https://www.ruby-forum.com/t/rexml-document-creation-speed/129512/1 "2008-02-19T06:30:58Z")

</div>

Hi,

Please excuse my ignorance, but I’m new to this.

I’ve written a script that downloads historical stock quotes as .csv,  
parses it, then writes out an XML doc that I can then use elsewhere.

It works as designed, but it’s _dog slow_ and I don’t see why. It can  
take over 5 minutes to run, and nearly all of that time is writing the  
XML docs (I’ve tested it running the script with the XML document  
creation lines commented out and it takes only seconds).

The script is below:

require ‘rubygems’  
require ‘net/http’  
require ‘FasterCSV’  
require ‘rexml/document’  
include REXML  
puts “Start #{Time.now()}”  
symbols = Array.new  
xml\_symbols\_doc = Document.new(File.new(“symbols.xml”))  
for i in 1…xml\_symbols\_doc.root.elements.size  
symbols[i] = xml\_symbols\_doc.root.elements[i].get\_text.value  
end  
for i in 1…(symbols.length - 1)  
quote\_source\_url = “[http://ichart.finance.yahoo.com/table.csv](http://ichart.finance.yahoo.com/table.csv)?  
s=#{URI.encode(symbols[i])}”  
quote\_response =  
Net::HTTP.get\_response(URI.parse(quote\_source\_url))  
csv\_quotes = FasterCSV.parse(quote\_response.body, {:headers =\>  
true, :header\_converters =\> :symbol})  
xml\_quotes = Document.new  
xml\_quotes \<\< XMLDecl.new  
xml\_quotes.add\_element(“quotes”, {“symbol” =\> “#{symbols[i]}”})  
for j in 0…(csv\_quotes.length - 1)  
quote = Element.new(“quote”)  
for k in 0…(csv\_quotes.headers.length - 1)  
quote.add\_element(“#{csv\_quotes.headers()[k]}”).text =  
“#{csv\_quotes[j][k]}”  
end  
xml\_quotes.root \<\< quote  
end  
xml\_quotes\_output\_file = File.new(“#{symbols[i]}.xml”, “w+”)  
xml\_quotes.write(xml\_quotes\_output\_file, 3)  
puts “#{symbols[i]} OK. File here: #{xml\_quotes\_output\_file.path}”  
end  
puts “End #{Time.now()}”

Any suggestions as to how I can make this script run (much) faster are  
greatly appreciated!

Thanks for your help!  
Ted

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [February 19, 2008, 7:55am UTC](https://www.ruby-forum.com/t/rexml-document-creation-speed/129512/2 "2008-02-19T07:55:54Z")

</div>

On 19.02.2008 06:29, removed\_email\_address@domain.invalid wrote:

> creation lines commented out and it takes only seconds).  
> REXML is not particularly fast but what makes you sure that it’s in  
> REXML an not in the way you prepare the data? Did you test with “-r  
> profile”? Did you notice that you have three nested levels of loops -  
> that may well be the source of the slowness.

A few stylistic remarks: you should use the block form of File.open in  
order to ensure proper and timely cleanup.

You can make your live easier by using Ruby’s iterating idioms and not  
for with array indexes.

Note also that there’s XPath expressions that you can use for iterating  
an XML document.

> The script is below:

Kind regards

robert

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [February 19, 2008, 9:00am UTC](https://www.ruby-forum.com/t/rexml-document-creation-speed/129512/3 "2008-02-19T09:00:50Z")

</div>

On Feb 19, 7:51 am, Robert K. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > take over 5 minutes to run, and nearly all of that time is writing the
> 
> Kind regards
> 
> ```
> robert
> 
> ```

From my personal experience the Hpricot was much faster then REXML.  
As Robert already mentioned iterate trough collections.  
The first thing you should do is to add some metrics to find what the  
slowest part of your program is. Without measurement you can not  
determine if you are on the right track of improvement. Not just from  
start to finish but add some check points.  
As you have a list of symbols to download you should make multi  
threaded approach to it.

You guess it right - there is a lot of space to improve this  
application but measure the performance first than act and measure the  
improvement