Replace REXML with Hpricot

Rails
1

Hi,

is there any way to replace REXML with Hpricot for rendering XML? I am
using Jruby so cant use libxml, and need to significantly increase the
speed
of XML rendering.

Thanks
Adam

2

AD wrote:

is there any way to replace REXML with Hpricot for rendering XML? I am
using Jruby so cant use libxml, and need to significantly increase the
speed of XML rendering.

What’s wrong with Builder::XmlMarkup (iirc)?

Hpricot uses a C module to get fast - like libxml. So if you can’t use
one in
Jruby…

3

I think I actually meant XML::Builder, I was under the initial
impression
this was actually a REXML Library.
At any rate my performance metrics show that 60-70% is done via
“Rendering”
b/c of some complicated XML generation so I am looking for some
optimization
mechanisms here. HPricot is faster than Builder and REXML and is
supported
in JRuby (running it now).

So basically I am looking to replace XML::Builder with something faster
,
that is not libXML (like Hpricot)

Adam

4

On Jun 27, 3:08 am, AD [email protected] wrote:

I think I actually meant XML::Builder, I was under the initial impression
this was actually a REXML Library.
At any rate my performance metrics show that 60-70% is done via “Rendering”
b/c of some complicated XML generation so I am looking for some optimization
mechanisms here. HPricot is faster than Builder and REXML and is supported
in JRuby (running it now).

So basically I am looking to replace XML::Builder with something faster ,
that is not libXML (like Hpricot)

There is a jruby version of hpricot. Builder and Rexml are unrelated.
The only bit I’ve ever found to be slow about builder is that it tries
to be a bit too clever with its escaping and tries to work out all
sorts of stuff that I didn’t need. Overriding the _escape method to
just replace &, <, > sped things up for me

Fred

5

well I am just noticing when the actions get rendered, its always like
(Rendering (78%)) and some of the XML rendering just seems slow in
relation
to the time for the DB query.

Do you have any profile recommendations besides ruby-prof ?

6

AD wrote:

So basically I am looking to replace XML::Builder with something faster
, that is not libXML (like Hpricot)

Have you time-tested and profiled your code & tests to see what’s fast?
it’s
usually a surprise…

7

On 27 Jun 2008, at 13:50, AD wrote:

here is some real world output… Any feedback is welcome, notice the
very high render time:

Fixnum.xchr is builder’s escaping stuff. See my previous tip for
possible ways out.

Fred

8

I believe you are referencing this method

def _escape(text)
text.to_xs
end

How do I go about overriding this ?

On Fri, Jun 27, 2008 at 8:54 AM, Frederick C. <

9

here is some real world output… Any feedback is welcome, notice the
very
high render time:

Rendering channels/index
Completed in 3.38310 (0 reqs/sec) | Rendering: 1.38117 (40%) | DB:
0.48588
(14%) | 200 OK [http://dev1.domain.com/users/1/2008/05/28/index.xml]
[http://dev1.domain.com/users/1/2008/05/28/index.xml]

Thread ID: 3067917240
Total: 2.920000

%self total self wait child calls name
5.82 0.42 0.17 0.00 0.25 18281 Fixnum#xchr
4.45 0.14 0.13 0.00 0.01 79 Array#select
4.11 0.56 0.12 0.00 0.44 2288 Array#map
3.08 0.10 0.09 0.00 0.01 29
ActiveRecord::Associations::ClassMethods#collection_accessor_methods
2.74 0.11 0.08 0.00 0.03 54266 Kernel#===
1.71 0.16 0.05 0.00 0.11 248 Hash#each
1.71 0.08 0.05 0.00 0.03 54327 Hash#[]
1.37 0.06 0.04 0.00 0.02 288
Module#define_method
1.37 0.04 0.04 0.00 0.00 15
Mysql::Result#each_hash
1.37 0.04 0.04 0.00 0.00 108 Array#-
1.03 0.03 0.03 0.00 0.00 39128 Hash#default
1.03 0.03 0.03 0.00 0.00 324 Module#constants
1.03 0.03 0.03 0.00 0.00 56670 Fixnum#==

Thread ID: 3067917860
Total: 0.770000

10

On 27 Jun 2008, at 14:25, Frederick C. wrote:

See also http://bogomips.org/fast_xs/ which is a c implementation of
the escaping thing (haven’t tried this, but is probably the fastest
of all).

to expand on that, just gem install fast_xs. It’s faster than what I
had previously

Fred

11

On 27 Jun 2008, at 14:09, AD wrote:

I believe you are referencing this method

def _escape(text)
text.to_xs
end

How do I go about overriding this ?

same way as you override any other method

class Builder::XmlMarkup
def _escape(text)
result = text.dup
result.gsub!(“&”, “&”)
result.gsub!(“<”, “<”)
result.gsub!(“>”, “>”)
end
end

in my case what i did was

xml = Builder::XmlMarkup.new(:target => $stdout, :indent => 2)
xml.instruct!

def xml.output(string)
result = string.dup
result.gsub!(“&”, “&”)
result.gsub!(“<”, “<”)
result.gsub!(“>”, “>”)
self << result
end

and then call output rather than text whenever i wanted to output stuff.
There was a thread a while back on one of the ruby list about the
fastest way to do this - do you do it in one pass fixing up each of
the 3 possible substitutions in one go, or in 3 passes like i did, and
concluded that the above version was the fastest. It might be
different if you had really long strings.

See also http://bogomips.org/fast_xs/ which is a c implementation of
the escaping thing (haven’t tried this, but is probably the fastest of
all).

Fred

12

yea fast_xs looks better, but right now its native C only so no Jruby
support.

Is it possible to generated XML without using builder ? If the XML
structure is not too complicated, and I am OK sacrificing some of the
validation/substitution, could I just @results.each { |r| } and print
some
XML ?

Thx
Adam

On Fri, Jun 27, 2008 at 10:05 AM, Frederick C. <

13

On 27 Jun 2008, at 15:09, AD wrote:

yea fast_xs looks better, but right now its native C only so no
Jruby support.

Is it possible to generated XML without using builder ? If the XML
structure is not too complicated, and I am OK sacrificing some of
the validation/substitution, could I just @results.each { |r| } and
print some XML ?

Course you can, it’s just text after all! I got a significant speed
boost overwriting _escape so that might be enough for you.

Fred

14

On 27 Jun 2008, at 15:40, AD wrote:

Did you just drop this in application.rb ?

class Builder::XmlMarkup
def _escape(text)
result = text.dup
result.gsub!("&", “&”)
result.gsub!("<", “<”)
result.gsub!(">", “>”)
end
end

just in a script i was writing. (but it should be ok in
application.rb), with the exception of one line which i left out :slight_smile:

it should be
def _escape(text)
result = text.dup
result.gsub!("&", “&”)
result.gsub!("<", “<”)
result.gsub!(">", “>”)
result
end

jruby is causing you to miss out on the fast_xs fun, but you might be
able to rewrite that method in java, with similar effects.

Fred

15

Did you just drop this in application.rb ?

class Builder::XmlMarkup
def _escape(text)
result = text.dup
result.gsub!("&", “&”)
result.gsub!("<", “<”)
result.gsub!(">", “>”)
end
end

On Fri, Jun 27, 2008 at 10:30 AM, Frederick C. <

16

Frederick,

Thanks for the help but i am getting an error. I dropped this outside
the
ApplicationController but is throwing an error

class Builder::XmlMarkup
def _escape(text)
result = text.dup
result.gsub!("&", “&”)
result.gsub!("<", “<”)
result.gsub!(">", “>”)
result
end
end

ActionView::TemplateError (You have a nil object when you didn’t expect
it!
The error occurred while evaluating nil.gsub) on line #1 of
channels/system.xml.builder:
1: xml.instruct!
2: xml.response {
3: xml.result {
4: xml.status “Success”

On Fri, Jun 27, 2008 at 10:54 AM, Frederick C. <

17

do you have it in application.rb ?

On Fri, Jun 27, 2008 at 1:10 PM, Frederick C. <

18

On 27 Jun 2008, at 17:49, AD wrote:

result.gsub!(">", "&gt;")
result

end
end

Weird, works well enough for me. the error you got looks like what you
would get with the version without the result on the last line.

Fred

19

On 27 Jun 2008, at 18:50, AD wrote:

I also noticed that the _escape function is in xmlbase.rb not
xmlmarkup.rb. Not sure if that is what is breaking here…

XmlMarkup inherits from XmlBase so it shouldn’t make any difference. I
didn’t have it in application.rb but I doubt that would make a
difference.

Fred

20

updating this thread for everyone’s sake. I had to drop it into an
initializers file in config/initializers and it worked.
however performance is not much better 36% of the time was just on
gsub.

Thread ID: 3063488800
Total: 2.750000

%self total self wait child calls name
36.36 1.00 1.00 0.00 0.00 38508 String#gsub!
16.36 1.62 0.45 0.00 1.17 1008
Builder::XmlBase#method_missing-5
3.64 0.10 0.10 0.00 0.00 1282
Class::File#expand_path
2.55 1.15 0.07 0.00 1.08 12836
Builder::XmlBase#_escape
2.18 0.06 0.06 0.00 0.00 16
Mysql::Result#each_hash
1.82 2.14 0.05 0.00 2.09 1097 Array#each-1
1.82 0.05 0.05 0.00 0.00 12836
String#initialize_copy
1.82 0.07 0.05 0.00 0.02 1015
ActiveRecord::AttributeMethods#read_attribute
1.45 0.04 0.04 0.00 0.00 31114 Fixnum#==
1.45 0.04 0.04 0.00 0.00 14171 Symbol#to_s
1.45 0.04 0.04 0.00 0.00 2159
Hash#initialize_copy
1.45 0.04 0.04 0.00 0.00 4432
Kernel#instance_variable_set
1.09 0.20 0.03 0.00 0.17 1008
Builder::XmlBase#method_missing-6
1.09 0.03 0.03 0.00 0.00 8262
Builder::XmlMarkup#_text
1.09 0.42 0.03 0.00 0.39 1126 Hash#each
1.09 2.62 0.03 0.00 2.59 102 Array#each
1.09 0.08 0.03 0.00 0.05 2154
Class::ActiveRecord::Base#instantiate

On Fri, Jun 27, 2008 at 2:05 PM, Frederick C. <