Fwd: Question jruby and java heap memory message

Ruby JRuby
1

Hi all,

I’m new using jruby and celerity, I was trying to run my script in
background with 100 user threads. If I ran the script with 10 users I
will
get the results that I want, but when I tried to execute the same script
with more than 10 threads I’m getting the following error, can someone
help
me with this error.

I running the script in a macbook pro with 2gb

/Users/moisessiles/jruby-1.4.0/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
require': IO error -- /Users/moisessiles/jruby-1.4.0/lib/ruby/gems/1.8/gems/celerity-0.7.9/lib/celerity/htmlunit/commons-io-1.4 (LoadError) from /Users/moisessiles/jruby-1.4.0/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:inrequire’
from
/Users/moisessiles/jruby-1.4.0/lib/ruby/gems/1.8/gems/celerity-0.7.9/lib/celerity/htmlunit.rb:3
from
/Users/moisessiles/jruby-1.4.0/lib/ruby/gems/1.8/gems/celerity-0.7.9/lib/celerity/htmlunit.rb:3:in
each' from /Users/moisessiles/jruby-1.4.0/lib/ruby/gems/1.8/gems/celerity-0.7.9/lib/celerity/htmlunit.rb:3 from /Users/moisessiles/jruby-1.4.0/lib/ruby/gems/1.8/gems/celerity-0.7.9/lib/celerity/htmlunit.rb:31:inrequire’
from
/Users/moisessiles/jruby-1.4.0/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
require' from /Users/moisessiles/jruby-1.4.0/lib/ruby/gems/1.8/gems/celerity-0.7.9/lib/celerity.rb:36 from /Users/moisessiles/jruby-1.4.0/lib/ruby/gems/1.8/gems/celerity-0.7.9/lib/celerity.rb:36:inrequire’
from
/Users/moisessiles/jruby-1.4.0/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in
`require’
from /Users/moisessiles/Desktop/Ruby and Watir/busqueda_vuelos.rb:3
Thread 10
Error: Your application used more memory than the safety cap of 500m.
Specify -J-Xmx####m to increase it (#### = cap size in MB).
Specify -w for full OutOfMemoryError stack trace

2

Try increasing the max heap size as suggested:

Error: Your application used more memory than the safety cap of 500m.
Specify -J-Xmx####m to increase it (#### = cap size in MB).
Specify -w for full OutOfMemoryError stack trace

On Wed, Feb 24, 2010 at 3:28 PM, Moises S. [email protected]
wrote:

/Users/moisessiles/jruby-1.4.0/lib/ruby/gems/1.8/gems/celerity-0.7.9/lib/celerity/htmlunit.rb:3:in
/Users/moisessiles/jruby-1.4.0/lib/ruby/gems/1.8/gems/celerity-0.7.9/lib/celerity.rb:36
Specify -w for full OutOfMemoryError stack trace

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
3

ok I will try, sorry I’m new with this, what should be the correct
command
to do that? :S

4

If you want 1024m of heap:

jruby -J-Xmx1024m my_script.rb

-Justin

On Wed, Feb 24, 2010 at 3:36 PM, Moises S. [email protected]
wrote:

get the results that I want, but when I tried to execute the same script

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
5

mmmmm I set the value to 1024 and was ok, but after few minutes to run
my
test I got the error

org/apache/commons/io/output/ByteArrayOutputStream.java:271:in
toByteArray': java.lang.OutOfMemoryError: Java heap space (NativeException) from org/apache/commons/io/IOUtils.java:219:intoByteArray’
from com/gargoylesoftware/htmlunit/WebResponseData.java:134:in
getBody' from com/gargoylesoftware/htmlunit/WebResponseData.java:87:in’
from com/gargoylesoftware/htmlunit/HttpWebConnection.java:452:in
newWebResponseDataInstance' from com/gargoylesoftware/htmlunit/HttpWebConnection.java:432:inmakeWebResponse’
from com/gargoylesoftware/htmlunit/HttpWebConnection.java:104:in
getResponse' from com/gargoylesoftware/htmlunit/WebClient.java:1407:inloadWebResponseFromWebConnection’
from com/gargoylesoftware/htmlunit/WebClient.java:1340:in
loadWebResponse' ... 24 levels... from /Users/moisessiles/jruby-1.4.0/lib/ruby/gems/1.8/gems/celerity-0.7.9/lib/celerity/container.rb:761:inrescue_status_code_exception’
from
/Users/moisessiles/jruby-1.4.0/lib/ruby/gems/1.8/gems/celerity-0.7.9/lib/celerity/browser.rb:99:in
goto' from /Users/moisessiles/Desktop/Ruby and Watir/busqueda_vuelos.rb:11 net/sourceforge/htmlunit/corejs/javascript/Parser.java:3258:increateStringLiteral’: java.lang.OutOfMemoryError: Java heap space
(NativeException)
from net/sourceforge/htmlunit/corejs/javascript/Parser.java:2837:in
primaryExpr' from net/sourceforge/htmlunit/corejs/javascript/Parser.java:2466:inmemberExpr’
from net/sourceforge/htmlunit/corejs/javascript/Parser.java:2376:in
unaryExpr' from net/sourceforge/htmlunit/corejs/javascript/Parser.java:2299:inmulExpr’
from net/sourceforge/htmlunit/corejs/javascript/Parser.java:2281:in
addExpr' from net/sourceforge/htmlunit/corejs/javascript/Parser.java:2260:inshiftExpr’
from net/sourceforge/htmlunit/corejs/javascript/Parser.java:2233:in
relExpr' from net/sourceforge/htmlunit/corejs/javascript/Parser.java:2203:ineqExpr’
… 24 levels…
from
/Users/moisessiles/jruby-1.4.0/lib/ruby/gems/1.8/gems/celerity-0.7.9/lib/celerity/container.rb:761:in
rescue_status_code_exception' from /Users/moisessiles/jruby-1.4.0/lib/ruby/gems/1.8/gems/celerity-0.7.9/lib/celerity/browser.rb:99:ingoto’

6

You can always increase the number you are passing to -J-Xmx####m.
The OutOfMemoryError indicates you are using more than 1GB of heap
space.

-Justin

On Wed, Feb 24, 2010 at 4:23 PM, Moises S. [email protected]
wrote:

from com/gargoylesoftware/htmlunit/HttpWebConnection.java:432:in

rescue_status_code_exception' memberExpr’
from net/sourceforge/htmlunit/corejs/javascript/Parser.java:2203:in

ok I will try, sorry I’m new with this, what should be the correct

Specify -w for full OutOfMemoryError stack trace
will

/Users/moisessiles/jruby-1.4.0/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
from

`require’
Specify -w for full OutOfMemoryError stack trace

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
7

got it, let me try and I will tell you

thanks

8

hi Justin, one more question,

could my script be affecting the memory usage? Because I’m only running
50
threads and I think this is not too much.

So, I will attach the 2 scripts

Script #1
require “thread”
require “rubygems”
require “celerity”

Runs a given +testScript+ for +numIterations+ using +numUsers+ with a

delay of +delayBetweenUsers+

+testScript+ should be the full path of a test script.

Results are logged in the specified +logFileName+

#NOTE: Watir scipts executed using this loadRunner method need to run
in
seperate IE processes.
#To do this be sure to use the following to create a new IE process in
your
testScript:

require ‘watir/contrib/ie-new-process’ #This helpfull method is

available as of watir development build 1.5.1.1100 see
http://wiki.openqa.org/display/WTR/Development+Builds

ie = Watir::IE.new_process

def
loadTestRunner(testScript,numItterations=1,numUsers=1,delayBetweenUsers=0,
logFileName = Dir.pwd + ‘/myLoadTest.log’)
total_passing = 0
total_failing = 0
log = File.open(logFileName,“a”)

log.puts(“\n\n*********************************\n**Load Test
Configuration: numItterations=#{numItterations}, numUsers=#{numUsers},
delayBetweenUsers=#{delayBetweenUsers} seconds.”)

(1…numItterations).each do |iteration|
itteration_passing = 0
itteration_failing = 0
log.puts(“**Begining Itteration #{iteration} - #{Time.now.strftime(”
%m/%d/%y @ %H:%M:%S “) }”)
puts “Iteration #{iteration}”
i=0
threads = []
numUsers.times do #each thread simulates a user running the
specified
testScript
threads << Thread.new do
i=i+1
startTime = Time.now
result = system(“ruby #{testScript}”) #execute the script and
wait for it to finish.
duration = Time.now-startTime

      if result  #Record our results.
        itteration_passing = itteration_passing +1
        log.puts("Thread #{i} executed #{testScript} Successfully -

Duration = #{duration}seconds")
else
itteration_failing = itteration_failing + 1
log.puts(“Thread #{i} executed #{testScript} FAILED -
Duration =
#{duration}seconds”)
end
end
sleep(delayBetweenUsers) #Wait for the specified
delayBetweenUsers
before we start the next user/thread.
end
threads.each {|t| t.join}
log.puts(“**Finished Itteration #{iteration}. #{itteration_passing}
out
of #{numUsers} tests Passed - #{Time.now.strftime(” %m/%d/%y @ %H:%M:%S
“)
}”)
total_passing = total_passing + itteration_passing
total_failing = total_failing + itteration_failing
end
log.puts(“**Load Test complete. #{total_passing} out of
#{numUsers*numItterations} tests Passed”)
log.close
end

#EXAMPLE USAGE:
#Here we are going to run 5 itterations of the ‘googleSearch.rb’ script
with
four concurent threads/users, using a 2 second delay between the start
of
each user.
#We will save/append the results to a file: ‘myLoadTest.log’
#loadTestRunner(Dir.pwd + ‘/googleSearch.rb’,5,4,2,Dir.pwd +
‘/myLoadTest.log’)
loadTestRunner(‘/Users/myuser/Desktop/Ruby\ and
Watir/my_test.rb’,1,50,4,Dir.pwd + ‘/myLoadTest.log’)

Script #2

You have to include/import the Celerity library

require “rubygems”
require “celerity”
require “thread”

Create a Browser object. Due to Watir compatibility Celerity::IE.new

would
also work. You can specify options, see
HTTP://celerity.rubyforge.org/doc/classes/Celerity/IE.html#M000141
browser = Celerity::Browser.new

test_site = “www.xxxxxxxxx.com”
browser.goto(test_site)
if browser.text.include? “test”
puts “TestPased!”
else
puts “Test Failed! - Couldn’t found test”
end
test_site = “www.xxxxxxxx.com”
browser.goto(test_site)
browser.goto(‘www.xxxxxxxxxx.com’)

9

The system(“ruby…”) call is very expensive:

“Currently we do a bit of magic to make sure that if you system or
exec something like ‘ruby blah blah blah’ it will just start another
JRuby instance in the same process.”

Justin

On Wed, Feb 24, 2010 at 8:54 PM, Moises S. [email protected]
wrote:

require “rubygems”
testScript:
log = File.open(logFileName,“a”)
puts “Iteration #{iteration}”

  sleep(delayBetweenUsers)  #Wait for the specified delayBetweenUsers

#{numUsers*numItterations} tests Passed")
‘/myLoadTest.log’)
require “celerity”
puts “TestPased!”

You can always increase the number you are passing to -J-Xmx####m.

mmmmm I set the value to 1024 and was ok, but after few minutes to run
<init>' ... 24 levels... createStringLiteral’: java.lang.OutOfMemoryError: Java heap space
addExpr' rescue_status_code_exception’

If you want 1024m of heap:

will

from
from

Specify -J-Xmx####m to increase it (#### = cap size in MB).

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
10

Yes, I’m certain your script is affecting the memory usage. I suggest
you set the limit higher.

-Justin

On Wed, Feb 24, 2010 at 8:54 PM, Moises S. [email protected]
wrote:

require “rubygems”
testScript:
log = File.open(logFileName,“a”)
puts “Iteration #{iteration}”

  sleep(delayBetweenUsers)  #Wait for the specified delayBetweenUsers

#{numUsers*numItterations} tests Passed")
‘/myLoadTest.log’)
require “celerity”
puts “TestPased!”

You can always increase the number you are passing to -J-Xmx####m.

mmmmm I set the value to 1024 and was ok, but after few minutes to run
<init>' ... 24 levels... createStringLiteral’: java.lang.OutOfMemoryError: Java heap space
addExpr' rescue_status_code_exception’

If you want 1024m of heap:

will

from
from

Specify -J-Xmx####m to increase it (#### = cap size in MB).

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
11

thanks

mmmmm, so that means that I will need a new script

12

thanks I will check that info

13

From my tests, 50 celerity runs in parallel do not work well on my
machine.

I’ve kept a limit of 10, and use a job queue.

hth,

– Thibaut

14

mmmmm job queue, but that won’t work if I want 50 concurrent users
connected
at same time

regards

On Thu, Feb 25, 2010 at 4:56 AM, Thibaut Barrère

15

Try increasing the max heap size as suggested:

Error: Your application used more memory than the safety cap of 500m.
Specify -J-Xmx####m to increase it (#### = cap size in MB).
Specify -w for full OutOfMemoryError stack trace

I assume it’s impossible to change these specifications
“programmatically” after you’ve started java?
-r

16

yep, also I’m using jmeter as second option, but I want to run that with
other tool like selenium rc, watir, jruby or celerity :S

thanks

17

You probably just need more hardware and more heap memory, the way you
want it done. It looks like your script uses one separate IE instance
(through Watir) for each “concurrent user”.

Watir (and Selenium, too) are known to be expensive to operate per
instance, and is generally used for functional tests. It is not a very
good way to do load tests. Try using load test tools such as JMeter.

Cheers,
Chiaming H.

From: Moises S. [email protected]
To: [email protected]
Sent: Thu, February 25, 2010 6:18:10 AM
Subject: Re: [jruby-user] Fwd: Question jruby and java heap memory
message

mmmmm job queue, but that won’t work if I want 50 concurrent users
connected at same time

regards

On Thu, Feb 25, 2010 at 4:56 AM, Thibaut Barrère
[email protected] wrote:

From my tests, 50 celerity runs in parallel do not work well on my
machine.

18

I think the main point is if you do need to use Selenium RC / Watir /
Celerity or not. These functional test tools use real browser instances
so that they are accurate in rendering, but that is exactly what makes
them memory intensive. If you do need tools working with real browser
instances, then I guess the only option is to add more hardware and
increase the heap size.

JMeter would help in some way, since with JMeter, you can spread out
your 50 concurrent users to multiple test client boxes, instead of
having them all in one box.

Cheers,
Chiaming H.

From: Moises S. [email protected]
To: [email protected]
Sent: Thu, February 25, 2010 9:57:25 AM
Subject: Re: [jruby-user] Fwd: Question jruby and java heap memory
message

yep, also I’m using jmeter as second option, but I want to run that with
other tool like selenium rc, watir, jruby or celerity :S

thanks

On Thu, Feb 25, 2010 at 11:46 AM, Chiaming H. [email protected]
wrote:

You probably just need more hardware and more heap memory, the way you
want it done. It looks like your script uses one separate IE instance
(through Watir) for each “concurrent user”.

Watir (and Selenium, too) are known to be expensive to operate per instance, and is generally used for functional tests. It is not a very good way to do load tests. Try using load test tools such as JMeter.

Cheers,

Chiaming H.

From: Moises S. [email protected]

19

Nope. Parameter nerfs to be set when jvm starts.

Sent from my iPhone

On 25-feb-2010, at 18:56, Roger P. [email protected] wrote:

–
Posted via http://www.ruby-forum.com/.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
20

Chiaming H. <camyhsu yahoo.com> writes:

I think the main point is if you do need to use Selenium RC / Watir / Celerity
or not. These functional test tools use real browser instances so that
they are
accurate in rendering, but that is exactly what makes them memory
intensive. If
you do need tools working with real browser instances, then I guess the
only
option is to add more hardware and increase the heap size.JMeter would
help in
some way, since with JMeter, you can spread out your 50 concurrent users
to
multiple test client boxes, instead of having them all in one
box.Cheers,Chiaming H.

From: Moises S. <moises.siles gmail.com>To: user
jruby.codehaus.orgSent: Thu, February 25, 2010 9:57:25 AMSubject: Re:
[jruby-
user] Fwd: Question jruby and java heap memory message
yep, also I’m using jmeter as second option, but I want to run that with other
tool like selenium rc, watir, jruby or celerity :SthanksOn Thu, Feb 25,
2010 at
11:46 AM, Chiaming H. <camyhsu yahoo.com> wrote:
You probably just need more hardware and more heap memory, the way you want it
done. It looks like your script uses one separate IE instance (through
Watir)
for each “concurrent user”.Watir (and Selenium, too) are known to be
expensive
to operate per instance, and is generally used for functional tests. It
is not
a very good way to do load tests. Try using load test tools such as
JMeter.Cheers,
Chiaming H.

From: Moises S. <moises.siles gmail.com>To: user
jruby.codehaus.orgSent: Thu, February 25, 2010 6:18:10 AMSubject: Re:
[jruby-
user] Fwd: Question jruby and java heap memory message

mmmmm job queue, but that won’t work if I want 50 concurrent users connected
at same timeregardsOn Thu, Feb 25, 2010 at 4:56 AM, Thibaut Barrère
<thibaut.barrere gmail.com> wrote:From my tests, 50 celerity runs
in
parallel do not work well on my machine.I’ve kept a limit of 10, and use
a job
queue.hth,-- Thibaut

Take a look at TestMaker by PushToTest.com. There is a community
version that
supports up to 200 concurrent users and you can spread the load or use
the
cloud. It can run Selenium scripts, Java, Ruby, PHP, Python, etc.

Hope this helps.

Troy Amyett

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email