Hi All,
In order to track down some high memory usage problems I have in a
server process that is using the IronRuby engine I have run a test using
a single line script and have encountered an issue that looks
suspiciously like a memory leak. I’m scouring through the IronRuby code
commenting out a few static lists as I find them to see if they make any
difference but to date haven’t come across anything useful.
The test I’m running is:
static void Main(string[] args)
{
ScriptRuntime scriptRuntime = IronRuby.CreateRuntime();
for (int index = 0; index < 1000; index++)
{
ScriptScope rubyScope = scriptRuntime.CreateScope(“IronRuby”);
rubyScope.Execute(“print ‘execute\n’”);
}
Console.WriteLine(“finished”);
Console.ReadLine();
}
Executing this three times on 3 separate runs the resultant process had
memory allocations of the folowing amounts as shown in the Windows Task
Manager:
31,940k
31,936k
31,936k
Executing the loop 10,000 times on 3 separate runs resulted in memory
usage of:
133,648k
134,036k
133,808k
Changing to
static void Main(string[] args)
{
for (int index = 0; index < 1000; index++)
{
Console.WriteLine(“execute”);
}
Console.WriteLine(“finished”);
Console.ReadLine();
}
For 1,000 iterations memory usage was 8,402k.
For 10,000 iterations memory usage was 8,704k.
For 1,000,000 iterations memory usage was 9,348.
The reason I’m looking into this is that I am having to jeep a close eye
on my server process and re-start it every 3 or 4 days when the memory
usage gets up to 300 to 400MB.
Regards,
Aaron
Additional info…
If you move the scriptRuntime.CreateScope method outside the loop then
there
is no increase in memory.
Pete
Also…
If you only have the CreateScope method and not the Execute method in
the
loop then there is no memory increase.
With both in the loop you get a linear increase in memory usage, even if
you
regularly force a Garbage Collection, of approximately 10K per
iteration.
Pete
Peter Bacon D. wrote:
Additional info…
If you move the scriptRuntime.CreateScope method outside the loop then
there
is no increase in memory.
But from my understanding of script scopes I need a new scope when using
local variables. I can have multiple scripts executing on different
threads with different local variables so as far as I’m aware creating a
new scope each time is the only way to allow that?
static void Main(string[] args)
{
ScriptRuntime scriptRuntime = IronRuby.CreateRuntime();
for (int index = 0; index < 1000; index++)
{
ScriptScope rubyScope = scriptRuntime.CreateScope(“IronRuby”);
rubyScope.SetVariable("local1", "hello");
rubyScope.SetVariable("local2", "world");
rubyScope.Execute("print \"#{local1} #{local2}\n\"");
}
Console.WriteLine(“finished”);
Console.ReadLine();
}
Regards,
Aaron
Peter Bacon D. wrote:
Also…
If you only have the CreateScope method and not the Execute method in
the
loop then there is no memory increase.
With both in the loop you get a linear increase in memory usage, even if
you
regularly force a Garbage Collection, of approximately 10K per
iteration.
Pete
Hi Pete,
If I take the Execute method out of the loop then I don’t get any
significant increase in memory usage. However as per my post above I
think I do need the CreateScope and Execute calls in the loop to allow
for multiple simultaneous script executions.
Regards,
Aaron
Hi Aaron,
Sorry I wasn’t trying to suggest a solution for your problem. I was
just
trying to narrow down where the increase in memory is coming from.
I suspect that the cause is initializing or calling into the core
libraries
but I can’t isolate that bit.
Pete
Looks like keeping a limit on the number of ScriptScope’s being used is
the best approach for a long running process which makes sense. The code
below does not have a memory usage problem.
class Program
{
private static Queue m_scriptScopes = new
Queue();
private static AutoResetEvent m_scriptScopeAvailable = new
AutoResetEvent(false);
static void Main(string[] args)
{
ScriptRuntime m_scriptRuntime = IronRuby.CreateRuntime();
m_scriptScopes.Enqueue(m_scriptRuntime.CreateScope("IronRuby"));
m_scriptScopes.Enqueue(m_scriptRuntime.CreateScope("IronRuby"));
m_scriptScopes.Enqueue(m_scriptRuntime.CreateScope("IronRuby"));
for (int index = 0; index < 10000; index++)
{
ScriptScope rubyScope = null;
while (rubyScope == null)
{
lock (m_scriptScopes)
{
if (m_scriptScopes.Count > 0)
{
rubyScope = m_scriptScopes.Dequeue();
}
else
{
m_scriptScopeAvailable.Reset();
}
}
if (rubyScope == null)
{
m_scriptScopeAvailable.WaitOne();
}
}
ThreadPool.QueueUserWorkItem(new WaitCallback(RunRubyScript), new
object[] {rubyScope, index});
}
Console.ReadLine();
}
private static void RunRubyScript(object state)
{
ScriptScope rubyScope = (ScriptScope)((object[])state)[0];
int count = (int)((object[])state)[1];
rubyScope.ClearVariables();
rubyScope.SetVariable("local1", count);
rubyScope.SetVariable("local2", count);
rubyScope.Execute("print \"#{local1} - #{local2}\n\"");
lock (m_scriptScopes)
{
m_scriptScopes.Enqueue(rubyScope);
m_scriptScopeAvailable.Set();
}
}
}
Regards,
Aaron
I’m working on working set issues right now. It should be better soon.
Will look at this one especially.
Thanks,
Tomas
Tomas M. wrote:
I’m working on working set issues right now. It should be better soon.
Will look at this one especially.
Thanks,
Tomas
Hi Tomas,
I didn’t quite nail down where the high memory usage was coming from
either but I did get to a stage where I could determine it was happening
after the RubyScope was being created. So up until the call to
InvokeTarget in the ScriptCode everything was fine. That obersvation is
probably not particularly useful since all the magic seems to happen
when InvokeTarget is called.
Another observation is that in the SymbolTable class in
Microsoft.Scripting.Core there are only objects being put onto the two
static dictionaries. Nothing is ever taken off them. I would have
thought the SymbolTable would need to somehow know to remove objects
after the ScriptScope that puts them on has diesd.
Regards,
Aaron
Peter Bacon D. wrote:
Hi Aaron,
Sorry I wasn’t trying to suggest a solution for your problem. I was
just
trying to narrow down where the increase in memory is coming from.
I suspect that the cause is initializing or calling into the core
libraries
but I can’t isolate that bit.
Pete
Hi Pete,
Thanks for the suggestion in the first place!
I realise IronRuby is still under heavy development so expect to have to
do the odd workaround.
Tomas thanks for taking a look.
Regards,
Aaron