Hi, my video sharing website (politube.org) that I developed with rails,
runs on lighttpd/fcgi (I know it is obsolete by now) and is severely
under load and regularly blocks because the lighttpd process chews up
50% of RAM (1GB).
So now I want to move the system to a new server with 8GB, 64bit ,
Quad core processor (intel i7 920) but I am not sure if hyperthreading
will really do anything better. Even if I switched to mongrel, would it
be able to leverage the Quadcore architecture?
Any advice would be greatly appreciated.
Kuros
Hi Kuros,
One thing that could certainly help is moving to a apache/nginx +
passenger setup. That would improve your memory usage.
But keep in mind that if you rely on ruby threads on your application,
ruby won’t take advantage of your QuadCore processor. Only if you
spawn multiple processes. But again, this depends on your application
and how it works.
That being said, I wrote a post a whlie ago explaining a bit how ruby
handles threads (in 1.8, 1.9 and in jruby). It might be useful as
well.
Cheers,
Leonardo Borges
On Jul 2, 10:34 am, Kuros Y. [email protected]
Kuros Y. wrote:
Hi, my video sharing website (politube.org) that I developed with rails,
runs on lighttpd/fcgi (I know it is obsolete by now) and is severely
under load and regularly blocks because the lighttpd process chews up
50% of RAM (1GB).
So now I want to move the system to a new server with 8GB, 64bit ,
Quad core processor (intel i7 920) but I am not sure if hyperthreading
will really do anything better. Even if I switched to mongrel, would it
be able to leverage the Quadcore architecture?
Please note that hyperthreading (which is a technique for splitting up
one procesor into two virtual processors) is not the same as application
threading.
Whether hyperthreading will be beneficial is highly dependent on your
workload. Benchmarks for your application are your friend.
In general, I would say, hyperthreading will be beneficial if you have a
workload that is a composite of two distinct and mutually exclusive
loads, such as video processing on the one hand and I/O on the other. If
the loads are not mutually exclusive then you will have a lot more cache
misses and hyperthreading may actually be detrimental.
–
Roderick van Domburg
http://www.nedforce.com
i found thin seems to run faster then mongrel in my app. just my 2
cents.
nayeem
2009/7/2 Kuros Y. [email protected]
Quad core processor (intel i7 920) but I am not sure if hyperthreading
In general, I would say, hyperthreading will be beneficial if you have a
hyperthreading is emulating multiple cpu’s then the multiprocess
–
cashflowclublondon.co.uk
("`-''-/").___..--''"`-._
`6_ 6 ) `-. ( ).`-.__.`)
(_Y_.)' ._ ) `._ `. ``-..-'
_..`--'_..-_/ /--'_.' ,'
(il),-'' (li),' ((!.-'
.
Leonardo Borges wrote:
Hi Kuros,
One thing that could certainly help is moving to a apache/nginx +
passenger setup. That would improve your memory usage.
But keep in mind that if you rely on ruby threads on your application,
ruby won’t take advantage of your QuadCore processor. Only if you
spawn multiple processes. But again, this depends on your application
and how it works.
That being said, I wrote a post a whlie ago explaining a bit how ruby
handles threads (in 1.8, 1.9 and in jruby). It might be useful as
well.
Cheers,
Leonardo Borges
www.leonardoborges.com
On Jul 2, 10:34�am, Kuros Y. [email protected]
well, I think it is unclear exactly how Quadcore distributes, on the one
hand, in Wikipedia, it is defined as Hyperthreading, with the
explanation that there are several virtual processors (which would
satisfy my requirements) but at the same time, wikipedia defines
Hyperthreading as SMT (simultaneous Multithreading) and if you follow
that link, SMT is described as supporting several parallel THREADS
(implying that all within the same process). In case of the latter, I
guess a Quadcore setup with out of the box rails wouldn’t really help.
You suggest BTW apache/nginx, which I never used before, any ideas on
how mongrail behaves under load?
thanks
Kuros
Roderick van Domburg wrote:
Kuros Y. wrote:
Hi, my video sharing website (politube.org) that I developed with rails,
runs on lighttpd/fcgi (I know it is obsolete by now) and is severely
under load and regularly blocks because the lighttpd process chews up
50% of RAM (1GB).
So now I want to move the system to a new server with 8GB, 64bit ,
Quad core processor (intel i7 920) but I am not sure if hyperthreading
will really do anything better. Even if I switched to mongrel, would it
be able to leverage the Quadcore architecture?
Please note that hyperthreading (which is a technique for splitting up
one procesor into two virtual processors) is not the same as application
threading.
Whether hyperthreading will be beneficial is highly dependent on your
workload. Benchmarks for your application are your friend.
In general, I would say, hyperthreading will be beneficial if you have a
workload that is a composite of two distinct and mutually exclusive
loads, such as video processing on the one hand and I/O on the other. If
the loads are not mutually exclusive then you will have a lot more cache
misses and hyperthreading may actually be detrimental.
–
Roderick van Domburg
http://www.nedforce.com
actually, this would presumably answer my question, since if
hyperthreading is emulating multiple cpu’s then the multiprocess
architecture of fcgi would benefit from that. However, I am not sure on
how load “mutual exclusivity” affects cache usage. Do you mean a kind of
instruction cache? Any Idea on how mongrel vs lighttpd behaves?
thanks
Kuros
Developer In London wrote:
i found thin seems to run faster then mongrel in my app. just my 2
cents.
nayeem
thanks, are you using a multi cpu (dual/quad) hardware?
Kuros
2009/7/2 Kuros Y. [email protected]
Quad core processor (intel i7 920) but I am not sure if hyperthreading
In general, I would say, hyperthreading will be beneficial if you have a
hyperthreading is emulating multiple cpu’s then the multiprocess
–
cashflowclublondon.co.uk
("`-''-/").___..--''"`-._
`6_ 6 ) `-. ( ).`-.__.`)
(_Y_.)' ._ ) `._ `. ``-..-'
_..`--'_..-_/ /--'_.' ,'
(il),-'' (li),' ((!.-'
.
Kuros Y. wrote:
actually, this would presumably answer my question, since if
hyperthreading is emulating multiple cpu’s then the multiprocess
architecture of fcgi would benefit from that. However, I am not sure on
how load “mutual exclusivity” affects cache usage. Do you mean a kind of
instruction cache? Any Idea on how mongrel vs lighttpd behaves?
Sure, multi-process architectures can leverage SMP architectures. But my
point is: be aware that having two equal processes run on two HTT CPU’s
(so actually a single CPU) will be detrimental to performance,
generally. The same code will be running on the same physical CPU
components and that needs to run serially, with all of the cache and
context issues that brings with it.
I prefer a cluster of Passengers, Mongrels or Thins (depending on your
performance profile) over FastCGI because I find it to be more stable
and easier to manage. I also find the Apache / Nginx / Lighttpd
discussion to be a bit of a bike-shed when it comes to serving dynamic
Rails requests. The performance differences are not significant except
for static assets or when you’re pressed for RAM.
All these deployment options are process-based. Apache can run
multi-threaded too. I don’t have any production experience with that.
–
Roderick van Domburg
http://www.nedforce.com
Leonardo Borges wrote:
Hi Kuros,
One thing that could certainly help is moving to a apache/nginx +
passenger setup. That would improve your memory usage.
But keep in mind that if you rely on ruby threads on your application,
ruby won’t take advantage of your QuadCore processor. Only if you
spawn multiple processes. But again, this depends on your application
and how it works.
That being said, I wrote a post a whlie ago explaining a bit how ruby
handles threads (in 1.8, 1.9 and in jruby). It might be useful as
well.
Cheers,
Leonardo Borges
www.leonardoborges.com
On Jul 2, 10:34�am, Kuros Y. [email protected]
Incidentally, I checked out apache/nginx and that seems also interesting
to follow, thanks for pointing it out. I looked for you post (above) but
couldn’t find it.
Kuros