Just buy a bunch of Quad Core Opterons and get over it !
If you don’t think Ruby is scalable just check with the folks at
Twitter - they have been successfully using Ruby for some time now -
all they had to do was buy a ton of compute gear and life is good !
Ruby M. wrote:
Just buy a bunch of Quad Core Opterons and get over it !
If you don’t think Ruby is scalable just check with the folks at
Twitter - they have been successfully using Ruby for some time now -
all they had to do was buy a ton of compute gear and life is good !
If Twitter is a for-profit venture, I’d be very surprised if usable
details about how they plan and manage load, scalability and capacity
are available without an NDA. This is something we performance engineers
consider “The Family Jewels”, although perhaps curious folk in other
disciplines wonder what the fuss is all about. 
And yes, Google does throw a ton of compute gear (and Map-Reduce) at
their searching and indexing, which, as far as I know, is not written
in Ruby and does not involve Rails.
And just on the off chance you would like to become a capacity
planner, there’s to my knowledge only one place outside of a university
where you can learn how:
http://www.perfdynamics.com/Classes/schedule.html
Tell Dr. Gunther I said “Hello”.
James Edward G. II wrote:
details about how they plan and manage load, scalability and capacity
are available without an NDA.That slideshow claims they use 8 Sun X4100s.
James Edward G. II
Cute … but they did use the E-word! And no family jewels.
On Sep 25, 2007, at 07:22 , M. Edward (Ed) Borasky wrote:
engineers
consider “The Family Jewels”, although perhaps curious folk in other
disciplines wonder what the fuss is all about.
These things are less secret than you think. Twitter’s solutions to
scaling issues are going to be inappropriate to your scaling issues.
Knowing the details won’t help you, knowing the techniques will.
On Sep 25, 2007, at 9:22 AM, M. Edward (Ed) Borasky wrote:
Ruby M. wrote:
Just buy a bunch of Quad Core Opterons and get over it !
If you don’t think Ruby is scalable just check with the folks at
Twitter - they have been successfully using Ruby for some time now -
all they had to do was buy a ton of compute gear and life is good !If Twitter is a for-profit venture, I’d be very surprised if usable
details about how they plan and manage load, scalability and capacity
are available without an NDA.
That slideshow claims they use 8 Sun X4100s.
James Edward G. II
Eric H. wrote:
are available without an NDA. This is something we performance engineers
consider “The Family Jewels”, although perhaps curious folk in other
disciplines wonder what the fuss is all about.These things are less secret than you think. Twitter’s solutions to
scaling issues are going to be inappropriate to your scaling issues.
Knowing the details won’t help you, knowing the techniques will.
The Twitter folks have been very forthcoming about the trials and
tribulations of scaling up their system to meet demand. All sorts of
details regarding language tweaks, hardware changes and DB schema
improvements were all available to the public.
Now, I’m sure not everything was made public, but they have been more
than transparent about their scalability issues.
On Sep 25, 9:55 am, Eric H. [email protected] wrote:
details about how they plan and manage load, scalability and capacity
are available without an NDA. This is something we performance
engineers
consider “The Family Jewels”, although perhaps curious folk in other
disciplines wonder what the fuss is all about.These things are less secret than you think. Twitter’s solutions to
scaling issues are going to be inappropriate to your scaling issues.
Knowing the details won’t help you, knowing the techniques will.
See also:
and
Some of this sort of thing seems a bit over the top in terms of what
one might be required to do when scaling web apps based on, shall we
say, more efficient languages that run faster.
Given the fact that Ruby can be no faster than 1/10th the speed of
let’s say almost any other language (take your pick) I would surmise
one might have to work 10x harder to scale a RoR app than if something
more efficient were being used.
RoR gives a double whammy for scalability - Ruby is not all that
speedy and Rails emits SQL Statements with a lot of “" chars -
combine both and you have scalability issues to be sure. Of course I
know some people who say "” chars in SQL Statements actually makes
the SQL run faster but then those who say that also don’t like to run
benchmarks because benchmarking is seen as being a waste of time and
effort. I did run a benchmark that demonstrate the use of “" in SQL
statements makes the SQL Engine run anywhere from 10% slower to 880%
slower even when very long lists of column names was not being used as
is the case when "” is used in place of a list of column names.
On the other side of the coin, RoR is very easy to code and this one
factor alone is generally what people cite as being the real reason
RoR was used. Having coded other languages I can tell you all
languages are easy to code once that language is understood well
enough to make the act of coding enjoyably quick and easy.
Keep in mind, I am a died in the wool Ruby lover like the rest of us
here however when it comes to scalability issues I have seen Ruby
enthusiasts spend a lot of time waiting for their code to run so a bug
can be fixed just so they can wait even more time to get to the next
instance of a bug fix. This tends to place some weight on the side of
the balance where coding with a more performance conscious language
may be more profitable in terms of where one spends their time as
opposed to simply focusing on the easy-to-code side of the equation.
Given the fact that Ruby can be no faster than 1/10th the speed of
let’s say almost any other language (take your pick) I would surmise
one might have to work 10x harder to scale a RoR app than if something
more efficient were being used.
That may be true when the language speed is the limiting factor.
I am not sure that’s always the case…
On Sep 25, 1:48 pm, “Rimantas L.” [email protected] wrote:
Rimantas
–http://rimantas.com/
Are you trying to say a slow inefficient language such as RoR should
be as easy to scale as a faster more efficient language given the
scalability requirements revolve around usability by a growing
population of users ?
Name me some instances when scalability does not require better
performance characteristics ? (Better means faster…)
Name me some instances when scalability does not require better
performance characteristics ? (Better means faster…)
Faster what? Language? Or DB? Or network?
Suppose you have two languages you can use for the same application.
Language A can process request in 0.001 second, language B can process
request in 0.00005 second (20x faster). Now they both sit and wait for
DB
which gives result to a query in 0.01 second.
Does it matter, which language does nothing faster?
Very small percentage of the request life cycle is spent in the part
that’s dependant
on language speed that is why language speed is not always as important.
Regards,
Rimantas
Soo… Ruby DOES use multiple cores? I read somewhere that it
didn’t (probably this thread).
And if it doesn’t use multiple cores, is there any work on it being
able to use them?
Ari
---------------------------------------------------------------|
~Ari
“I don’t suffer from insanity. I enjoy every minute of it” --1337est
man alive
Hi,
In message “Re: Ruby Scales just fine !”
on Wed, 26 Sep 2007 09:28:36 +0900, Ari B. [email protected]
writes:
|Soo… Ruby DOES use multiple cores? I read somewhere that it
|didn’t (probably this thread).
Single Ruby process does not. But we can use as much cores we have by
forking processes, and usually it’s the case for web applications.
Ari B. wrote:
alive
Well, if you have two Ruby processes on your dual-core box, all of the
concurrency techniques that work via inter-process communication will
work. “Green” threads in Ruby 1.8.x won’t, but native OS threads in Ruby
1.9 will work.
On Sep 25, 2:18 pm, “Rimantas L.” [email protected] wrote:
Very small percentage of the request life cycle is spent in the part
that’s dependant
on language speed that is why language speed is not always as important.Regards,
Rimantas
–http://rimantas.com/
I know what you point is but the issues dealing with scalability
always hinge on the performance of the langauge one chooses when
manipulating data from the data once the data arrives.
I would always go with the language that runs faster regardless of how
much time the user spends waiting for data because once the data
arrives the faster language will process the data faster.
Honestly I think the reasoning being used here makes me think those
who love Ruby have to manipulate their minds to keep them in love with
Ruby or else they might lose their love of Ruby. ← This is the
definition of a religion more than anything else.
Another favorite argument for those who love slow inefficient
languages revolves around the feeling that being disk I/O bound means
the language’s performance doesn’t matter. I mean, if the language is
only being executed 1% of the time and the disk light is on 99% of the
time who cares how fast the language is because it only gets 1% of the
CPU. ← This is an ancient argument without merit these days. But
even if the language only gets 1% of the CPU time I would choose the
language that runs 20x faster to make the best of the 1% of the CPU
the language runtime gets.
Think of any scenario you want to conjure-up that makes you feel like
the performance of the language doesn’t matter and I can certainly
come-up with a scenario that states the fastest language is the one
what should be used either because it can push more data or it can
handle requests faster or it just gets the job done more quickly or it
saves money on hardware.
On Sep 25, 2007, at 8:38 PM, Yukihiro M. wrote:
forking processes, and usually it’s the case for web applications.
So fork() moves to a new processor if there is one, else a new thread?
-------------------------------------------|
Nietzsche is my copilot
On Sep 25, 5:38 pm, Yukihiro M. [email protected] wrote:
Hi,
In message “Re: Ruby Scales just fine !”
on Wed, 26 Sep 2007 09:28:36 +0900, Ari B. [email protected] writes:|Soo… Ruby DOES use multiple cores? I read somewhere that it
|didn’t (probably this thread).Single Ruby process does not. But we can use as much cores we have by
forking processes, and usually it’s the case for web applications.
Ruby would have to use real OS threads to take advantage of how
threads may be assigned to more than one core or more than one CPU,
etc, depending on how the parent process was spawned in terms of the
affinity for CPU’s or cores.
Ruby M., you seem to like this recurrent topic, but why not give it
a
little rest before posting a new thread with the same old speed
arguments?
Its not as if your speed problems magically go away from
one day to the other, and I guess we know your opinion by now too 
You seem to continually start new and new threads about this.
Ruby M. wrote:
Another favorite argument for those who love slow inefficient
languages revolves around the feeling that being disk I/O bound means
the language’s performance doesn’t matter. I mean, if the language is
only being executed 1% of the time and the disk light is on 99% of the
time who cares how fast the language is because it only gets 1% of the
CPU. ← This is an ancient argument without merit these days.
Why does it have no merit? If I am writing an app that needs to write to
disk transactionally (a pretty common type of app), then the significant
latency involved in writing to disk is just as much (or more) of an
issue today as it was in “ancient” times.
http://herbsutter.spaces.live.com/blog/cns!2D4327CC297151BB!304.entry
On Sep 25, 2007, at 8:24 PM, Ari B. wrote:
|didn’t (probably this thread).
Single Ruby process does not. But we can use as much cores we
have by
forking processes, and usually it’s the case for web applications.So fork() moves to a new processor if there is one, else a new thread?
fork() creates another process which your schedular is free to handle
as it sees best. This may mean running that process on a different
CPU, if the schedular decides that’s the way to go.
James Edward G. II
Alex Y. wrote:
writes:
as it sees best. This may mean running that process on a different
CPU, if the schedular decides that’s the way to go.
OT, but is there a combination of libraries/OS/network-level
accoutrements that would allow a scheduler to migrate the process to
another host on fork()?
If you mean the “native” scheduler that’s built in to the OS, you or
someone would have to write a library extension in C to make that
happen, which is OS dependent, and in the case of Linux, kernel version
dependent as well.
In general, however, unless you really really know what you’re doing
and why you’re doing it, you’re better off letting the OS scheduler do
its things without even trying to give it hints on how to treat your
code. It’s not nice to fool the scheduler – that way lies deadlocks,
race conditions, starving philosophers, etc.
Just throw hardware at it – lots of cores, lots of RAM, lots of
light-weight processes communicating by message passage and immutable
objects a la Erlang. The next version of Event Machine will have the
underlying mechanisms to make this work.
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.
Sponsor our Newsletter | Privacy Policy | Terms of Service | Remote Ruby Jobs