“Right now I’m using awk and (to a lesser extent) perl to
do this.”
One advantage with python or ruby would be that you can reuse
a lot of that, still have the functionality (of like awk or
near to, at least a nicer syntax), while still being able to
maintain it nicely (different to perl, maintaining perl
scripts STINKS in my opinion)
“However, I’d like to learn a newer scripting language and at
the same time learn OOP, so I’m choosing between Python and Ruby.”
My Python is not that good…
I think python as a language is acceptable, more than perl in
practise. But Ruby’s OOP model does not feel second class, different
to the python differences about length(object) or object.foo().
For example, python feels the need to pass def foo(self).
This is MANDATORY (although you can use any other name than self (!).
When you invoke a method, such as cat.meow, in python you
MUST use the meow(). Python says this is to your advantage, but to
me it is a fundamental difference in philosophy …
It basically tells you, python way, or the highway.
Python forces, ruby allows … 
“2) (less easy) semi-automatic generation of graph, images, report,
etc., based on data in huge binary files (O(Gb));”
I think this can be a bit difficult in either language, but
whatever applies to python, will apply to ruby too. (Or to
perl in this regard, at least perl-ruby … ruby is really
how perl should have been from the start, perl’s OOP is
a shame, and perl offers 10000 ugly ways to do something
you write once, and then never look at it again …)
Each language has some other advantages here and there,
for example it was my impression that python has a little bit
more bindings in general than ruby (i.e. blender plugin, ogre3d
wrapper and so on). Ruby catches up to this, but it still will
take a bit more time …
“3) (hard) create GUIs for scientifc codes, which could enable
unexperienced users to run the codes”
I dare claim that ruby-gtk is (almost) as good as python-gtk.
So at least for gtk, there really is no difference IMHO to the
python variant (there is a tiny catch if you need to use a lot of
threads and stuff… but just have a look at recent mailing
list, in my opinion this is really no “showstopper” at all)
“Which are the main differences between Ruby and Python?”
I really think the attitude towards OOP is a huge difference.
In essence, ruby is a lot cleaner in this philosophical regard.
I can forgive python its indent-obsession, but i can not forgive
it the implicit self and forced () … by the way, the default
interactive python is also annoying in its mindset, “exit” or
“quit” does not work… I am just glad I can alias that for
irb without a fuzz. Again a little philosophical difference … 
(But in my opinion, ruby is actually a lot more fun than python
as ruby doesn’t superimpose as much as python does on you)
“Which is easier to learn?”
To be honest, I think BOTH are easy to learn but BOTH are
hard to master. I recommend you to use only a subset of any
language you choose, and slowly build up on it.
“I’ve seen a lot of numerical libraries/extensions for
Python: does Ruby also support scientific programming?”
Well there is gsl, also some stuff is in the Math module, then
we have some matrices and more stuff… I guess the hardcore
maths users go and use Haskell anyway 