If “web programming” is what you want to do then I recommend starting
with PHP. Any shared host or current OS install will probably support
PHP out of the box and writing your first PHP web script can be as
simple as:
From there you can grow it up and the http://php.net/ documentation
is excellent.
I have to STRONGLY DISAGREE.
I would NOT recommend learning PHP because it as an ABSOLUTELY AWFUL
LANGUAGE.
If you “learn” it first, you will CUT YOURSELF OFF from an entire
range of approaches, thinking and solutions! (If you’re making a
living on PHP, use it: if you want to learn, look elsewhere–it’s
easier to “scale thinking” down than to try and scale it up late.)
(Talking about the design, not how “practical” it is because every $2
web-host supports PHP 4/5. If you feel like “debating” how well PHP is
as a -language-, look up create_function() and provide a valid
argument for the implementation.)
Besides, the PHP documentation has to be the worst site ever. It also
promotes bad practices.
If you like the “embedded approach” that PHP encourages, there are
PLENTY of frameworks out there that will allow you write a mash of
something that is hard to maintain. Some people like mixing things.
Others don’t. The thing is, almost every OTHER language (save PHP, CF,
JSP and a few more I’m sure) allows you do to this without
establishing it “as the way”.
To get started doing web development in Ruby is a bit more work, in
fact an optimal way is probably to start with Rails (which does most
of the heavy lifting for you) and then work backwards from there.
Also, the online Ruby docs are considerably less beginner-forgiving
that the PHP docs but then again you have this list 
Ruby is not Rails. Rails is not Ruby. Rails is just the most popular
Ruby web framework.
You can’t compare Ruby to PHP in this sense.
It would be slightly more useful to compare Rails vs. Cake (or
whatever else you have).
There are MANY ruby web frameworks that are much easier to “whip out
small projects” (esp. assuming that you are a beginner that hasn’t
learned the “The Rails Way”) than Rails.
That said, if you want to become enamoured of a language and have a
ton of fun simply learning further programming for the sake of
programming, I’d start with Ruby and the command line. A first script
can be as simple as:
Ruby is a fun language and you’ll learn many new constructs (coming
from VB) if you learn it. Ruby will teach you to code in ways that
simply aren’t feasible in PHP. You’ll learn about powerful dynamic OO,
closures/blocks, mixins, singleton-methods, functional syntax,
duck-typing, how easy creating a DSL should be, testing, etc…
Ruby even comes with this nifty program called “irb” so you can try
things interactively. You can also get a start at
http://tryruby.hobix.com. (If you have a crazy sense of humor, _why
might just make you learn Ruby as an excuse to keep up with his crazy
antics…)
I personally would not start with Java. Perhaps later if neither Ruby
nor PHP touch you, but to start with Java might make you think that
web programming requires tedium and dreariness before the fun sets in
and that just ain’t so.
I would avoid starting with Java as well. Java is a mediocre OO
language at best and the boilerplate will make you cry–as far as a
language, it really teaches nil. (For much better statically/strongly
typed languages, see Scala, Eiffel and various ML implementations as
examples). That being said, the JVM is the best cross-platform VM I
know about. Which is why JRuby (yes, Ruby that runs in, and can
interact with, “Java”) runs on it 
There are MANY languages out there. Unless you are (or will be) paid
for using a crappy language, don’t use it. It’s not fun. And you’ll be
cramped in the limited language space. Learn -good- techniques while
you can. The more -good- approaches you know the better you can code.
Languages to look at (in no particular order, just a small random list
that comes to the top of my head of languages that can teach you
things–google will provided hundred of languages with multiple
variations if you ask):
Io, Scala, Lisp/Scheme, F#, Ruby, Python, Perl, Self, Smalltalk, Groovy
There are also a million Ruby frameworks.
Merb. Wuby. Ramaze. Rack+whatever, Camping, Sinatra, Rails, Nitro/Og,
etc, etc.
Each framework has its own design ideals. Some are huge, some are
small. But, with the core separation the choice is truly up to you for
what fits your current (web) project.
Opinionated: yes.