Solution to

Exact problem as chris, pentor and Vince – it was driving me crazy.
Deleting the cache did not help – the gem magically reappears each
time.

I managed to install ActiveSupport 2.1.0 by downloading from its
rubyforge project page manually and installing locally, and then “gem
install rails” goes through just fine. Looks like a bad repository
version to me.

Was upgrading from Rails 1.2.3 on Vista, using RubyGems 1.2.