Forum: Rails deployment Net::SSH AuthenticationFailed on some hosts but 'ssh' works

B252649cb7e16bde9f42809f78e1c9ee?d=identicon&s=25 Craig Taverner (craig-taverner)
on 2009-01-08 16:47
I have a custom capistrano recipe that runs commands on a couple of
remote hosts, using publickey ssh, and has been working for several
months. Recently (in the last week or so), something changed I have not
been able to identify, and the Net::SSH.start() method raise
AuthenticationFailed for only one of the remote hosts. The capistrano
recipe has not changed, the net-ssh gem is 2.0.8 and was updated long
ago. I can get the same error using either capistrano, or direct
Net::SSH.start within IRB.

I use ubuntu 8.04 on all computers involved, and follow regular security
updates, but have not been able to track which one might have caused
this, and why only one remote machine is affected.

Also, normal command-line 'ssh' works correctly every time for all
hosts.

To try and track this, I've done a series of tests connecting to two
remote machines, one working and one not, and comparing the differences,
both with command-line ssh and with Net::SSH with high debug levels set.
Most differences found were expected, like different host keys, but the
sequence of events was always the same, with one exception, the debug
output from Net::SSH showed a sequence of messages and responses, and
message 5 differed. Here follows a more explicit description of the
test:

Given that 'ssh -l userx goodhost' and 'ssh -l userx badhost' both work
correctly and identically with publickey authentication, we get the
following results in IRB (edited for clarity):

  Net::SSH.start('goodhost','userx',{:verbose => Logger::DEBUG})
    ...
    trying publickey
    queueing packet nr 5 type 50 len 508
    received packet nr 5 type 60 len 460
    queueing packet nr 6 type 50 len 556
    received packet nr 6 type 52 len 12
    publickey succeeded

  Net::SSH.start('badhost','userx',{:verbose => Logger::DEBUG})
    ...
    trying publickey
    queueing packet nr 5 type 50 len 508
    received packet nr 5 type 51 len 44
    all authorization methods failed

So, it seems packet 50 should receive 60 in response, but gets 51
instead. I have no idea what these numbers mean, and why different
responses are received by Net::SSH, when the command-line ssh works
fine.

As said before, this is a problem that has suddenly happened, with no
obvious change to the computer configurations, and all computers are
identically configured (with regards to ssh, ruby and Net::SSH).

Does anyone have any ideas, or further suggestions on where to look?

P.S. Things I've already tried that have not helped:
- removing ssh gateway from my configs
- downgrading net-ssh gem all the way back to 2.0.1
- run command-line ssh with maximum verbocity to find differences
2f72e804bd1efe88c285a3a3fd8b61fc?d=identicon&s=25 Teddy Ballgame (teddyballgame)
on 2009-01-10 01:56
Craig Taverner wrote:
> I have a custom capistrano recipe that runs commands on a couple of
> remote hosts, using publickey ssh, and has been working for several
> months. Recently (in the last week or so), something changed I have not
> been able to identify, and the Net::SSH.start() method raise
> AuthenticationFailed for only one of the remote hosts. The capistrano
> recipe has not changed, the net-ssh gem is 2.0.8 and was updated long
> ago. I can get the same error using either capistrano, or direct
> Net::SSH.start within IRB.
>
> I use ubuntu 8.04 on all computers involved, and follow regular security
> updates, but have not been able to track which one might have caused
> this, and why only one remote machine is affected.
>
> Also, normal command-line 'ssh' works correctly every time for all
> hosts.
>
> To try and track this, I've done a series of tests connecting to two
> remote machines, one working and one not, and comparing the differences,
> both with command-line ssh and with Net::SSH with high debug levels set.
> Most differences found were expected, like different host keys, but the
> sequence of events was always the same, with one exception, the debug
> output from Net::SSH showed a sequence of messages and responses, and
> message 5 differed. Here follows a more explicit description of the
> test:
>
> Given that 'ssh -l userx goodhost' and 'ssh -l userx badhost' both work
> correctly and identically with publickey authentication, we get the
> following results in IRB (edited for clarity):
>
>   Net::SSH.start('goodhost','userx',{:verbose => Logger::DEBUG})
>     ...
>     trying publickey
>     queueing packet nr 5 type 50 len 508
>     received packet nr 5 type 60 len 460
>     queueing packet nr 6 type 50 len 556
>     received packet nr 6 type 52 len 12
>     publickey succeeded
>
>   Net::SSH.start('badhost','userx',{:verbose => Logger::DEBUG})
>     ...
>     trying publickey
>     queueing packet nr 5 type 50 len 508
>     received packet nr 5 type 51 len 44
>     all authorization methods failed
>
> So, it seems packet 50 should receive 60 in response, but gets 51
> instead. I have no idea what these numbers mean, and why different
> responses are received by Net::SSH, when the command-line ssh works
> fine.
>
> As said before, this is a problem that has suddenly happened, with no
> obvious change to the computer configurations, and all computers are
> identically configured (with regards to ssh, ruby and Net::SSH).
>
> Does anyone have any ideas, or further suggestions on where to look?
>
> P.S. Things I've already tried that have not helped:
> - removing ssh gateway from my configs
> - downgrading net-ssh gem all the way back to 2.0.1
> - run command-line ssh with maximum verbocity to find differences

I'm having similar problems with Capistrano. I've setup a second server
on Slicehost using the exact same process (so I thought) as the first.
Cap deploy works fine for the first, but for the second prompts for my
password.  My Capistrano recipe is the same for both, but only the new
setup is requiring a password, the first simply connects through my ssh
key.  If I do enter the password, it fails every time with the response:

connection failed for: MY_SLICE_IP_ADDRESS
(Net::SSH::AuthenticationFailed: MY_SLICE_USERNAME)

SSH through the command line works just fine for both using ssh keys and
no password provided.

I'm not very familiar with Net::SSH but ran the two commands you
provided above and in both cases the public key was successfully
authorized.

So I guess I'm pretty stumped as well.

Thanks,
Ted
56a74b806f68191683d43f53e732269f?d=identicon&s=25 mla mla (mla)
on 2009-10-23 07:27
I was experiencing similar troubles. I was also able to ssh in fine
using openssh, but Net::SSH (2.0.15) was always returning auth failed.

Here's a script that was failing:

require 'rubygems'
require 'net/ssh'
require 'net/sftp'
Net::SSH.start('myserver', 'myuser', { :verbose => Logger::DEBUG }) do
|ssh|
end


And it was always failing with this error:
E, [2009-10-22T22:23:42.401052 #5381] ERROR --
net.ssh.authentication.session[3f9e1f693000]: all authorization methods
failed (tried hostbased)


However, if I specify the allowed method explicitly, it works:

require 'rubygems'
require 'net/ssh'
require 'net/sftp'
Net::SSH.start('myserver', 'myuser', { :verbose => Logger::DEBUG,
:auth_methods => %w{ publickey } }) do |ssh|
end


D, [2009-10-22T22:24:35.399982 #5388] DEBUG --
net.ssh.authentication.methods.publickey[3fa5b5a21044]: publickey
succeeded (86:90:0d:ad:70:44:cd:29:61:24:66:4d:9a:d3:29:0a)

So for some reason, Net::SSH isn't realizing that the public key is an
option.
This topic is locked and can not be replied to.