SSH hangs for 30 sec on AIX

dubstep · November 16, 2011, 1:41pm

Environment:

AIX 6.1
Java:
java version “1.6.0”
Java™ SE Runtime Environment (build pap3260sr9fp1-20110208_03(SR9
FP1))
IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 AIX ppc-32
jvmap3260sr9-20110203_74623 (JIT enabled, AOT enabled)
J9VM - 20110203_074623
JIT - r9_20101028_17488ifx3
GC - 20101027_AA)
JCL - 20110203_01
jruby 1.6.5 (ruby-1.8.7-p330) (2011-10-25 9dcd388) (IBM J9 VM 1.6.0)
[AIX-ppc-java]
Gems:
- bouncy-castle-java (1.5.0146.1)
- jruby-openssl (0.7.4)
- net-ssh (2.2.1)

Code:
require ‘rubygems’
require ‘net/ssh’
host = ‘xxxxxxx.xxxxxx.xxx’
username = ‘xxxx’
Net::SSH.start(host, username, :verbose => :debug) do |ssh|
puts ssh.exec!(“hostname”)
puts ssh.exec!(“ls”)
end

When running this program it hangs for 30 seconds at four points, as I
have shown below. Note that I have omitted a number of lines.

-
-
D, [2011-11-16T11:53:11.477000 #1213352018] DEBUG -- tcpsocket[7d4]:

received packet nr 6 type 52 len 12
D, [2011-11-16T11:53:11.478000 #1213352018] DEBUG –
net.ssh.authentication.methods.publickey[7dc]: publickey succeeded
(2d:32:27:ed:60:c8:28:28:77:50:7d:40:d7:83:ef:4b)
D, [2011-11-16T11:53:11.482000 #1213352018] DEBUG – tcpsocket[7d4]:
queueing packet nr 7 type 90 len 44
<<< Hangs for 30 seconds, then continues >>>
-
-
D, [2011-11-16T11:53:41.573000 #1213352018] DEBUG – tcpsocket[7d4]:
received packet nr 8 type 91 len 28
I, [2011-11-16T11:53:41.577000 #1213352018] INFO –
net.ssh.connection.session[7de]: channel_open_confirmation: 0 0 0 32768
I, [2011-11-16T11:53:41.579000 #1213352018] INFO –
net.ssh.connection.channel[7e0]: sending channel request “exec”
D, [2011-11-16T11:53:41.586000 #1213352018] DEBUG – tcpsocket[7d4]:
queueing packet nr 9 type 98 len 44
<<< Hangs for 30 seconds, then continues >>>
-
-
D, [2011-11-16T11:54:11.556000 #1213352018] DEBUG – tcpsocket[7d4]:
received packet nr 15 type 97 len 12
I, [2011-11-16T11:54:11.558000 #1213352018] INFO –
net.ssh.connection.session[7de]: channel_close: 0
D, [2011-11-16T11:54:11.560000 #1213352018] DEBUG – tcpsocket[7d4]:
queueing packet nr 11 type 97 len 28
<<< Output from hostname command >>>
D, [2011-11-16T11:54:11.563000 #1213352018] DEBUG – tcpsocket[7d4]:
queueing packet nr 12 type 90 len 44
<<< Hangs for 30 seconds, then continues >>>
-
-
D, [2011-11-16T11:54:41.510000 #1213352018] DEBUG – tcpsocket[7d4]:
received packet nr 17 type 91 len 28
I, [2011-11-16T11:54:41.514000 #1213352018] INFO –
net.ssh.connection.session[7de]: channel_open_confirmation: 1 1 0 32768
I, [2011-11-16T11:54:41.515000 #1213352018] INFO –
net.ssh.connection.channel[7e2]: sending channel request “exec”
D, [2011-11-16T11:54:41.518000 #1213352018] DEBUG – tcpsocket[7d4]:
queueing packet nr 13 type 98 len 28
<<< Hangs for 30 seconds, then continues >>>
-
-
<<< Output from ls command >>>
I, [2011-11-16T11:55:11.549000 #1213352018] INFO –
net.ssh.connection.session[7de]: closing remaining channels (0 open)

If I switch to Java 5 on AIX instead, the program runs without problems.
Also, I can run the same program without problems on Windows XP and Java
6 (from Oracle).

Any help in solving this problem would be much appreciated.

Claus

Claus_Folke_B · November 16, 2011, 4:22pm

On Wed, Nov 16, 2011 at 1:41 PM, Claus Folke B. [email protected]
wrote:

J9VM - 20110203_074623

net-ssh (2.2.1)

D, [2011-11-16T11:53:11.482000 #1213352018] DEBUG – tcpsocket[7d4]:
D, [2011-11-16T11:53:41.586000 #1213352018] DEBUG – tcpsocket[7d4]:
<<< Output from hostname command >>>
net.ssh.connection.channel[7e2]: sending channel request “exec”
Also, I can run the same program without problems on Windows XP and Java
6 (from Oracle).

Any help in solving this problem would be much appreciated.

Hmm… The first thing that comes to mind is “reverse lookups”. see
http://rubydoc.info/stdlib/socket/1.9.3/IPSocket#addr-instance_method

You could try strace or truss to see what is different in socket
handling between the two versions of Java which show a difference.

Kind regards

robert

Claus_Folke_B · November 16, 2011, 6:06pm

Having seen this sort of issue many times over the years, it is almost
always caused by problems with name resolution. However, since the
behavior is different with two different versions of Java on the same
system, I’m less confident about the problem being name resolution.

– Matt
It’s not what I know that counts.
It’s what I can remember in time to use.

Claus_Folke_B · November 21, 2011, 2:30pm

Robert K. wrote in post #1032175:

Hmm… The first thing that comes to mind is “reverse lookups”. see
Class: IPSocket — Documentation for socket (1.9.3)

I have verified that reverse lookup is working as it should, so reverse
lookup does not seem to be the problem.

Tried running the same test program on another AIX system, and it shows
the same problem when using Java 6.

I might try to run with the strace/truss but I am not sure I will be
able to use the output.

Anyone out there running AIX, who could try to run the test program
using JRuby and a Java 6 JRE?

Claus

Claus_Folke_B · December 29, 2011, 9:38am

-----Messaggio originale-----
Da: Matt L. [mailto:[email protected]]
Inviato: mercoled 16 novembre 2011 18:06
A: ruby-talk ML
Oggetto: Re: SSH hangs for 30 sec on AIX

Having seen this sort of issue many times over the years, it is almost
always caused by problems with name resolution. However, since the
behavior
is different with two different versions of Java on the same system, I’m
less confident about the problem being name resolution.

– Matt
It’s not what I know that counts.
It’s what I can remember in time to use.

–
Caselle da 1GB, trasmetti allegati fino a 3GB e in piu’ IMAP, POP3 e
SMTP autenticato? GRATIS solo con Email.it http://www.email.it/f

Sponsor:
Conto Arancio al 4,20%. Zero spese e massima liberta’, aprilo in due
minuti!
Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid922&d)-12

Claus_Folke_B · December 29, 2011, 9:20am

-----Messaggio originale-----
Da: Claus Folke B. [mailto:[email protected]]
Inviato: mercoled 16 novembre 2011 13:42
A: ruby-talk ML
Oggetto: SSH hangs for 30 sec on AIX

Environment:

AIX 6.1
Java:
java version “1.6.0”
Java™ SE Runtime Environment (build pap3260sr9fp1-20110208_03(SR9
FP1))
IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 AIX ppc-32
jvmap3260sr9-20110203_74623 (JIT enabled, AOT enabled)
J9VM - 20110203_074623
JIT - r9_20101028_17488ifx3
GC - 20101027_AA)
JCL - 20110203_01
jruby 1.6.5 (ruby-1.8.7-p330) (2011-10-25 9dcd388) (IBM J9 VM 1.6.0)
[AIX-ppc-java]
Gems:
- bouncy-castle-java (1.5.0146.1)
- jruby-openssl (0.7.4)
- net-ssh (2.2.1)

Code:
require ‘rubygems’
require ‘net/ssh’
host = ‘xxxxxxx.xxxxxx.xxx’
username = ‘xxxx’
Net::SSH.start(host, username, :verbose => :debug) do |ssh|
puts ssh.exec!(“hostname”)
puts ssh.exec!(“ls”)
end

When running this program it hangs for 30 seconds at four points, as I
have
shown below. Note that I have omitted a number of lines.

-
-
D, [2011-11-16T11:53:11.477000 #1213352018] DEBUG -- tcpsocket[7d4]:

received packet nr 6 type 52 len 12
D, [2011-11-16T11:53:11.478000 #1213352018] DEBUG –
net.ssh.authentication.methods.publickey[7dc]: publickey succeeded
(2d:32:27:ed:60:c8:28:28:77:50:7d:40:d7:83:ef:4b)
D, [2011-11-16T11:53:11.482000 #1213352018] DEBUG – tcpsocket[7d4]:
queueing packet nr 7 type 90 len 44
<<< Hangs for 30 seconds, then continues >>>
-
-
D, [2011-11-16T11:53:41.573000 #1213352018] DEBUG – tcpsocket[7d4]:
received packet nr 8 type 91 len 28
I, [2011-11-16T11:53:41.577000 #1213352018] INFO –
net.ssh.connection.session[7de]: channel_open_confirmation: 0 0 0 32768
I, [2011-11-16T11:53:41.579000 #1213352018] INFO –
net.ssh.connection.channel[7e0]: sending channel request “exec”
D, [2011-11-16T11:53:41.586000 #1213352018] DEBUG – tcpsocket[7d4]:
queueing packet nr 9 type 98 len 44
<<< Hangs for 30 seconds, then continues >>>
-
-
D, [2011-11-16T11:54:11.556000 #1213352018] DEBUG – tcpsocket[7d4]:
received packet nr 15 type 97 len 12
I, [2011-11-16T11:54:11.558000 #1213352018] INFO –
net.ssh.connection.session[7de]: channel_close: 0
D, [2011-11-16T11:54:11.560000 #1213352018] DEBUG – tcpsocket[7d4]:
queueing packet nr 11 type 97 len 28
<<< Output from hostname command >>>
D, [2011-11-16T11:54:11.563000 #1213352018] DEBUG – tcpsocket[7d4]:
queueing packet nr 12 type 90 len 44
<<< Hangs for 30 seconds, then continues >>>
-
-
D, [2011-11-16T11:54:41.510000 #1213352018] DEBUG – tcpsocket[7d4]:
received packet nr 17 type 91 len 28
I, [2011-11-16T11:54:41.514000 #1213352018] INFO –
net.ssh.connection.session[7de]: channel_open_confirmation: 1 1 0 32768
I, [2011-11-16T11:54:41.515000 #1213352018] INFO –
net.ssh.connection.channel[7e2]: sending channel request “exec”
D, [2011-11-16T11:54:41.518000 #1213352018] DEBUG – tcpsocket[7d4]:
queueing packet nr 13 type 98 len 28
<<< Hangs for 30 seconds, then continues >>>
-
-
<<< Output from ls command >>>
I, [2011-11-16T11:55:11.549000 #1213352018] INFO –
net.ssh.connection.session[7de]: closing remaining channels (0 open)

If I switch to Java 5 on AIX instead, the program runs without problems.
Also, I can run the same program without problems on Windows XP and Java
6 (from Oracle).

Any help in solving this problem would be much appreciated.

Claus

–
Posted via http://www.ruby-forum.com/.

–
Caselle da 1GB, trasmetti allegati fino a 3GB e in piu’ IMAP, POP3 e
SMTP autenticato? GRATIS solo con Email.it http://www.email.it/f

Sponsor:
Capodanno a Riccione, Pacchetto Relax: Mezza Pensione + bagno turco +
solarium + massaggio. Wifi e parcheggio gratis. 2 giorni euro 199 a
persona
Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid977&d)-12