[rancid] clogin and rancid good, rancid-run fails

[Ken Celenza]

> Sent: Tuesday, October 27, 2015 at 3:32 PM
> From: "Ken Celenza" <ken.celenza at mail.com>
> To: rancid-discuss at shrubbery.net
> Subject: Re: [rancid] clogin and rancid good, rancid-run fails
>
> 
> 
> > Sent: Tuesday, October 27, 2015 at 2:48 PM
> > From: "Jethro R Binks" <jethro.binks at strath.ac.uk>
> > To: rancid-discuss at shrubbery.net
> > Subject: Re: [rancid] clogin and rancid good, rancid-run fails
> >
> > On Tue, 27 Oct 2015, Ken Celenza wrote:
> > 
> > > > Sent: Tuesday, October 27, 2015 at 8:35 AM
> > > > From: "Alex DEKKER" <rancid at ale.cx>
> > > >
> > > > Can you SSH onto them from that box without any special parameters to 
> > > > SSH? ISTR recent-ish versions of OpenSSH deprecating the algorithms [or 
> > > > the default key size, perhaps?] used by older IOS, which means you have 
> > > > to add some -o option to make it work.
> > > > 
> > > > alexd
> > > 
> > > I think this is it. It's still weird that it works fine with ./rancid 
> > > but not ./rancid-run. That being said, I turned on telnet, it worked 
> > > fine, and I got a list of the packages that were updated. No changes to 
> > > perl or expect, but openssh was updated and I found this.
> > 
> > Holy Batman;
> > 
> > I've had a problem with a couple of systems for a while which I've only 
> > half-heartedly looked at, and then when I set them to 'down' forgot about 
> > completely for a while more.
> > 
> > But inspired by the above comments, I tested each of /usr/bin/ssh and 
> > /usr/local/bin/ssh, and the latter works but the former does not.  This 
> > explains why, like one of the OPs, rancid-run on the command-line worked, 
> > but not when run from cron - a variant of the usual reason, that the 
> > environment is different (in this case, $PATH).
> > 
> > I changed the order in the PATH in rancid.conf, and now it can connect to 
> > the systems concerned (and I see form the diffs that they started to fail 
> > after an update that changed some SSL/TLS settings).
> > 
> > The system /usr/bin/ssh was giving the following error:
> > 
> > no matching cipher found: client aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc at lysator.liu.se server aes128-ctr,aes192-ctr,aes256-ctr
> > 
> > Unfortunately his never made it to a rancid logfile that I could see so I 
> > was completely in the dark.  Is there any way that ssh errors like this 
> > could be caught and logged?
> > 
> > Happy Jethro.
> > 
> > .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .
> > Jethro R Binks, Network Manager,
> > Information Services Directorate, University Of Strathclyde, Glasgow, UK
> > 
> > The University of Strathclyde is a charitable body, registered in
> > Scotland, number SC015263.
> > _______________________________________________
> > Rancid-discuss mailing list
> > Rancid-discuss at shrubbery.net
> > http://www.shrubbery.net/mailman/listinfo/rancid-discuss
> > 
> 
> 
> Brilliant!! So yes, I can confirm when running ssh from /usr/bin it fails, when I run the ssh I have it works no problem. Now what's still weird is my $path it shows /usr/bin second, but when I run it via rancid-run, it comes up first and fails, not exactly sure why. I was able to confirm this by monitoring my processes spawning with "strace -feprocess $SHELL" 
> 
> I saw this:
> [pid  6384] execve("/src/rancid/rancid/bin/ssh", ["ssh", "-c", "3des", "-x", "-l", "user", "device", "-o", "UserKnownHostsFile=/dev/null", "-o", "StrictHostKeyChecking=no"], [/* 68 vars */]) = -1 ENOENT (No such file or directory)
> [pid  6384] execve("/src/rancid/rancid//ssh", ["ssh", "-c", "3des", "-x", "-l", "user", "device", "-o", "UserKnownHostsFile=/dev/null", "-o", "StrictHostKeyChecking=no"], [/* 68 vars */]) = -1 ENOENT (No such file or directory)
> [pid  6384] execve("/usr/bin/ssh", ["ssh", "-c", "3des", "-x", "-l", "user", "device", "-o", "UserKnownHostsFile=/dev/null", "-o", "StrictHostKeyChecking=no"], [/* 68 vars */]) = 0
> [pid  6384] arch_prctl(ARCH_SET_FS, 0x7fc8024117c0) = 0
> [pid  6384] exit_group(255)             = ?
> Process 6384 detached
> [pid  6383] --- SIGCHLD (Child exited) @ 0 (0) ---
> [pid  6383] wait4(6384, [{WIFEXITED(s) && WEXITSTATUS(s) == 255}], 0, NULL) = 6384
> [pid  6383] clone(Process 6387 attached
> child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f4eddb709d0) = 6387
> [pid  6387] --- SIGWINCH (Window changed) @ 0 (0) ---
> [pid  6387] clone(Process 6388 attached
> child_stack=0, flags=CLONE_PARENT_SETTID|SIGCHLD, parent_tidptr=0x7ffe9f44aeb8) = 6388
> 
> In reference to: 
> """  This explains why, like one of the OPs, rancid-run on the command-line worked, but not when run from cron - a variant of the usual reason, that the environment is different (in this case, $PATH). """
> 
> Actually didn't work via command line or cron. 
> _______________________________________________
> Rancid-discuss mailing list
> Rancid-discuss at shrubbery.net
> http://www.shrubbery.net/mailman/listinfo/rancid-discuss
> 

Just to finalize this, my /usr/bin/ssh was downgraded and now everything is working fine. I'm still perplexed as to why it didn't take my $PATH ordering into account. 

Just something to keep in mind if people are having similar issue in the future.