[rancid] xrrancid vs. IOS-XR on Cisco ASR9000 (ASR9K)

[Alan McKinnon]

On 09/10/2014 10:01, Buchan Milne wrote:
> We have been using rancid successfully with IOS- and IOS-XE-based
> devices for some time, and we haven't had any real problems.
> 
> However, we have replaced some of these devices with Cisco ASR9000s, and
> have had trouble with rancid on these. At present we are running IOS-XR
> 4.3.1 on most, but we have a few running 5.1.2.
> 
> The problem seems to be that xrrancid doesn't realise it has retrieved
> all the results of the commands, and thus times out waiting for more
> data.
> 
> If I manually run xrrancid, this still happens:
> 
> $ time xrrancid -d $device
> executing clogin -t 90 -c"terminal no-timestamp;terminal exec prompt
> no-timestamp;admin show version;admin show install summary;admin show
> license;admin show variables boot;show redundancy secondary;show install
> active;admin show env all;dir /all nvram:;dir /all bootflash:;dir /all
> compactflash:;dir /all compactflasha:;dir /all slot0:;dir /all
> disk0:;dir /all disk0a:;dir /all slot1:;dir /all disk1:;dir /all
> disk1a:;dir /all slot2:;dir /all disk2:;dir /all harddisk:;dir /all
> harddiska:;dir /all harddiskb:;dir /all slavenvram:;dir /all
> slavebootflash:;dir /all slaveslot0:;dir /all slavedisk0:;dir /all
> slaveslot1:;dir /all slavedisk1:;dir /all slaveslot2:;dir /all
> slavedisk2:;dir /all sec-nvram:;dir /all sec-bootflash:;dir /all
> sec-slot0:;dir /all sec-disk0:;dir /all sec-slot1:;dir /all
> sec-disk1:;dir /all sec-slot2:;dir /all sec-disk2:;show
> controllers;admin show running;admin show diag;admin show inventory
> raw;show vlan;show debug;show rpl maximum;show running-config" $device
> 
> ...
> 
> $device: missed cmd(s): show running-config,dir /all slot0:,admin show
> env all,show debug,show vlan,show rpl maximum
> $device: missed cmd(s): show running-config,dir /all slot0:,admin show
> env all,show debug,show vlan,show rpl maximum
> $device: End of run not found
> $device: End of run not found
> !
> 
> real    3m46.481s
> user    0m0.900s
> sys     0m0.120s
> 
> 
> Naturally, if I use clogin and run the commands manually, they complete.
> 
> How can I troubleshoot this further?
> 
> This far I have been using the distro-supplied tcl/expect packages in
> production, but I have applied the patches on my workstation, but they
> made no difference.
> 
> On my workstation I am using the 2.3.8-1 package I created for Mageia:
> http://svnweb.mageia.org/packages/cauldron/rancid/current/
> 
> (I have a 3.1 build, but need to consider whether I should add
> post-upgrade scripts to change : to ; in router.db files - the behaviour
> above was the same with this package)
> 
> Incidentally, I have in the past done some terminal-scraping with
> IPC::Run, is there a reason why rancid still uses expect, when it might
> be more reliable to handle this in the rancid perl script?
> 
> Regards,
> Buchan

Hi Buchan

[ it's been a while :-) ]

You're on the right track, rancid is not detecting the start of a
command in the output (or perhaps not detecting the end of the previous
command).

Ideally I'd need the entire output of an xrrancid -d run on a failing
device, but that's not usually feasible, let's try this instead:

For each failing command, grab 5 or so lines of text from the output of
xrrancid -d above and below where clogin runs that command. There will
be something there that confuses the prompt detection code:

There's a regex that searches for /^$prompt/ and it gets easily
confused. I well recall one case with IS's XRs where one of the admin
show commands output wasn't ending properly with a newline making the
regex fail. Those are the kinds of oddities you need to look for.

There's also the old classic problem of having "#" or ">" characters in
your login banner, that one causes the entire run to fail.





-- 
Alan McKinnon
alan.mckinnon at gmail.com