[rancid] Script help / pointers

[AJ Schroeder]

Wow, thank you for the detailed reply! I wasn't really clear on *how* the scripts all flowed from beginning to end until I read this.  

In this case, Nortel seems to have a penchant for being different (read: difficult) in the way that there is not an "exit" in the config that I can key in on to tell the script that "show run" is completed. However, I think the way to deal with this is to look for the prompt ending with the '#' character and that will tell the script when a particular "&{$commands{$cmd}}" is done. The one good thing is that the term length can be set per session by the rancid script so I don't have to deal with how Nortel paginates the output, I should always get a prompt back whenever a command is done outputting to the session.

AJ


-----Original Message-----
From: Rancid-discuss [mailto:rancid-discuss-bounces at shrubbery.net] On Behalf Of Alan McKinnon
Sent: Tuesday, November 03, 2015 1:52 PM
To: rancid-discuss at shrubbery.net
Subject: Re: [rancid] Script help / pointers

On 03/11/2015 18:37, AJ Schroeder wrote:
> Hello list,
> 
>  
> 
> A while ago I hacked together a couple of scripts to get some Nortel 
> and Avaya devices backing up. I basically copied some of the existing 
> scripts to make things work, however the issue is that they “just work”.
> We are going to be supporting these devices for the foreseeable future 
> so I’d like to improve the scripts from their current form.
> 
>  
> 
> Specifically, I am having a hard time processing the logouts properly 
> because I am not sure what the scripts are looking for to say “I’m 
> done with show run, continue on”. I have gotten things to work by just 
> adding a logout command to the run_commands sub in the nortellogin, 
> but I feel this is not the most graceful way of exiting. I believe not 
> being able to process things is causing backups to take longer than it should.
> 
>  
> 
> The scripts are located here:
> 
>  
> 
> https://github.com/ajschroeder/rancid/blob/master/nortelrancid
> 
> https://github.com/ajschroeder/rancid/blob/master/nortellogin
> 
>  
> 
> If anyone has time to help or critique it would be greatly appreciated.


I haven't looked at your scripts yet (might do so later), but most rancid parser scripts work basically the same way. Most devices with a sane Unix-like cli behave much like a telnet session - there's a login process and when that completes there's a prompt. These two are in that order, and the prompt follows a pattern - text followed by a ">" or "#".
This can be quite reliably found with a regex so the script knows what it looks like. Then there is a sequence of commands run by the script itself which are echoed to the terminal along with the command output.
The script sees a prompt, followed by a command (that it defined and
knows) so it passes all text received until the next prompt to the sub defined for that command. The commands are all in @commandtable.

The main loop looks like this:

TOP: while(<INPUT>) {

    if (/[>#]\s?exit$/) {
        last;
    }
    while (/[>#]\s*($cmds_regexp)\s*$/) {
        $cmd = $1;
        if (!defined($prompt)) {
            $prompt = ($_ =~ /^([^#>]+[#>])/)[0];
            $prompt =~ s/([][}{)(\\])/\\$1/g;
        }
        if (! defined($commands{$cmd})) {
            last TOP;
        }
        $rval = &{$commands{$cmd}};		<==== magic goes here
        delete($commands{$cmd});
        if ($rval == -1) {
            last TOP;
        }
    }
}

It's all driven by the line marked "magic goes here" and @commandtable is an array of hashes. Each hash contains the router command as the name and the sub that processes the output as the value. The call "&{$commands{$cmd}}" calls the sub, which runs until it finds a prompt, whereupon the main loop above continues. The whole thing keeps going round and round till the main loop finds the command "exit" issued - this ends the loop and the script terminate. rancid-run then carries on with the next item in router.db. It's all very simple, logical classic Perl, and TheSimplestThingThatCouldPossiblyWork which probably explains why rancid is still here after so many years, why every ISP I know of uses it, and why every competitor seems to fall by the wayside.

To set things up for a new device type, you need to research the device first. Define exactly what it does with prompts and echoing of command and output, and how you log out.

The one important bit that isn't in the rancid parser script is how to logout/exit - that is in clogin and can differ a lot between device types. This is why so often you'll find a new type called say xyzrancid with a corresponding modified xyzlogin. Modifying the parser script often involves ripping out how chunks of each sub that simply do not apply to your device. Knowledge of IOS helps here so you can tell what the Cisco parser is doing and adapt it to what your device does.

That's it really, truthfully it's lots of common sense and donkey work.
To see how fast your parser runs, call it directly in debug mode and copy the clogin command echoed at the beginning. Run that command directly and observe what happens. You should be familiar with your devices enough to know if it's moving along at the right speed.

The last thing is if you are worried about processing time, don't be afraid to crank PAR_COUNT waaaaaaaay up. This is 2015 and 1995 called - they want their Pentium ones back. Modern hosts, even small VMs, cope very nicely with 100 telnet sessions with ease (they are idle most of the time anyway). By example, I found a teeny little VM with 1 cpu and
512 MB on an over-committed ESXi host levelled out at about 80 parallel processes. The size of command output does not affect rancid at all.
Whether it's a 100 line show run or a huge beast from a 9K, rancid still chugs along processing one line at a time till it finds a logout.


--
Alan McKinnon
alan.mckinnon at gmail.com

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss at shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss