[rancid] Parrallel Execution of Rancid

[Alan McKinnon]

On 28/11/2013 18:13, Callum Scott wrote:
> Hi All,
> 
> I've been searching around for an answer to this and can't seem to find
> anything.
> 
> My organisation currently has a large number of sites with a varying
> number of network devices each of which we would like to back up with
> rancid.  Im using dotwaffle's patched version to include git support,
> though this shouldn't make a difference for this issue.
> 
> Because I am sorting by SITE I have over 1000 rancid groups. Some have
> only 5 networking devices, whilst others will have tens of networking
> devices.
> 
> My problem is that the rancid-run is taking a massive amount of time (in
> the order of days).  I have tried playing around with the PAR_COUNT to
> increase concurrency and also reduce the MAX_ROUNDS to reduce the amount
> of time spent on each device.  It looks to me that the PAR_COUNT only
> comes in during the execution of the control_rancid script, which means
> concurrency only hits in within the GROUP.
> 
> I'd like a way to run rancid_run on the groups in parallel.  Am I
> missing something obvious here?  Does anyone have any ideas on how to
> achieve this?

Per the man page:

rancid-run <groupname>

You could organize your crontabs so that several instances of rancid-run
run in parallel, each with their own unique list of groups that are
handled in series. With 1000 groups this could get interesting.

Or you could write a wrapper script that tries to do something
intelligent. The main information you need in LIST_OF_GROUPS in
rancid.conf and the number of up devices in each router.db

AFAIK such a script isn't out there yet.

Also, have you considered consolidating your devices somehow to reduce
the number of groups? I don't know how you separate things out but if
you are doing it by physical location, would it not work if you grouped
them by town, state or even continent?


-- 
Alan McKinnon
alan.mckinnon at gmail.com