[rancid] rancid suddenly stopped backing up devices

[heasley]

Wed, Oct 05, 2022 at 03:21:42PM +0000, Adam Thompson:
> OK!  Having  clogin work proves that RANCID can connect to the host and (I assume?) authenticate.

yes.

> No change in the SVN/GIT/whatever-you-use could mean that either RANCID wasn’t able to retrieve the configuration, wasn’t able to parse the configuration, or wasn’t able to check in the new version to the repo.

if it were unable to commit the new version, you would see the same diffs
repeatedly.

I would review the logs for the group.  if nothing there indicates the
problem, try the clogin tests outlined in the FAQ.  If these work, verify
that the entry in the router.db is correct and the device is marked as 'up';
it would appears in the routers.up file too.

other useful tests are running rancid manually, like

	rancid -t <type> -dl <devicename>

that shows misc parsing & logs, the commands run & filters called, the 
exit value, and leaves <devicename>.{raw,new} output files.

then
	rancid -t <type> -dl -f <devicename>.raw
can be used for further debugging that I would not expect to be useful to
most.


> Unfortunately, I’ve found RANCID’s own debugging output to be either nonexistent or unhelpful.  I have run into a very similar situation in the past, but sadly I don’t remember what the problem was.  I do remember I had to use system tools to find out what was wrong:
> 
> If you’re running on Linux, you have the strace(1) tool; on OpenBSD, you’ve got ktrace(1); on Solaris you’ve got truss(8); other UNIXes will have roughly-equivalent tools – all these are similar in that they will trace system calls for you and produce (somewhat-) meaningful output that may help you identify where the failure occurs.  E.g. if it’s a permissions error, the strace(8) output may show you the EPERM error message, and if you look carefully, the path to the file in question.  Generally speaking, the strace/ktrace/truss/etc. output will show a failure somewhere in the later part of RANCID’s execution that you can use as a starting point.

what did you find to be missing?  Tell me so that I can try to improve the
debug output.

> Dear All,
> 
> I had rancid installed couple of months back and working fine with no issues
> i have rancid version 3.9 with viewvc 1.3.0
> 
> I just realized that any changes to most devices are not backed up.
> there is no update in revison version neither I receive mail
> It stopped on sept 21 2022
> attached a screen shot of the firewall
> 
> changes to above device is done almost daily.
> i did run clogin deviceip and it logs in succesfully
> 
> appreciate if someone can advise and help in troubleshooting the issue
> I checked the logs but there is no clue
> 
> Thanks and regards
> 
> simon