[rancid] Newline issue

[heasley]

Mon, Jan 30, 2017 at 08:51:53PM +0000, Merijn Evertse:
> Hello,
> 
> I have run it 20 times on 2 different Fortigate units and it never happens.
> I have run the normal rancid-run 1 time and it happened on 2 units.
> Always the same lines, around line 138 or 140 on every Fortigate device.
> 
> Merijn Evertse

hrm, lack of a tty - probably.  Are you absolutely positive that the pager
is being disabled?

if so, I suppose this process has to be repeated for the cron & rancid-run
process.  set NOPIPE=YES in rancid.conf and change rancid.types.base:
-fortigate;script;fnrancid
+fortigate;script;fnrancid -d

and run rancid-run or let cron run.  I expect this to leave the .raw file
behind as below.

> -----Oorspronkelijk bericht-----
> Van: heasley [mailto:heas at shrubbery.net] 
> Verzonden: maandag 30 januari 2017 21:28
> Aan: Merijn Evertse <merijn at trans-ix.nl>
> CC: heasley <heas at shrubbery.net>; rancid-discuss at shrubbery.net
> Onderwerp: Re: [rancid] Newline issue
> 
> Mon, Jan 30, 2017 at 08:04:24PM +0000, Merijn Evertse:
> > Hello,
> > 
> > Sorry it seems this has not happened for quite some time, so you can 
> > disregard the mentioning of Cisco :-)
> 
> ok; then as the rancid user, please repeatedly run
> 
> 	export NOPIPE=YES
> 	fnrancid -dl hostname
> 
> until you see the problem in the hostname.new file.  Then provide the hostname.raw file to me as an attachment.  Hopefully I can identify the cause from that.
> 
> > Merijn Evertse
> > 
> > -----Oorspronkelijk bericht-----
> > Van: heasley [mailto:heas at shrubbery.net]
> > Verzonden: maandag 30 januari 2017 20:39
> > Aan: Merijn Evertse <merijn at trans-ix.nl>
> > CC: heasley <heas at shrubbery.net>; rancid-discuss at shrubbery.net
> > Onderwerp: Re: [rancid] Newline issue
> > 
> > Mon, Jan 30, 2017 at 02:47:40PM +0000, Merijn Evertse:
> > > Hello,
> > > 
> > > Thank you for the response.
> > > I checked the commands for disabling pager and they work. I also verified that debugging is disabled and is not logging to console.
> > > Also the issue is not exactly every hour, some runs go without problems and some have this issue.
> > > 
> > > Merijn Evertse
> > 
> > Can you show us an example from a cisco that is doing this?
> > 
> > > -----Oorspronkelijk bericht-----
> > > Van: heasley [mailto:heas at shrubbery.net]
> > > Verzonden: donderdag 26 januari 2017 16:24
> > > Aan: Merijn Evertse <merijn at trans-ix.nl>
> > > CC: rancid-discuss at shrubbery.net
> > > Onderwerp: Re: [rancid] Newline issue
> > > 
> > > Thu, Jan 26, 2017 at 08:42:35AM +0000, Merijn Evertse:
> > > > Hello,
> > > > 
> > > > The rancid run on our Fortigate units has a small issue. Every hour or so we get the following:
> > > > 
> > > > -     set two-factor-ftm-expiry 72
> > > > 
> > > > +     set two-fact
> > > > 
> > > > + or-ftm-expiry 72
> > > > 
> > > > Or
> > > > 
> > > > -     set virtual-server-hardware-acceleration enable
> > > > 
> > > > +     set vi
> > > > 
> > > > + rtual-server-hardware-acceleration enable
> > > 
> > > This is most likely either the pager or some out-of-band like logs being written to the vty while the config is being collected.  fnlogin should be disabling the pager; verify that your aaa configuration is allowing it to do so.
> > > 
> > > > I also see this sometimes on other devices, Cisco, Vyos etc.
> > > > It seems that something goes wrong reading the lines and the code sees a newline where it shouldn't.
> > > 
> > > vyos is not supported by rancid and i've never used it, but it is probably the same cause.  For the cisco too, and that is almost definitely your aaa not allowing it to disable the pager; but there are a few cisco platforms that have no way to disable the pager.
> > > 
> > > > On Vyos it looks like:
> > > > 
> > > > - # chassis_asset_tag: show hardware pci
> > > > 
> > > > - # 3091-1101-9766-6577-4039-0090-06
> > > > 
> > > > + # chassis_asset_tag: 3091-1101-9766-6577-4039-0090-06
> > > > 
> > > > Related one hour earlier:
> > > > 
> > > > - # chassis_asset_tag: 3091-1101-9766-6577-4039-0090-06
> > > > 
> > > > + # chassis_asset_tag: show hardware pci #
> > > > 
> > > > + 3091-1101-9766-6577-4039-0090-06
> > > > 
> > > > And here the command itself is shown.
> > > > 
> > > > Kind regards,
> > > > 
> > > > Merijn Evertse
> > > 
> > > > _______________________________________________
> > > > Rancid-discuss mailing list
> > > > Rancid-discuss at shrubbery.net
> > > > http://www.shrubbery.net/mailman/listinfo/rancid-discuss