[rancid] RANCiD with Fortinet FortiGate firewalls and cfg-save revert

[heasley]

Thu, Feb 27, 2025 at 12:02:26PM -0500, Gary T. Giesen:
> We were recently troubleshooting an issue with our deployed Fortinet Fortigate firewalls and noticed that they're rebooting every night. The reboot was quick enough that it wasn't being picked up by our monitoring system (which polls every 5 minutes), and tracked down the issue to RANCiD. We set up remote syslogging and were able to glean this from the logs:
>  
> Feb 27 03:14:47 fortigate date=2025-02-26 time=22:02:51 devname="fortigate" devid="FG40FITK00000001" eventtime=1740625371766672700 tz="-0500" logid="0100032002" type="event" subtype="system" level="alert" vd="root" logdesc="Admin login failed" sn="0" user="rancid" ui="ssh(192.0.2.10)" method="ssh" srcip=192.0.2.10 dstip=198.51.100.10 action="login" status="failed" reason="ssh_key_invalid" msg="Administrator rancid login failed from ssh(192.0.2.10) because of invalid ssh key"
> Feb 27 03:14:49 fortigate date=2025-02-26 time=22:02:53 devname="fortigate" devid="FG40FITK00000001" eventtime=1740625372916992740 tz="-0500" logid="0100032001" type="event" subtype="system" level="information" vd="root" logdesc="Admin login successful" sn="1740625372" user="rancid" ui="ssh(192.0.2.10)" method="ssh" srcip=192.0.2.10 dstip=198.51.100.10 action="login" status="success" reason="none" profile="super_admin" msg="Administrator rancid logged in successfully from ssh(192.0.2.10)"
> Feb 27 03:14:49 fortigate date=2025-02-26 time=22:02:53 devname="fortigate" devid="FG40FITK00000001" eventtime=1740625373254849640 tz="-0500" logid="0100044546" type="event" subtype="system" level="information" vd="root" logdesc="Attribute configured" user="rancid" ui="ssh(192.0.2.10)" action="Edit" cfgtid=1982529536 cfgpath="system.console" cfgattr="output[more->standard]" msg="Edit system.console "
> Feb 27 03:14:56 fortigate date=2025-02-26 time=22:03:00 devname="fortigate" devid="FG40FITK00000001" eventtime=1740625380414160400 tz="-0500" logid="0100032003" type="event" subtype="system" level="information" vd="root" logdesc="Admin logout successful" sn="1740625372" user="rancid" ui="ssh(192.0.2.10)" method="ssh" srcip=192.0.2.10 dstip=198.51.100.10 action="logout" status="success" duration=8 state="Config-Changed" reason="exit" msg="Administrator rancid logged out from ssh(192.0.2.10)"
> Feb 27 03:14:56 fortigate date=2025-02-26 time=22:03:00 devname="fortigate" devid="FG40FITK00000001" eventtime=1740625380414186840 tz="-0500" logid="0100032102" type="event" subtype="system" level="alert" vd="root" logdesc="Configuration changed" user="rancid" ui="ssh(192.0.2.10)" msg="Configuration is changed in the admin session"
> ...
> Feb 27 03:44:52 fortigate date=2025-02-26 time=22:32:56 devname="fortigate" devid="FG40FITK00000001" eventtime=1740627176128914740 tz="-0500" logid="0100036881" type="event" subtype="system" level="notice" vd="root" logdesc="Configuration reverted due to timeout" msg="Configuration reverted due to cfg-revert-timeout reached"
>  
> If the fortigate script is anything like the cisco ones, I assume rancid is sending some commands to disable paging, the fortigate detects this as a config change. Our Fortigates have cfg-save revert set, which auto-reverts the config because it's not being saved (which involves rebooting the device).

That is correct, it sends "config global".  Is there another way to disable
the pager?

This discussion from 2023 confirms that there was no way at the time:
https://community.fortinet.com/t5/Support-Forum/Change-pagination-option-without-configuration-change/m-p/268835
maybe use your support contract to ask for a command to disable the
pager per-session.

A user there did suggest a hack of piping commands to grep.  Maybe 'cat' is
one of the offered utilities?  That might be usable for fnlogin, but would
require some code to handle user input.

Maybe causing ssh to pass an insanely high number of rows in the terminal
attributes.  Again, not the cleanest solution.

Doesn't support netconf or gnmi.

> Has anyone dealt with this issue with RANCiD and cfg-save revert on Fortinet FortiGate firewalls? Is there any solution other than to just disable cfg-save revert (by setting it to automatic or manual).

No one has mentioned this behavior before, but I do not know how commonly
this 'cfg-save revert' knob is configured.  If it is common, maybe you are
using a feature that causes this reboot or have encountered a bug.