[rancid] Rancid isn't scanning the f5s from cron job.

[Matt Almgren]

I could see that being a problem if it was an all-or-nothing type of scenario.  But in my situation – 40 devices are working just fine.  But 4 are not.  And the only thing they have in common is that they are f5.    Are you saying this is specific to the f5rancid script?

I’m trying to figure out if this has any thing to do with it "/config/ssl/ssl.key” that the log mentions?

I’ll try the remove environment test and see what happens, but if that’s it – how do I fix it for cron?

 — Matt


From: Ehud Gavron <gavron at wetwork.net<mailto:gavron at wetwork.net>>
Date: Monday, July 13, 2015 at 12:51 PM
To: Matt Almgren <matta at surveymonkey.com<mailto:matta at surveymonkey.com>>, "rancid-discuss at shrubbery.net<mailto:rancid-discuss at shrubbery.net>" <rancid-discuss at shrubbery.net<mailto:rancid-discuss at shrubbery.net>>
Subject: Re: [rancid] Rancid isn't scanning the f5s from cron job.

I've found that some of the older rancid scripts work great from a terminal
logged in with an environment, but fail when run by cron.

Specifically I have ensured that $HOME is defined for all rancid jobs, and
have also set the terminal type.

Debugging: in f5rancid, right after the creation of $host.new and $host.raw
preserve those files, by adding
   cp $host.* /tmp/

That will give you a very good indication of
- whether the login succeeded
- what commands were sent
- what commands got an expected response
- what commands broke the script

Note: if you want to try and figure out if it's an environmental variable,
then before doing rancid-run clear all your environmental variables and
try rancid-run then.  If it fails, add them back in to figure out what it
needs.

Ehud

On 07/13/2015 10:31 AM, Matt Almgren wrote:
Very strange behavior here that I can’t pinpoint.
When the cron job runs once per day, it doesn’t scan our f5s in multiple locations.  But when I run “rancid-run” manually (with no
args) it picks them up – no problem.
Cron: 59 9 * * * /usr/lib/rancid/bin/rancid-run
router.db:
<snip>
mt-lr01.endor.lan:juniper:up
mt-lb03.endor.lan:f5:up
mt-lb04.endor.lan:f5:up
<snip>
Files:
-rw-r----- 1 rancid rancid 1072279 Jul  8 11:36 mt-lb04.endor.lan
-rw-r----- 1 rancid rancid 1072475 Jul  8 11:36 mt-lb03.endor.lan
-rw-r----- 1 rancid rancid   22647 Jul 10 10:01 mt-er01.endor.lan
-rw-r----- 1 rancid rancid   22211 Jul 10 10:01 mt-er02.endor.lan
Again, running manually, it works just fine.
The logs from the cron job show me what I already know:
Trying to get all of the configs.
mt-lb03.corp.surveymonkey.com: missed cmd(s): ls --full-time --color=never /config/ssl/ssl.crt,ls --full-time --color=never
/config/ssl/ssl.key
mt-lb04.corp.surveymonkey.com: missed cmd(s): ls --full-time --color=never /config/ssl/ssl.crt,ls --full-time --color=never
/config/ssl/ssl.key
=====================================
Getting missed routers: round 1.
mt-lb03.corp.surveymonkey.com: missed cmd(s): ls --full-time --color=never /config/ssl/ssl.crt,ls --full-time --color=never
/config/ssl/ssl.key
mt-lb04.corp.surveymonkey.com: missed cmd(s): ls --full-time --color=never /config/ssl/ssl.key,ls --full-time --color=never
/config/ssl/ssl.crt
=====================================
Getting missed routers: round 2.
mt-lb04.corp.surveymonkey.com: missed cmd(s): ls --full-time --color=never /config/ssl/ssl.crt,ls --full-time --color=never
/config/ssl/ssl.key
mt-lb03.corp.surveymonkey.com: missed cmd(s): ls --full-time --color=never /config/ssl/ssl.crt,ls --full-time --color=never
/config/ssl/ssl.key
=====================================
Getting missed routers: round 3.
mt-lb04.corp.surveymonkey.com: missed cmd(s): ls --full-time --color=never /config/ssl/ssl.key,ls --full-time --color=never
/config/ssl/ssl.crt
mt-lb03.corp.surveymonkey.com: missed cmd(s): ls --full-time --color=never /config/ssl/ssl.key,ls --full-time --color=never
/config/ssl/ssl.crt
=====================================
Getting missed routers: round 4.
mt-lb03.corp.surveymonkey.com: missed cmd(s): ls --full-time --color=never /config/ssl/ssl.key,ls --full-time --color=never
/config/ssl/ssl.crt
mt-lb04.corp.surveymonkey.com: missed cmd(s): ls --full-time --color=never /config/ssl/ssl.crt,ls --full-time --color=never
/config/ssl/ssl.key
But when I run it manually:
Trying to get all of the configs.
All routers sucessfully completed.
Anyone have any clues/ideas?
Thanks, Matt
_______________________________________________
Rancid-discuss mailing list
Rancid-discuss at shrubbery.net<mailto:Rancid-discuss at shrubbery.net>
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.shrubbery.net/pipermail/rancid-discuss/attachments/20150713/9085c343/attachment.html>