[tac_plus] Two TACACS+ server and primary one is always busy

[Asif Iqbal]

On Wed, Jul 10, 2013 at 5:00 PM, Alan McKinnon <alan.mckinnon at gmail.com>wrote:

> On 10/07/2013 19:55, Asif Iqbal wrote:
> > Hi All
> >
> > We have two TACACS+ server and only one of them is heavily loaded.
> >
> > What is the best practice on balancing the load. Once in a while we
> > need to restart tacacs+ since the CPU usage goes over 50%, on the primary
> > server
> > while the secondary one is almost idle.
> >
> > We are using x2270 servers and they are 4G each with 2 Intel 2.00GHz
> > Quad-Core Xeon E5504
> > on each.
> >
> > I see about 31 tac_plus running on primary, while secondary one has just
> 1.
> >
> >
> > Thanks
> >
>
>
> Hi Asif,
>
> Before doing anything else, you need to sort out those cpu load numbers
> as they should not be anywhere near that level. For a point of
> reference, I have 3 main tacacs servers, they do about 1800 requests
> (login and command in total) a minute, and one of them takes about half
> that load. Occasionally the munin graph creeps above 1% or 2% and that's
> an oldish Dell dual core.
>
> 50% load on your hardware spells something badly wrong and in my
> experience that behaviour with tcp connections is almost always IO
> blocking.
>

I usually restart the tac_plus and that fixes it immediately. That sounds
like
a memory leak. How to find out the total memory usage for tac_plus? VSZ or
RSS count of 25 threads is not it.



> Do you do per-device controls in your tac_plus.conf somehow? Do you need
> to do DNS lookups for this, and is your DNS setup fast and reliable?
>

No per device config. Yes using the -L and DNS cache is running with
dnscache
for local lookup and that is fast. We need that to co-relate the events for
cisco
syslog and AAAs in splunk for reporting.



> What are the hash types you use for your passwords and is it a method
> that be hashed quickly by the OS
>
>
using PAM -> AD.



> Those would be the first thing I'd look at. Second is to post your
> tac_plus.conf. there aren't really any best practices as such for this,
> tac_plus is more than adequate to deal with just about any realistic
> scenario so the "best practice" is whatever works for you and gives
> *you* the control *you* need.
>


Need to sanitize a lot before posting it, but I have 31 group stanzas, 1325
user stanzas,
19 acl stanzas and some of those acls have about 130 permit lines.

currently I have 24 tac_plus instance running like below

$ ps -e -o pid,ppid,vsz,rss,cmd | grep tac_pl[u]s
 4692     1  78296 53708 /usr/local/bin/tac_plus -L -B 192.168.6.20 -l
/var/log/tacacs.daemon.log -C /etc/tacacs.conf
27276  4692  78296 53340 /usr/local/bin/tac_plus -L -B 192.168.6.20 -l
/var/log/tacacs.daemon.log -C /etc/tacacs.conf



> --
> Alan McKinnon
> alan.mckinnon at gmail.com
>
> _______________________________________________
> tac_plus mailing list
> tac_plus at shrubbery.net
> http://www.shrubbery.net/mailman/listinfo.cgi/tac_plus
>



-- 
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.shrubbery.net/pipermail/tac_plus/attachments/20130712/b3d9ca66/attachment.html>