[rancid] Extreme XOS (etc.) slow buffered output & clogin

[Zenon Mousmoulas]

Hi,

There is an issue with clogin and buffered output, easily reproducible
with Extreme XOS devices, which by default show a sizable banner
post-login, but I have also been able to reproduce it with Cisco switches,
when a large banner is displayed:

On Thu, 01 Dec 2011 03:44:32 +0200, Zenon Mousmoulas <zmousm at noc.grnet.gr>
wrote:
> [...]
> However there's still a problem in this version with buffered output
from
> the extreme. After login, when you send "\r", the following expect
> catches the first prompt, not the one after "\r". This becomes evident
> later on, when the expect statement in the loop within run_commands
> catches
> "disable clipaging\r\n\<prompt>" rather than the command echoed and
> its' output. So you're always one step behind, until the end, where you
> erroneously see a prompt after sending "quit\r" so you also
> send -h "exit\r". This obviously messes up xrancid parsing as
> well. See the attached clogin log for more evidence (with exp_internal
> added after login).

Some time ago I had posted a patch which tried to address this, among
other things related to XOS & rancid. Other changes were accepted in 2.3.7
(thanks for that), however this issue is still present in 2.3.8.
My rather ugly workaround was to add this code after login and before
prompt detection:

diff -ru rancid-2.3.8/bin/clogin.in rancid-2.3.8-zmousm/bin/clogin.in
--- rancid-2.3.8/bin/clogin.in	2012-02-01 00:42:14.000000000 +0200
+++ rancid-2.3.8-zmousm/bin/clogin.in	2012-06-06 09:00:59.000000000 +0300
@@ -858,6 +858,16 @@
 	# if login failed or rsh was unsuccessful, move on to the next device
 	continue
     }
+    # Get all (output after login) you can eat in 1s
+    # (hopefully up to and including the first prompt).
+    # This is mostly necessary to work around
+    # stoopid extreme output buffering.
+    expect {
+	-timeout 1
+	-re "\[\n\r]+"	{ exp_continue }
+	-re "\[^\n\r]+"	{ exp_continue -continue_timer }
+	timeout		{}
+    }
     # Figure out the prompt.
     if { [regexp -- "(#| \\(enable\\))" $prompt_match junk] == 1 } {
 	set enable 0

Unfortunately this introduces a 1 second delay for all (not just XOS)
clogin-controlled devices, but there is no obvious way to avoid this, since
we haven't done prompt detection yet and haven't set $platform.
Furthermore, this issue is not strictly limited to XOS devices, as noted
previously, therefore a slightly different approach would be perhaps to
trigger this delay when the output received after login exceeds some
conservative threshold, but I'm not really sure if and how one could do
such a thing.

I can send again (off-list) clogin output with expect debugging enabled,
depicting the issue described above, as well as the correct output of
clogin patched with the code above.
While going through the list archives, I also found this post which shows
how xrancid parsing (see: missed cmds) and output is messed up exactly due
to this issue:
http://www.shrubbery.net/pipermail/rancid-discuss/2012-April/006308.html

Please advise how this issue can be addressed.

Thanks,
Zenon Mousmoulas