Network Switch/Router Wizard hangs during switch scanning

[eclypse]

I am running into trouble while attempting to monitor a particular switch. During the scanning phase, the browser hangs and ultimately returns to an error stating "nagiosxi didn’t send any data.". I suspect there is something configured badly on the switch since other switches can be scanned without issue. I'm hoping for some debug log that I could view to help pinpoint the cause before digging more deeply into the switch itself.

Does any such log exist for the wizard scanning process that I can parse, or is there a command-line option that is being run by the wizard behind the scenes that I could run manually to help debug this?

To the best of my knowledge, all other switch monitoring via MRTG is working properly and the

Code: Select all

LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg

command runs cleanly and in a timely manner without any errors from other switches.

[bolson]

Hello eclypse,

Code: Select all

grep "mrtg" /var/log/*

and

Code: Select all

grep "mrtg" /usr/local/nagios/var/*

Might turn up something.

[tgriep]

If the switch has a lot more ports than the other switches, it could be a PHP and Apache limit causing it to fail.
To increase the memory and timeout values that PHP / Apache uses, follow this procedure.
Edit the /etc/php.ini file and change the following from
max_execution_time = 30
max_input_time = 60
memory_limit = 128M
to

Code: Select all

max_execution_time = 60
max_input_time = 120
memory_limit = 512M

add this to the bottom of that file

Code: Select all

max_input_vars=5000

Save the file

Then edit this file

Code: Select all

/etc/httpd/conf/httpd.conf

add this to the bottom of that file

Code: Select all

LimitRequestLine 100000

Save the file and restart Apache for the changes to take affect.
service httpd restart

Then run the Wizard and see if it completes. If not, run the following as root in a shell
tail -f /var/log/httpd/*_log

Then run the Wizard and post the output of the tail command.

One question, is there any special characters on the community string if using SNMP2 or in the username / password if using SNMP3?

[eclypse]

Thanks for the responses. Using some process sleuthing (compare output of ps -ef before/after triggering the wizard), I determined that it was the MRTG /usr/bin/cfgmaker process that was taking forever to complete, causing the wizard (or at least my HTTP session) to become unresponsive.

Code: Select all

sh -c /usr/bin/cfgmaker --show-op-down --noreversedns --zero-speed '100000000' 'MYSNMPSTRING@MYIP:MYPORT::::2' > /usr/local/nagiosxi/tmp/mrtg-MYIP ; touch /usr/local/nagiosxi/tmp/mrtg-MYIP.done.

What I found is that ultimately the MRTG config file did get created after 10 minutes in /usr/local/nagiosxi/tmp. I was able to workaround this by stripping out all of the unnecessary interfaces (I only care about the uplinks), moving the MRTG config file into /etc/mrtg/conf.d and used the clone wizard to add the switch using a similar switch as my template and updating the interface numbers as needed. All of my RRD files are updating nicely and I should be getting graphs soon.

I'm currently running the cfgmaker command manually to determine what is causing the scan to be delayed. This switch has the same number of ports as another rack, so there's likely just some switch config issue, and not a Nagios issue. For those curious, I'm seeing many errors like shown below which are causing long delays in the scan. I don't see these errors if I repeat the cfgmaker command with a known good network switch:

Code: Select all

--base: check for HighspeedCounters failed ... Dropping back to V1

This thread can likely be closed as this looks to be an issue specific to this network switch configuration, though perhaps it can serve as a reference in the future for someone attempting to troubleshoot a similar issue.

[bolson]

Closing topic as resolved.

Thank you for using the Nagios Support Forum.