Cisco Switch check issue

[BanditBBS]

used the Switch Wizard and added a new switch with a bunch of ports selected for bw and up/down checks. With the bw check, they keep cycling between Ok and UNKNOWN with this error: /var/lib/mrtg/10.97.242.92_437284864.rrd does not exist.

Then after the next check it'll go OK again and then sometime down the road go UNKNOWN again.

Also, every once in a while I get a CRITICAL with this error:

Code: Select all

[Can't locate Net/SNMP.pm in @INC (@INC contains: /usr/local/nagios/libexec /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /usr/local/nagios/libexec/check_ifo

and after the next check it goes back to OK.

[BanditBBS]

Ok, I think we can chalk this one up to my stupidity. I'll report back in a few minutes, just didn't want anyone to waste anytime trying to reproduce....in case you were.

[BanditBBS]

Ok, half of the issue is resolved. The Net/SNMP issue seems to be resolved after I installed Net::SNMP on a gearman worker(woops).

Still getting the UNKNOWN results every so often. No clue why it is saying the rrd doesn't exist. HELP!

[lmiltchev]

Did you run the wizard more than once?
Have you tried disabling mod gearman temporarily (for testing purposes) to see if this is going to make a difference? Are you having issue only on the "new" switch that you added or on all of the switches?

[BanditBBS]

Did you run the wizard more than once?
Have you tried disabling mod gearman temporarily (for testing purposes) to see if this is going to make a difference? Are you having issue only on the "new" switch that you added or on all of the switches?

I knew someone would finally respond to this one as I was on my way home, grrr, LOL.

1.) Disabling gearman is not an option. Over half my environment would instantly alert as down. I don't just distribute checks for load balancing, I also use it to get around firewalls with only needing the one port opened.
2.) Believe it or not, this is the first switch that I used the switch wizard on at this job(I have used it in the past at previous job). I have been using a different plugin to monitor stuff about the switch, have not monitored ports until now. Ran it and everything seems fine except the attached image. For some reason it doesn't find the rrd almost every other time. gearman shouldn't have any affect on this as the check comes back to nagios server and it processes the performance data anyway.

I do graph other stuff perfectly. FYI this is on a 2.8c server.

Capture (1).JPG

[lmiltchev]

I believe there were some bugs in the older switch wizards, that should be fixed in the latest version. What is the Network Switch/Router Wizard's version that you are using? If it is 1.6, try installing 1.7, and re-run the wizard. Let me know if this helped.

[BanditBBS]

I believe there were some bugs in the older switch wizards, that should be fixed in the latest version. What is the Network Switch/Router Wizard's version that you are using? If it is 1.6, try installing 1.7, and re-run the wizard. Let me know if this helped.

Already on 1.7. And regardless, it ran fine and setup the stuff fine. Just every other time it runs Nagios can't find the rrd.

[lmiltchev]

Run the following commands and show us the output:

Code: Select all

LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg -check
ls -ld /var/lib/mrtg/
su nagios -c 'ls -l /var/lib/mrtg/10.97.242.92_437284864.rrd'
/usr/local/nagios/libexec/check_rrdtraf -vv -f /var/lib/mrtg/10.97.242.92_437284864.rrd -w 1000,2000 -c 2000,3000

[BanditBBS]

Code: Select all

[root@svwdcnagios02 ~]# LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg -check
[root@svwdcnagios02 ~]# ls -ld /var/lib/mrtg/
drwxr-xr-x. 2 root root 4096 Feb 11 10:20 /var/lib/mrtg/
[root@svwdcnagios02 ~]# su nagios -c 'ls -l /var/lib/mrtg/10.97.242.92_437284864.rrd'
-rw-r--r-- 1 root root 105312 Feb 12 12:20 /var/lib/mrtg/10.97.242.92_437284864.rrd
[root@svwdcnagios02 ~]# /usr/local/nagios/libexec/check_rrdtraf -vv -f /var/lib/mrtg/10.97.242.92_437284864.rrd -w 1000,2000 -c 2000,3000
Using RRD file: /var/lib/mrtg/10.97.242.92_437284864.rrd
Input warning level(kb/s): 1000
Output warning level(kb/s): 2000
Input critical level (kb/s): 2000
Output critical level (kb/s): 3000
Fetching data with command: rrdtool fetch /var/lib/mrtg/10.97.242.92_437284864.rrd AVERAGE -s-10minutes | grep -vi "nan"
RRD File Data:
                            ds0                 ds1

1392225300: 6.7796100316e+06 7.1261573853e+06
1392225600: 7.0131488179e+06 5.6349043828e+06
Raw Input Traffic Value (b/s): 56105190.543200
Raw Output Traffic Value (b/s): 45079235.062400
Decimal Input Traffic Value (b/s): 56105190
Decimal Output Traffic Value (b/s): 45079235
Traffic IN scalar: 1000000
Traffic OUT scalar: 1000000
CRITICAL - Current BW in: 56.10Mbps Out: 45.07Mbps|in=56.105190Mb/s;1000;2000 out=45.079235Mb/s;2000;3000

[lmiltchev]

You don't have errors in the mrtg.cfg, and the permissions look fine.
It is strange that you don't get the "rrd does not exist" error, when you run the check from the CLI... but you get it in the GUI...