Page 1 of 2
Cisco Switch check issue
Posted: Tue Feb 11, 2014 10:36 am
by BanditBBS
used the Switch Wizard and added a new switch with a bunch of ports selected for bw and up/down checks. With the bw check, they keep cycling between Ok and UNKNOWN with this error: /var/lib/mrtg/10.97.242.92_437284864.rrd does not exist.
Then after the next check it'll go OK again and then sometime down the road go UNKNOWN again.
Also, every once in a while I get a CRITICAL with this error:
Code: Select all
[Can't locate Net/SNMP.pm in @INC (@INC contains: /usr/local/nagios/libexec /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /usr/local/nagios/libexec/check_ifo
and after the next check it goes back to OK.
Re: Cisco Switch check issue
Posted: Tue Feb 11, 2014 1:49 pm
by BanditBBS
Ok, I think we can chalk this one up to my stupidity. I'll report back in a few minutes, just didn't want anyone to waste anytime trying to reproduce....in case you were.
Re: Cisco Switch check issue
Posted: Tue Feb 11, 2014 2:03 pm
by BanditBBS
Ok, half of the issue is resolved. The Net/SNMP issue seems to be resolved after I installed Net::SNMP on a gearman worker(woops).
Still getting the UNKNOWN results every so often. No clue why it is saying the rrd doesn't exist. HELP!
Re: Cisco Switch check issue
Posted: Tue Feb 11, 2014 5:21 pm
by lmiltchev
Did you run the wizard more than once?
Have you tried disabling mod gearman temporarily (for testing purposes) to see if this is going to make a difference? Are you having issue only on the "new" switch that you added or on all of the switches?
Re: Cisco Switch check issue
Posted: Tue Feb 11, 2014 7:12 pm
by BanditBBS
lmiltchev wrote:Did you run the wizard more than once?
Have you tried disabling mod gearman temporarily (for testing purposes) to see if this is going to make a difference? Are you having issue only on the "new" switch that you added or on all of the switches?
I knew someone would finally respond to this one as I was on my way home, grrr, LOL.
1.) Disabling gearman is not an option. Over half my environment would instantly alert as down. I don't just distribute checks for load balancing, I also use it to get around firewalls with only needing the one port opened.
2.) Believe it or not, this is the first switch that I used the switch wizard on at this job(I have used it in the past at previous job). I have been using a different plugin to monitor stuff about the switch, have not monitored ports until now. Ran it and everything seems fine except the attached image. For some reason it doesn't find the rrd almost every other time. gearman shouldn't have any affect on this as the check comes back to nagios server and it processes the performance data anyway.
I do graph other stuff perfectly. FYI this is on a 2.8c server.
Capture (1).JPG
Re: Cisco Switch check issue
Posted: Wed Feb 12, 2014 11:07 am
by lmiltchev
I believe there were some bugs in the older switch wizards, that should be fixed in the latest version. What is the Network Switch/Router Wizard's version that you are using? If it is 1.6, try installing 1.7, and re-run the wizard. Let me know if this helped.
Re: Cisco Switch check issue
Posted: Wed Feb 12, 2014 11:32 am
by BanditBBS
lmiltchev wrote:I believe there were some bugs in the older switch wizards, that should be fixed in the latest version. What is the Network Switch/Router Wizard's version that you are using? If it is 1.6, try installing 1.7, and re-run the wizard. Let me know if this helped.
Already on 1.7. And regardless, it ran fine and setup the stuff fine. Just every other time it runs Nagios can't find the rrd.
Re: Cisco Switch check issue
Posted: Wed Feb 12, 2014 12:16 pm
by lmiltchev
Run the following commands and show us the output:
Code: Select all
LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg -check
ls -ld /var/lib/mrtg/
su nagios -c 'ls -l /var/lib/mrtg/10.97.242.92_437284864.rrd'
/usr/local/nagios/libexec/check_rrdtraf -vv -f /var/lib/mrtg/10.97.242.92_437284864.rrd -w 1000,2000 -c 2000,3000
Re: Cisco Switch check issue
Posted: Wed Feb 12, 2014 12:20 pm
by BanditBBS
Code: Select all
[root@svwdcnagios02 ~]# LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg -check
[root@svwdcnagios02 ~]# ls -ld /var/lib/mrtg/
drwxr-xr-x. 2 root root 4096 Feb 11 10:20 /var/lib/mrtg/
[root@svwdcnagios02 ~]# su nagios -c 'ls -l /var/lib/mrtg/10.97.242.92_437284864.rrd'
-rw-r--r-- 1 root root 105312 Feb 12 12:20 /var/lib/mrtg/10.97.242.92_437284864.rrd
[root@svwdcnagios02 ~]# /usr/local/nagios/libexec/check_rrdtraf -vv -f /var/lib/mrtg/10.97.242.92_437284864.rrd -w 1000,2000 -c 2000,3000
Using RRD file: /var/lib/mrtg/10.97.242.92_437284864.rrd
Input warning level(kb/s): 1000
Output warning level(kb/s): 2000
Input critical level (kb/s): 2000
Output critical level (kb/s): 3000
Fetching data with command: rrdtool fetch /var/lib/mrtg/10.97.242.92_437284864.rrd AVERAGE -s-10minutes | grep -vi "nan"
RRD File Data:
ds0 ds1
1392225300: 6.7796100316e+06 7.1261573853e+06
1392225600: 7.0131488179e+06 5.6349043828e+06
Raw Input Traffic Value (b/s): 56105190.543200
Raw Output Traffic Value (b/s): 45079235.062400
Decimal Input Traffic Value (b/s): 56105190
Decimal Output Traffic Value (b/s): 45079235
Traffic IN scalar: 1000000
Traffic OUT scalar: 1000000
CRITICAL - Current BW in: 56.10Mbps Out: 45.07Mbps|in=56.105190Mb/s;1000;2000 out=45.079235Mb/s;2000;3000
Re: Cisco Switch check issue
Posted: Wed Feb 12, 2014 3:07 pm
by lmiltchev
You don't have errors in the mrtg.cfg, and the permissions look fine.
It is strange that you don't get the "rrd does not exist" error, when you run the check from the CLI... but you get it in the GUI...