check_snmp, performance graph and graph explorer discrepancy

[tgriep]

Can you go to Core Config Manager > commands and see if the command is active?
Can you provide screen captures of the command's settings in Core Config Manager and post them here?

[iptribe]

Yes it is active:

check_snmp_gauge.PNG

check_snmp_gauge2.PNG

[tgriep]

Could you PM me your system profile?
To send us your system profile
Click "Admin"
Under "System Config", click "System Profile"
Click the "Download Profile" button
Save the file and PM it to me.

[iptribe]

Sent.

[tgriep]

Thanks for the profile.

If looks like the graph template is missing from your system and that is why it is not working for you.
Attached to this post is the check_snmp_gauge.cfg file and it needs to be uploaded to this folder in the XI system /usr/local/nagios/etc/pnp/check_commands/
Change the permissions of the check_snmp_gauge.cfg file to the following

Code: Select all

chown apache.nagios check_snmp_gauge.cfg
chmod 666 check_snmp_gauge.cfg

Delete the .rrd and .xml files for that service and let us know if it works for you.

[iptribe]

I believe we have done this before. I did it again and it did not help.

[root@NFVI-COMMON-NAGIOS-01 check_commands]# cp /home/iptribe/check_snmp_gauge.cfg .
[root@NFVI-COMMON-NAGIOS-01 check_commands]# ls -l
total 8
-rw-rw-r-- 1 apache nagios 755 Aug 14 2014 check_nwstat.cfg-sample
-rw------- 1 root nagios 752 Sep 30 11:49 check_snmp_gauge.cfg
[root@NFVI-COMMON-NAGIOS-01 check_commands]# chown apache.nagios check_snmp_gauge.cfg
[root@NFVI-COMMON-NAGIOS-01 check_commands]# chmod 666 check_snmp_gauge.cfg
[root@NFVI-COMMON-NAGIOS-01 check_commands]# ls -l
total 8
-rw-rw-r-- 1 apache nagios 755 Aug 14 2014 check_nwstat.cfg-sample
-rw-rw-rw- 1 apache nagios 752 Sep 30 11:49 check_snmp_gauge.cfg
[root@NFVI-COMMON-NAGIOS-01 check_commands]#
[root@NFVI-COMMON-NAGIOS-01 check_commands]#
[root@NFVI-COMMON-NAGIOS-01 check_commands]# rm /usr/local/nagios/share/perfdata/RCS-PROD-SSC-02/Active_Sessions.*
rm: remove regular file `/usr/local/nagios/share/perfdata/RCS-PROD-SSC-02/Active_Sessions.rrd'? y
rm: remove regular file `/usr/local/nagios/share/perfdata/RCS-PROD-SSC-02/Active_Sessions.xml'? y
[root@NFVI-COMMON-NAGIOS-01 check_commands]# rm /usr/local/nagios/share/perfdata/RCS-PROD-SSC-01/Active_Sessions.*
rm: remove regular file `/usr/local/nagios/share/perfdata/RCS-PROD-SSC-01/Active_Sessions.rrd'? y
rm: remove regular file `/usr/local/nagios/share/perfdata/RCS-PROD-SSC-01/Active_Sessions.xml'? y
[root@NFVI-COMMON-NAGIOS-01 check_commands]# rm /usr/local/nagios/share/perfdata/RCS-PROD-SSC-03/Active_Sessions.*
rm: remove regular file `/usr/local/nagios/share/perfdata/RCS-PROD-SSC-03/Active_Sessions.rrd'? y
rm: remove regular file `/usr/local/nagios/share/perfdata/RCS-PROD-SSC-03/Active_Sessions.xml'? y
[root@NFVI-COMMON-NAGIOS-01 check_commands]#

[tgriep]

It should be working now.
Can you run the following tail command in a shell, delete one of the xml and rrd files for the service, schedule an immediate check for that service and wait until you see that check show up on the screen and then post the output of the tail command?

Code: Select all

tail -f /usr/local/nagios/var/perfdata.log

[iptribe]

Still does not work...

Code: Select all

2015-10-02 14:57:50 [18697] [1] Found Performance Data for RCS-PROD-SSC-02 / Active_Sessions (PERIMETA-MIB::gl
obalStatActiveCalls.summaryPeriod1Current5Minutes=0;;10000;) 
2015-10-02 14:57:50 [18697] [3] DEBUG: Normal perfdata
2015-10-02 14:57:50 [18697] [3] DEBUG: RAW Command -> check_xi_service_snmp!-P 2c -o globalStatActiveCalls.sum
maryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -c 10000!!!!!!!
2015-10-02 14:57:50 [18697] [2] No Custom Template found for check_xi_service_snmp (/usr/local/nagios/etc/pnp/
check_commands/check_xi_service_snmp.cfg) 
2015-10-02 14:57:50 [18697] [3] RRD Datatype is GAUGE
2015-10-02 14:57:50 [18697] [2] Template is check_xi_service_snmp.php
2015-10-02 14:57:50 [18697] [2] data2rrd called
2015-10-02 14:57:50 [18697] [3]  -- Job 0 
2015-10-02 14:57:50 [18697] [3]   -- auth_hostname -> RCS-PROD-SSC-02
2015-10-02 14:57:50 [18697] [3]   -- auth_servicedesc -> Active Sessions
2015-10-02 14:57:50 [18697] [3]   -- critical -> 10000
2015-10-02 14:57:50 [18697] [3]   -- disp_hostname -> RCS-PROD-SSC-02
2015-10-02 14:57:50 [18697] [3]   -- disp_servicedesc -> Active Sessions
2015-10-02 14:57:50 [18697] [3]   -- dstype -> GAUGE
2015-10-02 14:57:50 [18697] [3]   -- hostname -> RCS-PROD-SSC-02
2015-10-02 14:57:50 [18697] [3]   -- label -> PERIMETA-MIB::globalStatActiveCalls.summaryPeriod1Current5Minute
s
2015-10-02 14:57:50 [18697] [3]   -- multi -> 0
2015-10-02 14:57:50 [18697] [3]   -- name -> PERIMETA-MIB__globalStatActiveCalls.summaryPeriod1Current5Minutes
2015-10-02 14:57:50 [18697] [3]   -- rrd_heartbeat -> 8460
2015-10-02 14:57:50 [18697] [3]   -- rrd_max -> U
2015-10-02 14:57:50 [18697] [3]   -- rrd_min -> U
2015-10-02 14:57:50 [18697] [3]   -- rrd_storage_type -> SINGLE
2015-10-02 14:57:50 [18697] [3]   -- servicedesc -> Active_Sessions
2015-10-02 14:57:50 [18697] [3]   -- template -> check_xi_service_snmp
2015-10-02 14:57:50 [18697] [3]   -- uom -> 
2015-10-02 14:57:50 [18697] [3]   -- value -> 0
2015-10-02 14:57:50 [18697] [3] DEBUG: SINGLE Storage Type
2015-10-02 14:57:50 [18697] [3] DEBUG: TPL-> check_xi_service_snmp
2015-10-02 14:57:50 [18697] [3] DEBUG: CRE-> DS:1:GAUGE:8460:U:U
2015-10-02 14:57:50 [18697] [3] DEBUG: UPD-> :0
2015-10-02 14:57:50 [18697] [2] Reading /usr/local/nagios/etc/pnp/rra.cfg
2015-10-02 14:57:50 [18697] [2] RRDs::create /usr/local/nagios/share/perfdata/RCS-PROD-SSC-02/Active_Sessions.
rrd RRA:AVERAGE:0.5:1:2880 RRA:AVERAGE:0.5:5:2880 RRA:AVERAGE:0.5:30:4320 RRA:AVERAGE:0.5:360:5840 RRA:MAX:0.5
:1:2880 RRA:MAX:0.5:5:2880 RRA:MAX:0.5:30:4320 RRA:MAX:0.5:360:5840 RRA:MIN:0.5:1:2880 RRA:MIN:0.5:5:2880 RRA:
MIN:0.5:30:4320 RRA:MIN:0.5:360:5840 DS:1:GAUGE:8460:U:U --start=1443769047 --step=60

[tgriep]

From the output of that log, it looks like the Active Session service check command was put back to the template and is currently not using the check_snmp_gauge command.

Edit your Active Session service and change this from

Code: Select all

check_command                   check_xi_service_snmp!-P 2c -o globalStatActiveCalls.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -c 10000!!!!!!!

to

Code: Select all

check_command                   check_snmp_gauge!-P 2c -o globalStatActiveCalls.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -c 10000!!!!!!!

Delete the rrd and xml files for that service check and wait 20 minutes to see if the graphs are correct now.

[iptribe]

It is what I have already configured via GUI. I also tried modifying it via CLI. Any idea why it is not being followed during checks?

active sessions.PNG

Code: Select all

[iptribe@NFVI-COMMON-NAGIOS-01 ~]$ cat /usr/local/nagios/etc/services/Active\ Sessions.cfg
###############################################################################
#
# Service configuration file
#
# Created by: Nagios QL Version 3.0.3
# Date:	      2015-09-03 15:25:12
# Version:    Nagios 3.x config file
#
# --- DO NOT EDIT THIS FILE BY HAND --- 
# Nagios QL will overwite all manual settings during the next update
#
###############################################################################

define service {
	host_name			RCS-PROD-SSC-01,RCS-PROD-SSC-02,RCS-PROD-SSC-03,RCS-TELCO-SSC-01
	service_description		Active Sessions
	check_command			check_snmp_gauge!-P 2c -o globalStatActiveCalls.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -c 10000!!!!!!!
	initial_state			u
	max_check_attempts		10
	check_interval			5
	retry_interval			10
	active_checks_enabled		1
	check_period			24x7
	event_handler			xi_service_event_handler
	register			1
	}	

###############################################################################
#
# Service configuration file
#
# END OF FILE
#
###############################################################################

2015-10-05 11:48:37 [506] [1] Found Performance Data for RCS-PROD-SSC-01 / Active_Sessions (PERIMETA-MIB::globalStatActiveCalls.summaryPeriod1Current5Minutes=5;;10000;)
2015-10-05 11:48:37 [506] [3] DEBUG: Normal perfdata
2015-10-05 11:48:37 [506] [3] DEBUG: RAW Command -> check_xi_service_snmp!-P 2c -o globalStatActiveCalls.summaryPeriod1Current5Minutes -m /usr/share/snmp/mibs/perimeta.mib -c 10000!!!!!!!
2015-10-05 11:48:37 [506] [2] No Custom Template found for check_xi_service_snmp (/usr/local/nagios/etc/pnp/check_commands/check_xi_service_snmp.cfg)
2015-10-05 11:48:37 [506] [3] RRD Datatype is GAUGE
2015-10-05 11:48:37 [506] [2] Template is check_xi_service_snmp.php
2015-10-05 11:48:37 [506] [2] data2rrd called
2015-10-05 11:48:37 [506] [3] -- Job 0
2015-10-05 11:48:37 [506] [3] -- auth_hostname -> RCS-PROD-SSC-01
2015-10-05 11:48:37 [506] [3] -- auth_servicedesc -> Active Sessions
2015-10-05 11:48:37 [506] [3] -- critical -> 10000
2015-10-05 11:48:37 [506] [3] -- disp_hostname -> RCS-PROD-SSC-01
2015-10-05 11:48:37 [506] [3] -- disp_servicedesc -> Active Sessions
2015-10-05 11:48:37 [506] [3] -- dstype -> GAUGE
2015-10-05 11:48:37 [506] [3] -- hostname -> RCS-PROD-SSC-01
2015-10-05 11:48:37 [506] [3] -- label -> PERIMETA-MIB::globalStatActiveCalls.summaryPeriod1Current5Minutes
2015-10-05 11:48:37 [506] [3] -- multi -> 0
2015-10-05 11:48:37 [506] [3] -- name -> PERIMETA-MIB__globalStatActiveCalls.summaryPeriod1Current5Minutes
2015-10-05 11:48:37 [506] [3] -- rrd_heartbeat -> 8460
2015-10-05 11:48:37 [506] [3] -- rrd_max -> U
2015-10-05 11:48:37 [506] [3] -- rrd_min -> U
2015-10-05 11:48:37 [506] [3] -- rrd_storage_type -> SINGLE
2015-10-05 11:48:37 [506] [3] -- servicedesc -> Active_Sessions
2015-10-05 11:48:37 [506] [3] -- template -> check_xi_service_snmp
2015-10-05 11:48:37 [506] [3] -- uom ->
2015-10-05 11:48:37 [506] [3] -- value -> 5
2015-10-05 11:48:37 [506] [3] DEBUG: SINGLE Storage Type
2015-10-05 11:48:37 [506] [3] DEBUG: TPL-> check_xi_service_snmp
2015-10-05 11:48:37 [506] [3] DEBUG: CRE-> DS:1:GAUGE:8460:U:U
2015-10-05 11:48:37 [506] [3] DEBUG: UPD-> :5
2015-10-05 11:48:37 [506] [2] Reading /usr/local/nagios/etc/pnp/rra.cfg
2015-10-05 11:48:37 [506] [2] RRDs::create /usr/local/nagios/share/perfdata/RCS-PROD-SSC-01/Active_Sessions.rrd RRA:AVERAGE:0.5:1:2880 RRA:AVERAGE:0.5:5:2880 RRA:AVERAGE:0.5:30:4320 RRA:AVERAGE:0.5:360:5840 RRA:MAX:0.5:1:2880 RRA:MAX:0.5:5:2880 RRA:MAX:0.5:30:4320 RRA:MAX:0.5:360:5840 RRA:MIN:0.5:1:2880 RRA:MIN:0.5:5:2880 RRA:MIN:0.5:30:4320 RRA:MIN:0.5:360:5840 DS:1:GAUGE:8460:U:U --start=1444016897 --step=60
2015-10-05 11:48:37 [506] [2] /usr/local/nagios/share/perfdata/RCS-PROD-SSC-01/Active_Sessions.rrd created
2015-10-05 11:48:37 [506] [1] 3 lines processed
2015-10-05 11:48:37 [506] [1] /usr/local/nagios/var/spool/perfdata//1444016903.perfdata.service-PID-506 deleted
2015-10-05 11:48:37 [506] [1] PNP exiting (runtime 0.035341s) ...