Page 1 of 3

Pending check issue

Posted: Fri Feb 26, 2016 11:02 am
by haile711
hey guys,
I have added this ping service check for these servers for over a month now, but their ping service is still pending
Even tho, it's say Service check is pending... Check is scheduled for 2016-02-26 11:01:56, but it's actually never check it. Can you guys help?

Re: Pending check issue

Posted: Fri Feb 26, 2016 11:24 am
by lmiltchev
Can you post the service config, along with the configs of the templates this service is using (if any)?

CCM->Services-><your ping service>->View Text Config (the diskette icon)

Re: Pending check issue

Posted: Fri Feb 26, 2016 11:26 am
by bwallace
....also, take a look in the nagios.log for anything related. Post it here so we could review it as well.
/usr/local/nagios/var/nagios.log

Re: Pending check issue

Posted: Fri Feb 26, 2016 1:41 pm
by haile711
bwallace,

this is all I could see in the nagios.log file that is concerning.
@lmiltchev i have a template that i use. All these service are under "static" directory. the the ping service template that im using is

Code: Select all

xiwizard_linuxserver_ping_service
I have used this same template for 100 plus ping services for different host. But these last 20 kept staying in the pending mode. Is there a limit to how many services Nagios can handle?
thanks you for taking a look everyone

Code: Select all

[1456506301] wproc: SERVICE EVENTHANDLER job 7642 from worker Core Worker 28399 is a non-check helper but exited with return code 1
[1456506301] wproc:   early_timeout=0; exited_ok=1; wait_status=256; error_code=0;

Re: Pending check issue

Posted: Fri Feb 26, 2016 2:58 pm
by lmiltchev
@lmiltchev i have a template that i use. All these service are under "static" directory.
We still need to see all of the relevant configs in order to make sure that these services are defined properly, i.e. they have a valid "check_command", etc.

You can go to:

CCM->Services
CCM->Service Tempaltes
CCM->Commnads

click on the "View Text Config" (the diskette icon), and show us the configs.

Note: Because of inheritance in Nagios, you could have a template, that is using a template, that is using a template..., etc. The same is valid for commands.

Here's an example of a "Ping" service definition + the template + the check commands.

Code: Select all

define service {
	host_name			Windows7
	service_description		Ping
	use				xiwizard_windowsserver_ping_service
	max_check_attempts		5
	check_interval			5
	retry_interval			1
	check_period			xi_timeperiod_24x7
	notification_interval		60
	notification_period		xi_timeperiod_24x7
	notifications_enabled		0
	contacts			nagiosadmin
	_xiwizard			windowsserver
	register			1
	}

define service {
       name                          		xiwizard_windowsserver_ping_service
       use                           		xiwizard_generic_service
       check_command                 		check_xi_service_ping!3000.0!80%!5000.0!100%
       register                    		0

}

define command {
       command_name                  		check_xi_service_ping
       command_line                  		$USER1$/check_icmp -H $HOSTADDRESS$ -w $ARG1$,$ARG2$ -c $ARG3$,$ARG4$ -p 5
}

define command {
       command_name                  		check_icmp
       command_line                  		$USER1$/check_ping -H $HOSTADDRESS$ -w $ARG1$ -c $ARG2$
}

define command {
       command_name                  		check_ping
       command_line                  		$USER1$/check_ping -H $HOSTADDRESS$ -w $ARG1$ -c $ARG2$ -p 5
}

Re: Pending check issue

Posted: Fri Feb 26, 2016 3:57 pm
by haile711
lmiltchev,
Below are the "command, template and service" configuration
Command use

Code: Select all

define command {
       command_name                             check_xi_service_ping
       command_line                             $USER1$/check_icmp -H $HOSTADDRESS$ -w $ARG1$,$ARG2$ -c $ARG3$,$ARG4$ -p 5
}
Service template

Code: Select all

define service {
       name                                     xiwizard_linuxserver_ping_service
       use                                      xiwizard_generic_service
       check_command                            check_xi_service_ping!3000.0!80%!5000.0!100%
       register                                 0

}
Actual Service configure

Code: Select all

define service {
        host_name                       ma100mlvews004
        service_description             Ping
        servicegroups                   Services
        use                             xiwizard_linuxserver_ping_service
        max_check_attempts              1
        check_interval                  5
        retry_interval                  1
        check_period                    24x7
        notification_interval           0
        notification_period             24x7
        notifications_enabled           0
        contact_groups                  siteops
        notification_options            w,c
        _xiwizard                       nrpe
        register                        1
        }

Re: Pending check issue

Posted: Sun Feb 28, 2016 11:03 pm
by Box293
All of your configs look correct.

Can you find the service in the file /usr/local/nagios/var/objects.cache as this is the final object once all the templates have been combined and paste it here please.

I suspect this may be database related.

Run this command

Code: Select all

tail /var/log/mysqld.log -n 100 > /tmp/mysqld_log.txt
Send us this file:
/tmp/mysqld_log.txt

If you notice it says there are crashed tables in the file mysqld_log.txt then can you please follow this article to run the database repair process as I believe you may have some database issues:

https://support.nagios.com/kb/article.php?id=24

This may take a while to complete. Once it has, please scroll back up through the history to make sure there were not any database errors (you may see an ndo2db error when stopping the service but this is ok).

Re: Pending check issue

Posted: Mon Feb 29, 2016 9:01 am
by haile711
good morning Box,

Here is the ping service i got from "objects.cache"

Code: Select all

define service {
        host_name       MA100DLVAPL823
        service_description     Ping
        check_period    24x7
        check_command   check_xi_service_ping!3000.0!80%!5000.0!100%
        contact_groups  siteops
        notification_period     24x7
        initial_state   o
        importance      0
        check_interval  5.000000
        retry_interval  1.000000
        max_check_attempts      1
        is_volatile     0
        parallelize_check       1
        active_checks_enabled   1
        passive_checks_enabled  1
        obsess  1
        event_handler_enabled   1
        low_flap_threshold      0.000000
        high_flap_threshold     0.000000
        flap_detection_enabled  1
        flap_detection_options  a
        freshness_threshold     0
        check_freshness 0
        notification_options    w,c
        notifications_enabled   0
        notification_interval   0.000000
        first_notification_delay        0.000000
        stalking_options        n
        process_perf_data       1
        retain_status_information       1
        retain_nonstatus_information    1
        _XIWIZARD       nrpe
        }

Re: Pending check issue

Posted: Mon Feb 29, 2016 2:50 pm
by ssax
First, let's run this from the XI server command line to see what it outputs, maybe there is a segfault or something:
- Make sure to change HOSTADDRESS to what the host has listed for it's address.

Code: Select all

su - nagios
/usr/local/nagios/libexec/check_icmp -H HOSTADDRESS -w 3000.0,80% -c 5000.0,100% -p 5
Let us know the results.

Thank you

Re: Pending check issue

Posted: Tue Mar 01, 2016 9:44 am
by haile711
good morning ssax,

here is the result from the check

Code: Select all

[root@MA100DLVMON813 maas]# /usr/local/nagios/libexec/check_icmp -H ma100dlvapl823 -w 3000.0,80% -c 5000.0,100% -p 5
OK - ma100dlvapl823: rta 0.148ms, lost 0%|rta=0.148ms;3000.000;5000.000;0; pl=0%;80;100;; rtmax=0.186ms;;;; rtmin=0.123ms;;;;