Pending check issue

[haile711]

hey guys,
I have added this ping service check for these servers for over a month now, but their ping service is still pending
Even tho, it's say Service check is pending... Check is scheduled for 2016-02-26 11:01:56, but it's actually never check it. Can you guys help?

[lmiltchev]

Can you post the service config, along with the configs of the templates this service is using (if any)?

CCM->Services-><your ping service>->View Text Config (the diskette icon)

[bwallace]

....also, take a look in the nagios.log for anything related. Post it here so we could review it as well.
/usr/local/nagios/var/nagios.log

[haile711]

bwallace,

this is all I could see in the nagios.log file that is concerning.
@lmiltchev i have a template that i use. All these service are under "static" directory. the the ping service template that im using is

Code: Select all

xiwizard_linuxserver_ping_service

I have used this same template for 100 plus ping services for different host. But these last 20 kept staying in the pending mode. Is there a limit to how many services Nagios can handle?
thanks you for taking a look everyone

Code: Select all

[1456506301] wproc: SERVICE EVENTHANDLER job 7642 from worker Core Worker 28399 is a non-check helper but exited with return code 1
[1456506301] wproc:   early_timeout=0; exited_ok=1; wait_status=256; error_code=0;

[lmiltchev]

@lmiltchev i have a template that i use. All these service are under "static" directory.

We still need to see all of the relevant configs in order to make sure that these services are defined properly, i.e. they have a valid "check_command", etc.

You can go to:

CCM->Services
CCM->Service Tempaltes
CCM->Commnads

click on the "View Text Config" (the diskette icon), and show us the configs.

Note: Because of inheritance in Nagios, you could have a template, that is using a template, that is using a template..., etc. The same is valid for commands.

Here's an example of a "Ping" service definition + the template + the check commands.

Code: Select all

Code: Select all

define service {
	host_name			Windows7
	service_description		Ping
	use				xiwizard_windowsserver_ping_service
	max_check_attempts		5
	check_interval			5
	retry_interval			1
	check_period			xi_timeperiod_24x7
	notification_interval		60
	notification_period		xi_timeperiod_24x7
	notifications_enabled		0
	contacts			nagiosadmin
	_xiwizard			windowsserver
	register			1
	}

define service {
       name                          		xiwizard_windowsserver_ping_service
       use                           		xiwizard_generic_service
       check_command                 		check_xi_service_ping!3000.0!80%!5000.0!100%
       register                    		0

}

define command {
       command_name                  		check_xi_service_ping
       command_line                  		$USER1$/check_icmp -H $HOSTADDRESS$ -w $ARG1$,$ARG2$ -c $ARG3$,$ARG4$ -p 5
}

define command {
       command_name                  		check_icmp
       command_line                  		$USER1$/check_ping -H $HOSTADDRESS$ -w $ARG1$ -c $ARG2$
}

define command {
       command_name                  		check_ping
       command_line                  		$USER1$/check_ping -H $HOSTADDRESS$ -w $ARG1$ -c $ARG2$ -p 5
}

[haile711]

lmiltchev,
Below are the "command, template and service" configuration
Command use

Code: Select all

define command {
       command_name                             check_xi_service_ping
       command_line                             $USER1$/check_icmp -H $HOSTADDRESS$ -w $ARG1$,$ARG2$ -c $ARG3$,$ARG4$ -p 5
}

Service template

Code: Select all

define service {
       name                                     xiwizard_linuxserver_ping_service
       use                                      xiwizard_generic_service
       check_command                            check_xi_service_ping!3000.0!80%!5000.0!100%
       register                                 0

}

Actual Service configure

Code: Select all

define service {
        host_name                       ma100mlvews004
        service_description             Ping
        servicegroups                   Services
        use                             xiwizard_linuxserver_ping_service
        max_check_attempts              1
        check_interval                  5
        retry_interval                  1
        check_period                    24x7
        notification_interval           0
        notification_period             24x7
        notifications_enabled           0
        contact_groups                  siteops
        notification_options            w,c
        _xiwizard                       nrpe
        register                        1
        }

[Box293]

All of your configs look correct.

Can you find the service in the file /usr/local/nagios/var/objects.cache as this is the final object once all the templates have been combined and paste it here please.

I suspect this may be database related.

Run this command

Code: Select all

tail /var/log/mysqld.log -n 100 > /tmp/mysqld_log.txt

Send us this file:
/tmp/mysqld_log.txt

If you notice it says there are crashed tables in the file mysqld_log.txt then can you please follow this article to run the database repair process as I believe you may have some database issues:

https://support.nagios.com/kb/article.php?id=24

This may take a while to complete. Once it has, please scroll back up through the history to make sure there were not any database errors (you may see an ndo2db error when stopping the service but this is ok).

[haile711]

good morning Box,

Here is the ping service i got from "objects.cache"

Code: Select all

define service {
        host_name       MA100DLVAPL823
        service_description     Ping
        check_period    24x7
        check_command   check_xi_service_ping!3000.0!80%!5000.0!100%
        contact_groups  siteops
        notification_period     24x7
        initial_state   o
        importance      0
        check_interval  5.000000
        retry_interval  1.000000
        max_check_attempts      1
        is_volatile     0
        parallelize_check       1
        active_checks_enabled   1
        passive_checks_enabled  1
        obsess  1
        event_handler_enabled   1
        low_flap_threshold      0.000000
        high_flap_threshold     0.000000
        flap_detection_enabled  1
        flap_detection_options  a
        freshness_threshold     0
        check_freshness 0
        notification_options    w,c
        notifications_enabled   0
        notification_interval   0.000000
        first_notification_delay        0.000000
        stalking_options        n
        process_perf_data       1
        retain_status_information       1
        retain_nonstatus_information    1
        _XIWIZARD       nrpe
        }

[ssax]

First, let's run this from the XI server command line to see what it outputs, maybe there is a segfault or something:
- Make sure to change HOSTADDRESS to what the host has listed for it's address.

Code: Select all

su - nagios
/usr/local/nagios/libexec/check_icmp -H HOSTADDRESS -w 3000.0,80% -c 5000.0,100% -p 5

Let us know the results.

Thank you

[haile711]

good morning ssax,

here is the result from the check

Code: Select all

[root@MA100DLVMON813 maas]# /usr/local/nagios/libexec/check_icmp -H ma100dlvapl823 -w 3000.0,80% -c 5000.0,100% -p 5
OK - ma100dlvapl823: rta 0.148ms, lost 0%|rta=0.148ms;3000.000;5000.000;0; pl=0%;80;100;; rtmax=0.186ms;;;; rtmin=0.123ms;;;;