Service Status stuck in pending

[weapon]

I am running XI 2012R2.2, and have a service stuck in pending. I'm not sure why, as I have everything built off templates, so the only thing different with this check is that it uses check_by_ssh.

Digging in a bit deeper, the service is displaying pending in NagiosXI, but in nagios core it is in an OK state. Running https://nagiosxi01/nagiosxi/backend/?cm ... vicestatus I find the following:

Code: Select all

 <name>Distribution Check</name> 
  <host_display_name /> 
  <display_name>Distribution Check</display_name> 
  <status_update_time>2015-03-19 12:12:16</status_update_time> 
  <status_text>OK - Removed for confidentiality </status_text> 
  <status_text_long /> 
  <current_state>0</current_state> 
  <performance_data>Removed for confidentiality</performance_data> 
  <should_be_scheduled>0</should_be_scheduled> 
  <check_type>0</check_type> 
  <last_state_change>1970-01-01 10:00:00</last_state_change> 
  <last_hard_state_change>1970-01-01 10:00:00</last_hard_state_change> 
  <last_hard_state>0</last_hard_state> 
  <last_time_ok>1970-01-01 10:00:00</last_time_ok> 

it looks like the last_state_change has not been updated correctly. I have tried restarting nagios and nagiosxi services, and I have tried renaming the service, but nothing seems to help. I have had a google around which has gotten me this far, but I can't get any further. Can someone advise how to resolve this issue?

Regards,

David

[Box293]

Can you show us the final service object definition in /usr/local/nagios/var/objects.cache

Also,

Run this command

Code: Select all

tail /var/log/mysqld.log -n 100 > /tmp/mysqld_log.txt

Send us this file:
/tmp/mysqld_log.txt

[weapon]

Because of confidentiality and size (2MB) I have only included the host and service entries

Code: Select all

define host {
	host_name	distro
	alias	
	address	10.4.1.43
	check_period	24x7
	check_command	check-host-alive!!!!!!!!
	contact_groups	<Contact Groups removed>
	notification_period	24x7
	initial_state	o
	check_interval	10.000000
	retry_interval	2.000000
	max_check_attempts	5
	active_checks_enabled	1
	passive_checks_enabled	1
	obsess_over_host	1
	event_handler_enabled	1
	low_flap_threshold	0.000000
	high_flap_threshold	0.000000
	flap_detection_enabled	1
	flap_detection_options	o,d,u
	freshness_threshold	0
	check_freshness	0
	notification_options	d,u,r,f
	notifications_enabled	1
	notification_interval	120.000000
	first_notification_delay	0.000000
	stalking_options	n
	process_perf_data	0
	failure_prediction_enabled	1
	icon_image	document_server.png
	statusmap_image	document_server.png
	retain_status_information	1
	retain_nonstatus_information	1
	_XIWIZARD	genericnetdevice
	}
	
	define service {
	host_name	distro
	service_description	Distribution Check
	check_period	24x7
	check_command	check_xi_by_ssh!-C "./nagios_131940_distro.pl -p"!!!!!!!
	contact_groups	<Contact Groups removed>
	notification_period	24x7
	initial_state	o
	check_interval	10.000000
	retry_interval	2.000000
	max_check_attempts	3
	is_volatile	0
	parallelize_check	1
	active_checks_enabled	1
	passive_checks_enabled	1
	obsess_over_service	1
	event_handler_enabled	1
	low_flap_threshold	0.000000
	high_flap_threshold	0.000000
	flap_detection_enabled	1
	flap_detection_options	o,w,u,c
	freshness_threshold	0
	check_freshness	0
	notification_options	u,w,c,r,f
	notifications_enabled	1
	notification_interval	120.000000
	first_notification_delay	0.000000
	stalking_options	n
	process_perf_data	1
	failure_prediction_enabled	1
	retain_status_information	1
	retain_nonstatus_information	1
	}

[tgriep]

The ndo2mod service might need to be restarted.
Try restarting it and see if that fixes it for you.
Run this on the Nagios Server.

Code: Select all

service ndo2db restart

[weapon]

That didn't seem to help. I restarted ndo2db, and it still was an issue. I tired restarting ndo2db, nagios and nagiosxi, and it still did not help. I looked at the backend getservicestatus again and everything is still the same (except the status update time is updated a bit).

[lmiltchev]

I may be wrong, but it seems like you don't have the full path in your check_command... You have:

Code: Select all

check_command   check_xi_by_ssh!-C "./nagios_131940_distro.pl -p"

It should probably look like this:

Code: Select all

check_command   check_xi_by_ssh!-C "/usr/local/nagios/libexec/nagios_131940_distro.pl -p"

Note: The path to your nagios plugins directory could be different.

[weapon]

The issue is not with the check command, I can run that successfully as the nagios user. The path is a relative path, not absolute - and the plugin is in the home directory on the remote machine, it is not a path to a script on the Nagios machine.

[weapon]

I've just rebooted Nagios (due to some other display issues) but this issue still remains.

[Box293]

Something to try:
Go into CCM
Disable the Service
Apply Configuration
Confirm it does not appear in XI or Core
Enable the Service
Apply Configuration
Does it now report correctly in XI?

[weapon]

Tried that but it did not work. Still ok in Core, still stuck in pending in NagiosXI.