Page 1 of 2
Service Status stuck in pending
Posted: Wed Mar 18, 2015 10:59 pm
by weapon
I am running XI 2012R2.2, and have a service stuck in pending. I'm not sure why, as I have everything built off templates, so the only thing different with this check is that it uses check_by_ssh.
Digging in a bit deeper, the service is displaying pending in NagiosXI, but in nagios core it is in an OK state. Running
https://nagiosxi01/nagiosxi/backend/?cm ... vicestatus I find the following:
Code: Select all
<name>Distribution Check</name>
<host_display_name />
<display_name>Distribution Check</display_name>
<status_update_time>2015-03-19 12:12:16</status_update_time>
<status_text>OK - Removed for confidentiality </status_text>
<status_text_long />
<current_state>0</current_state>
<performance_data>Removed for confidentiality</performance_data>
<should_be_scheduled>0</should_be_scheduled>
<check_type>0</check_type>
<last_state_change>1970-01-01 10:00:00</last_state_change>
<last_hard_state_change>1970-01-01 10:00:00</last_hard_state_change>
<last_hard_state>0</last_hard_state>
<last_time_ok>1970-01-01 10:00:00</last_time_ok>
it looks like the last_state_change has not been updated correctly. I have tried restarting nagios and nagiosxi services, and I have tried renaming the service, but nothing seems to help. I have had a google around which has gotten me this far, but I can't get any further. Can someone advise how to resolve this issue?
Regards,
David
Re: Service Status stuck in pending
Posted: Thu Mar 19, 2015 12:06 am
by Box293
Can you show us the final service object definition in /usr/local/nagios/var/objects.cache
Also,
Run this command
Code: Select all
tail /var/log/mysqld.log -n 100 > /tmp/mysqld_log.txt
Send us this file:
/tmp/mysqld_log.txt
Re: Service Status stuck in pending
Posted: Thu Mar 19, 2015 12:34 am
by weapon
Because of confidentiality and size (2MB) I have only included the host and service entries
Code: Select all
define host {
host_name distro
alias
address 10.4.1.43
check_period 24x7
check_command check-host-alive!!!!!!!!
contact_groups <Contact Groups removed>
notification_period 24x7
initial_state o
check_interval 10.000000
retry_interval 2.000000
max_check_attempts 5
active_checks_enabled 1
passive_checks_enabled 1
obsess_over_host 1
event_handler_enabled 1
low_flap_threshold 0.000000
high_flap_threshold 0.000000
flap_detection_enabled 1
flap_detection_options o,d,u
freshness_threshold 0
check_freshness 0
notification_options d,u,r,f
notifications_enabled 1
notification_interval 120.000000
first_notification_delay 0.000000
stalking_options n
process_perf_data 0
failure_prediction_enabled 1
icon_image document_server.png
statusmap_image document_server.png
retain_status_information 1
retain_nonstatus_information 1
_XIWIZARD genericnetdevice
}
define service {
host_name distro
service_description Distribution Check
check_period 24x7
check_command check_xi_by_ssh!-C "./nagios_131940_distro.pl -p"!!!!!!!
contact_groups <Contact Groups removed>
notification_period 24x7
initial_state o
check_interval 10.000000
retry_interval 2.000000
max_check_attempts 3
is_volatile 0
parallelize_check 1
active_checks_enabled 1
passive_checks_enabled 1
obsess_over_service 1
event_handler_enabled 1
low_flap_threshold 0.000000
high_flap_threshold 0.000000
flap_detection_enabled 1
flap_detection_options o,w,u,c
freshness_threshold 0
check_freshness 0
notification_options u,w,c,r,f
notifications_enabled 1
notification_interval 120.000000
first_notification_delay 0.000000
stalking_options n
process_perf_data 1
failure_prediction_enabled 1
retain_status_information 1
retain_nonstatus_information 1
}
Re: Service Status stuck in pending
Posted: Thu Mar 19, 2015 2:07 pm
by tgriep
The ndo2mod service might need to be restarted.
Try restarting it and see if that fixes it for you.
Run this on the Nagios Server.
Re: Service Status stuck in pending
Posted: Thu Mar 19, 2015 6:10 pm
by weapon
That didn't seem to help. I restarted ndo2db, and it still was an issue. I tired restarting ndo2db, nagios and nagiosxi, and it still did not help. I looked at the backend getservicestatus again and everything is still the same (except the status update time is updated a bit).
Re: Service Status stuck in pending
Posted: Fri Mar 20, 2015 10:01 am
by lmiltchev
I may be wrong, but it seems like you don't have the full path in your check_command... You have:
Code: Select all
check_command check_xi_by_ssh!-C "./nagios_131940_distro.pl -p"
It should probably look like this:
Code: Select all
check_command check_xi_by_ssh!-C "/usr/local/nagios/libexec/nagios_131940_distro.pl -p"
Note: The path to your nagios plugins directory could be different.
Re: Service Status stuck in pending
Posted: Sun Mar 22, 2015 5:46 pm
by weapon
The issue is not with the check command, I can run that successfully as the nagios user. The path is a relative path, not absolute - and the plugin is in the home directory on the remote machine, it is not a path to a script on the Nagios machine.
Re: Service Status stuck in pending
Posted: Sun Mar 22, 2015 6:31 pm
by weapon
I've just rebooted Nagios (due to some other display issues) but this issue still remains.
Re: Service Status stuck in pending
Posted: Sun Mar 22, 2015 7:03 pm
by Box293
Something to try:
Go into CCM
Disable the Service
Apply Configuration
Confirm it does not appear in XI or Core
Enable the Service
Apply Configuration
Does it now report correctly in XI?
Re: Service Status stuck in pending
Posted: Sun Mar 22, 2015 7:26 pm
by weapon
Tried that but it did not work. Still ok in Core, still stuck in pending in NagiosXI.