Page 1 of 2
After upgrade all hosts and service checks stuck in pending
Posted: Sun Jun 18, 2017 3:28 pm
by ks6764
I have a server that was running 5.2.3 and I now have on 5.4.4.
I saved the previous config files and used the Nagios import tool to import the configs after installing 5.4.4.
The procedure seems to have run fine and I made sure the token is in place as the monitoring of our client servers is passive using nrdp.
Currently, all servers under Hostgroup Overview show as green and ok for everything. If I drill down to a server and its services, I see them, but the Status column has "pending", the "Duration" and "Last Check" columns have "N/A". Lastly, the "Status Information" column has No check results for service yet...
I have restarted all the services (Nagios, MySQL, and Apache) with no issues. I also logged on to one of the client servers being monitored and can run the nrdp based monitoring check as listed under the Nagios crontab and it states that it has sent 15 checks to the server.
What am I missing ??
Re: After upgrade all hosts and service checks stuck in pend
Posted: Mon Jun 19, 2017 9:53 am
by tgriep
If you login to the XI GUI and go to the Admin > Unconfigured Objects menu, do those host and service show up there?
You may need to re-configure them so the XI server will receive the passive checks again.
Re: After upgrade all hosts and service checks stuck in pend
Posted: Mon Jun 19, 2017 10:21 am
by ks6764
I checked the unconfigured objects and there is nothing there to import.
Re: After upgrade all hosts and service checks stuck in pend
Posted: Mon Jun 19, 2017 12:34 pm
by tgriep
Can you open the NRDP Server API by opening a Web Browser and use the following URL, replacing xxx.xxx.xxx.xxx for the IP address of the Nagios XI server
SSL not enabled on the Nagios server
SSL is enabled on the Nagios server
In the Submit Check Data field, put in the NRDP Token and click on the Submit Check Data button.
Then go to the Admin > Unconfigured Objects menu and do you see the test host called somehost?
Re: After upgrade all hosts and service checks stuck in pend
Posted: Mon Jun 19, 2017 1:40 pm
by ks6764
Bad Check results Dir error
Re: After upgrade all hosts and service checks stuck in pend
Posted: Mon Jun 19, 2017 1:51 pm
by tgriep
Without knowing if the system is setup to use a ramdisk, I will need you run some commands in the Nagios server and post the output.
Code: Select all
grep checkresults /usr/local/nagios/etc/nagios.cfg
grep check_results_dir /usr/local/nrdp/server/config.inc.php
ls -l /usr/local/nagios/var/spool/
ls -l /var/nagiosramdisk/spool/
Re: After upgrade all hosts and service checks stuck in pend
Posted: Mon Jun 19, 2017 2:14 pm
by ks6764
Code: Select all
# grep checkresults /usr/local/nagios/etc/nagios.cfg
check_result_path=/usr/local/nagios/var/spool/checkresults
# grep check_results_dir /usr/local/nrdp/server/config.inc.php
$cfg["check_results_dir"] = "/usr/local/nagios/var/spool/checkresults";
# ls -l /usr/local/nagios/var/spool
total 12
drwsrwsr-- 2 nagios apache 4096 Jun 19 14:12 checkresults
drwxrwxr-x 2 nagios nagios 4096 Jun 19 14:12 perfdata
drwxrwxr-x 2 nagios nagios 4096 Jun 19 14:12 xidpe
/var/nagiosramdisk/spool/ - no such file or directory
Re: After upgrade all hosts and service checks stuck in pend
Posted: Mon Jun 19, 2017 3:07 pm
by tgriep
The permissions are not correct for the checkresults folder. Login as root, run the following to fix it.
Code: Select all
chown nagios.nagcmd /usr/local/nagios/var/spool/checkresults
chmod 775 /usr/local/nagios/var/spool/checkresults
chmod g+s /usr/local/nagios/var/spool/checkresults
service nagios restart
service httpd restart
Then login to the NRDP API and see id you can submit a command. Then check and see if the server starts to receive the passive data from your remote hosts.
Re: After upgrade all hosts and service checks stuck in pend
Posted: Mon Jun 19, 2017 3:20 pm
by ks6764
I changed the permissions as follows and restarted the services. Still getting the Bad Check Results Dir message.
Code: Select all
# ls -l
total 12
drwxrwsr-x 2 nagios nagcmd 4096 Jun 19 15:17 checkresults
drwxrwxr-x 2 nagios nagios 4096 Jun 19 15:17 perfdata
drwxrwxr-x 2 nagios nagios 4096 Jun 19 15:17 xidpe
As expected, when I run a
Code: Select all
/usr/local/nrdp/clients/nrds/nrds.pl -H 'myhost'
from a client box that is being monitored, it returns
Code: Select all
ERROR: The NRDP Server said BAD CHECK RESULTS DIR
Re: After upgrade all hosts and service checks stuck in pend
Posted: Mon Jun 19, 2017 3:50 pm
by tgriep
Are any of the folders in the following path a symlink from another folder?
Code: Select all
/usr/local/nagios/var/spool/checkresults
If so edit the /usr/local/nrdp/server/config.inc.php file and update the check_results_dir to point the real location and not the symlink.
Code: Select all
$cfg["check_results_dir"] = "/usr/local/nagios/var/spool/checkresults";
Save and restart the Apache process by running
The PHP function the NRDP PHP software uses has issues with symlinks.