Problems scheduling downtime

[NAGIOS SUPPORT ADMIN]

Hello, when I attempt to schedule downtime when in the host details area nothing is added to the "Scheduled Downtime" section. Additionally when I attempt to schedule downtime under "Incident Management" I receive an error message saying "One or more scheduled commands could not be sent to Nagios Core." I have verified that the machine is able to resolve "localhost" per some earlier resolutions. When I check the audit log I see that the command to schedule the downtime was run.
64 bit VM
Nagios version: 5.4.11
CentOS Linux release 7.8.2003 (Core)
This server does not have internet access.

Please let me know how to resolve this issue.
Thank you.
Dave

[jbrunkow]

Hello Dave!

Welcome to the forum.

Do you see anything in your Apache log for when the down time was supposed to occur?

Code: Select all

/var/log/httpd/error_log

You may want to try restarting the Apache service.

Code: Select all

systemctl restart httpd.service

Troubleshooting Scheduled Downtime

It never hurts to send ( PM ) us a profile either! We can find lots of information about your system in there. You can download the profile by navigating to System Profile under System Config on the left pane > and clicking the Download Profile button on that page.

[NAGIOS SUPPORT ADMIN]

Thanks for your reply.
I do not see any logs in when the downtime is supposed to occur or when I am trying to schedule the down time.
When I utilize the reoccurring downtime I am able to schedule downtime and there are no issues.
I have restarted the httpd service with no changes.
Thanks
Dave

[jbrunkow]

What are the output of these commands?

Code: Select all

ls -ld /usr/local/nagios/var/rw
ls -l /usr/local/nagios/var/rw
chage -l nagios

What do you see in your cron log?

Code: Select all

/var/log/cron

Please send ( PM ) me a system profile. Navigate to System Profile under System Config on the left pane > then click the Download Profile button on that page.

[NAGIOS SUPPORT ADMIN]

This is a tail -f of the cron log when I try to schedule downtime. I started it before and stopped after.
Aug 13 11:27:01 nagios CROND[5100]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Aug 13 11:28:01 nagios CROND[13969]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Aug 13 11:28:01 nagios CROND[13971]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Aug 13 11:28:01 nagios CROND[13973]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Aug 13 11:28:01 nagios CROND[13975]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Aug 13 11:28:01 nagios CROND[13980]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Aug 13 11:28:01 nagios CROND[13981]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Aug 13 11:28:01 nagios CROND[13979]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Aug 13 11:28:01 nagios CROND[13989]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Aug 13 11:28:01 nagios CROND[13990]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Aug 13 11:29:01 nagios CROND[22946]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Aug 13 11:29:01 nagios CROND[22948]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Aug 13 11:29:01 nagios CROND[22958]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Aug 13 11:29:01 nagios CROND[22959]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Aug 13 11:29:01 nagios CROND[22957]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Aug 13 11:29:01 nagios CROND[22956]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Aug 13 11:29:01 nagios CROND[22960]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Aug 13 11:29:01 nagios CROND[22961]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Aug 13 11:29:01 nagios CROND[22969]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)


Here is the output of the other commands:
[root@nagios ~]# ls -ld /usr/local/nagios/var/rw
drwxrwsr-x 2 nagios nagcmd 58 Aug 7 11:26 /usr/local/nagios/var/rw
[root@nagios ~]# ls -l /usr/local/nagios/var/rw
total 172
prw-rw---- 1 nagios nagcmd 0 Aug 9 12:01 nagios.cmd
srw-rw---- 1 nagios nagcmd 0 Aug 7 11:26 nagios.qh
-rw-r--r-- 1 root nagcmd 173808 Aug 6 08:54 u - postgres
[root@nagios ~]# chage -l nagios
Last password change : May 22, 2017
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7
[root@nagios ~]# ^C
You have new mail in /var/spool/mail/root
[root@nagios ~]#

I am not able to send a system profile right now as this is an offline system with no external connectivity. Is there something spacific that I could pull up from the profile?

[jbrunkow]

Thanks for sending that output. I don't see anything wrong there.

Run this tail command as root (and leave it running):

Code: Select all

tail -Fn0 /var/log/httpd/error_log /var/log/httpd/ssl_error_log /usr/local/nagiosxi/var/cmdsubsys.log

Then try submitting downtime a couple times and send us the full output from the still running tail command.

The other thing that could be causing it is if the associated contact has can submit commands to off. Navigate to Configure using the top bar of XI > click Core Config Manager under Advanced Configuration on the left pane > then click Contacts under Alerting > click on the relevant contact > select the Alert Settings tab > and make sure that Can Submit Commands is set to On.

Please PM this file as well:

Code: Select all

/usr/local/nagios/etc/cgi.cfg

[NAGIOS SUPPORT ADMIN]

I turned on submit commands for the user account (that was disabled). This did not change the results when scheduling down time.
Attached is the tail output you requested.

[jbrunkow]

Please go to the host details section and try submitting the scheduled downtime again a couple times, once you've done that PM one of us a FRESH copy of your profile so we can analyze the logs, you can download the profile.zip file from Admin > System Profile by clicking the Download Profile button.

[NAGIOS SUPPORT ADMIN]

Is there a specific file from the profile that you want to see? This server is on an offline government network; I cannot pull the entire profile.

[jbrunkow]

I see.

The system profile gathers many logs, settings, and configuration files in one fell swoop.

I suppose that the relevant logs would be the command subsystem log or recurring downtime log. You could take a look at what is being committed to those files.

Code: Select all

/usr/local/nagiosxi/var/cmdsubsys.log
/usr/local/nagiosxi/var/recurringdowntime.log

Log Locations and Descriptions

The trouble is that it's hard to know what's related until we see it. Do you think you could create a profile, then redact the information you identify as sensitive?