Page 1 of 2

Problems scheduling downtime

Posted: Mon Aug 10, 2020 9:04 am
by NAGIOS SUPPORT ADMIN
Hello, when I attempt to schedule downtime when in the host details area nothing is added to the "Scheduled Downtime" section. Additionally when I attempt to schedule downtime under "Incident Management" I receive an error message saying "One or more scheduled commands could not be sent to Nagios Core." I have verified that the machine is able to resolve "localhost" per some earlier resolutions. When I check the audit log I see that the command to schedule the downtime was run.
64 bit VM
Nagios version: 5.4.11
CentOS Linux release 7.8.2003 (Core)
This server does not have internet access.

Please let me know how to resolve this issue.
Thank you.
Dave

Re: Problems scheduling downtime

Posted: Tue Aug 11, 2020 1:45 pm
by jbrunkow
Hello Dave!

Welcome to the forum. :D

Do you see anything in your Apache log for when the down time was supposed to occur?

Code: Select all

/var/log/httpd/error_log
You may want to try restarting the Apache service.

Code: Select all

systemctl restart httpd.service
Troubleshooting Scheduled Downtime

It never hurts to send ( PM ) us a profile either! We can find lots of information about your system in there. You can download the profile by navigating to System Profile under System Config on the left pane > and clicking the Download Profile button on that page.

Re: Problems scheduling downtime

Posted: Wed Aug 12, 2020 10:37 am
by NAGIOS SUPPORT ADMIN
Thanks for your reply.
I do not see any logs in when the downtime is supposed to occur or when I am trying to schedule the down time.
When I utilize the reoccurring downtime I am able to schedule downtime and there are no issues.
I have restarted the httpd service with no changes.
Thanks
Dave

Re: Problems scheduling downtime

Posted: Wed Aug 12, 2020 4:42 pm
by jbrunkow
What are the output of these commands?

Code: Select all

ls -ld /usr/local/nagios/var/rw
ls -l /usr/local/nagios/var/rw
chage -l nagios
What do you see in your cron log?

Code: Select all

/var/log/cron
Please send ( PM ) me a system profile. Navigate to System Profile under System Config on the left pane > then click the Download Profile button on that page.

Re: Problems scheduling downtime

Posted: Thu Aug 13, 2020 1:12 pm
by NAGIOS SUPPORT ADMIN
This is a tail -f of the cron log when I try to schedule downtime. I started it before and stopped after.
Aug 13 11:27:01 nagios CROND[5100]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Aug 13 11:28:01 nagios CROND[13969]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Aug 13 11:28:01 nagios CROND[13971]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Aug 13 11:28:01 nagios CROND[13973]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Aug 13 11:28:01 nagios CROND[13975]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Aug 13 11:28:01 nagios CROND[13980]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Aug 13 11:28:01 nagios CROND[13981]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Aug 13 11:28:01 nagios CROND[13979]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Aug 13 11:28:01 nagios CROND[13989]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Aug 13 11:28:01 nagios CROND[13990]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Aug 13 11:29:01 nagios CROND[22946]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Aug 13 11:29:01 nagios CROND[22948]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Aug 13 11:29:01 nagios CROND[22958]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Aug 13 11:29:01 nagios CROND[22959]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Aug 13 11:29:01 nagios CROND[22957]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Aug 13 11:29:01 nagios CROND[22956]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Aug 13 11:29:01 nagios CROND[22960]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Aug 13 11:29:01 nagios CROND[22961]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Aug 13 11:29:01 nagios CROND[22969]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)


Here is the output of the other commands:
[root@nagios ~]# ls -ld /usr/local/nagios/var/rw
drwxrwsr-x 2 nagios nagcmd 58 Aug 7 11:26 /usr/local/nagios/var/rw
[root@nagios ~]# ls -l /usr/local/nagios/var/rw
total 172
prw-rw---- 1 nagios nagcmd 0 Aug 9 12:01 nagios.cmd
srw-rw---- 1 nagios nagcmd 0 Aug 7 11:26 nagios.qh
-rw-r--r-- 1 root nagcmd 173808 Aug 6 08:54 u - postgres
[root@nagios ~]# chage -l nagios
Last password change : May 22, 2017
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7
[root@nagios ~]# ^C
You have new mail in /var/spool/mail/root
[root@nagios ~]#

I am not able to send a system profile right now as this is an offline system with no external connectivity. Is there something spacific that I could pull up from the profile?

Re: Problems scheduling downtime

Posted: Fri Aug 14, 2020 10:48 am
by jbrunkow
Thanks for sending that output. I don't see anything wrong there.

Run this tail command as root (and leave it running):

Code: Select all

tail -Fn0 /var/log/httpd/error_log /var/log/httpd/ssl_error_log /usr/local/nagiosxi/var/cmdsubsys.log
Then try submitting downtime a couple times and send us the full output from the still running tail command.

The other thing that could be causing it is if the associated contact has can submit commands to off. Navigate to Configure using the top bar of XI > click Core Config Manager under Advanced Configuration on the left pane > then click Contacts under Alerting > click on the relevant contact > select the Alert Settings tab > and make sure that Can Submit Commands is set to On.

Please PM this file as well:

Code: Select all

/usr/local/nagios/etc/cgi.cfg

Re: Problems scheduling downtime

Posted: Mon Aug 17, 2020 10:38 am
by NAGIOS SUPPORT ADMIN
I turned on submit commands for the user account (that was disabled). This did not change the results when scheduling down time.
Attached is the tail output you requested.

Re: Problems scheduling downtime

Posted: Tue Aug 18, 2020 2:23 pm
by jbrunkow
Please go to the host details section and try submitting the scheduled downtime again a couple times, once you've done that PM one of us a FRESH copy of your profile so we can analyze the logs, you can download the profile.zip file from Admin > System Profile by clicking the Download Profile button.

Re: Problems scheduling downtime

Posted: Wed Aug 19, 2020 6:11 am
by NAGIOS SUPPORT ADMIN
Is there a specific file from the profile that you want to see? This server is on an offline government network; I cannot pull the entire profile.

Re: Problems scheduling downtime

Posted: Wed Aug 19, 2020 4:56 pm
by jbrunkow
I see.

The system profile gathers many logs, settings, and configuration files in one fell swoop.

I suppose that the relevant logs would be the command subsystem log or recurring downtime log. You could take a look at what is being committed to those files.

Code: Select all

/usr/local/nagiosxi/var/cmdsubsys.log
/usr/local/nagiosxi/var/recurringdowntime.log
Log Locations and Descriptions

The trouble is that it's hard to know what's related until we see it. Do you think you could create a profile, then redact the information you identify as sensitive?