Page 1 of 2
Configuration files not updating
Posted: Tue Jan 05, 2016 1:00 pm
by drug
Periodically, after making changes to services within Nagios XI (5.2.3) I am notified that the database has been updated, but the red notification about requiring "Apply Configuration" does not appear. If I do click on Apply Configuration it reports "Configuration applied successfully." with no errors. I have verified that the changes are not written to the respective services configuration file. The directory and file permissions also look sane to me but I ran "/usr/local/nagiosxi/scripts/reset_config_perms" just in case.
I am able to Write Configs To File without receiving any errors but despite this the configuration file is not updated. Is there anything else I can try?
Thanks.
Re: Configuration files not updating
Posted: Tue Jan 05, 2016 1:54 pm
by hsmith
How often would you say that you are experiencing this issue?
Is the server under a very heavy load?
Re: Configuration files not updating
Posted: Tue Jan 05, 2016 2:29 pm
by drug
It's not being used very often since we're still setting it up during our trial period. There are presently only about 8 hosts and 30 services configured. The server has been using very little resources.
Re: Configuration files not updating
Posted: Tue Jan 05, 2016 5:20 pm
by tgriep
Can you run the following command as root on the XI server, click on Apply Config and post the output from the tail command?
Code: Select all
tail -f /usr/local/nagiosxi/var/cmdsubsys.log
Also can you run this and post the output too?
Thanks
Re: Configuration files not updating
Posted: Tue Jan 05, 2016 5:34 pm
by drug
Please find the (sanitized) output attached.
Re: Configuration files not updating
Posted: Wed Jan 06, 2016 1:02 pm
by drug
I should also note that the "Last Check" timestamp for one of the service checks in question is 6 years old. I have verified that the check was just recently executed. I've stopped nagios, removed the object cache files and retention.dat but the timestamp is still years behind (presently "2000-04-07 21:38:04").
Re: Configuration files not updating
Posted: Wed Jan 06, 2016 1:13 pm
by hsmith
What is the output of the following commands?
Code: Select all
date
php -r "echo date('r').PHP_EOL;"
mysql -unagiosxi -pn@gweb -e "SELECT NOW();"
Re: Configuration files not updating
Posted: Wed Jan 06, 2016 1:30 pm
by drug
Our database actually resides on a dedicated system but the results all return a year of 2016:
Wed Jan 6 13:29:31 EST 2016
Wed, 06 Jan 2016 18:29:40 +0000
2016-01-06 13:28:53
I should clarify that both systems are synced with NTP. The minute/second discrepancies appear because I executed these commands at different times in a different order.
Re: Configuration files not updating
Posted: Wed Jan 06, 2016 6:18 pm
by ssax
Are you seeing any errors (crashed tables, etc) in your offloaded DBs msyql log? What about on your XI server in /var/lib/pgsql/data/pg_log/postgresql-DAY.log.
Are you able to see the changes after saving in the CCM?
Please post the output of these commands:
Code: Select all
sestatus
chage -l nagios
df -h
df -i
grep nag /etc/group
grep "User \|Group " /etc/httpd/conf/httpd.conf
ls -ld /usr/local/nagios/etc
ls -l /usr/local/nagios/etc
ls -l /usr/local/nagios/etc/your/hosts/or/service/config/file/that/isn't/updating
Re: Configuration files not updating
Posted: Thu Jan 07, 2016 10:18 am
by drug
I do see some Warnings for MySQL but they appear to only relate to replication (we have a read-only slave not used directly by our XI system):
Unsafe statement written to the binary log using statement format since BINLOG_FORMAT
I will investigate and remedy these.
I should also note that I was able to make the same change to the affected service today and the red warning appeared indicating I needed to Apply Configuration, which I did and the update was written to the file. Undoing the change did not result in the warning and an Apply Configuration did not write the "undone" change to the file. I.e. it seems if enough time passes the data will eventually be written to a file. Could there be a cron job that's purging or cleaning something?
Meanwhile, I've attached the output from those commands as well as the most recent errors from pg_log.