Multiple Comments For Services With Scheduled Down Time

[vornado]

Hello, all.

We have a good number of services that have scheduled downtime. Lately, when we view the details for any of these services, the comment under 'Acknowledgements and Comments' that the service has been scheduled for downtime is repeated many, many times (over 60 times, in some cases). Soon after I delete them (tedious doing one at a time and waiting between each one), they reappear.

I'm not sure when this started, but I recently upgraded php to version 7.2 and Nagios XI to version 5.6.8.

Please advise how to fix this so that the comment appears only once.

Thank you for your usual quick reply.

Steve

2019-11-27_11-14-36.png

[benjaminsmith]

Hi Steve,

Thanks for uploading the screenshot. Let's tail the Apache log by running the following command when you load that page to capture any relevant error messages.

Code: Select all

tail -f /var/log/httpd/*error_log

Also, if you got to the Home > Scheduled Downtime, is there just one entry for each service?

Lastly, please PM the system profile so we can review the system specs and logs files? Thank you.

To send us your system profile.
Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and share this in a private message and then reply to this post to bring it up in the queue.

[vornado]

Hi Benjamin.

[Home | Scheduled Downtime] lists 1,189 entries with many duplicates (see attached screenshot, 2019-12-02_15-13-10.png). I have also attached profile.zip and error-log-20191202.txt) When I sort by service, I can see that there are many duplicates for the same service.

If you need anything else, please let me know.

Steve





Moderator's Note: The profile has been shared with the support team but has been removed from the public forum.

[benjaminsmith]

Hi Steve,

Thanks for sending over the system profile. The database log is showing crashed tables, please log in as root to the shell and run the following command.

Code: Select all

/usr/local/nagiosxi/scripts/repair_databases.sh

Let me know if the issue is resolved.

[vornado]

I ran the command but it does not appear to have fixed the problem. I still have a ridiculous number of duplicate comments for services with scheduled downtime.

Yesterday, 'Home | Scheduled Downtime' reported there were 1,189 records. Today there are 2,095!!!

UPDATE

It looks like new records are generated every hour according to Entry Time in the Scheduled Downtime screen. In an example I'm looking at in the Service Status Detail screen, the first comment has the most recent time (14:01:02). All the comments that follow it have the same time (currently 13:21:05). Please see the attached file.

The number of comments matches the number of records in Scheduled Downtime for this service. Maybe I'm wrong, but I believe what should be shown on the Service Status Detail is only the most recent entry. If these records are being generated, can I just delete them all?

[benjaminsmith]

Hello @vornado,

The recurring downtime script is running as a cron job and the time settings are not in sync, and most likely causing the duplications.

set-timezone.png

Follow the article below to set the correct timezones and then go to Admin > System Config > System Profile to make the timezone, php time and system time are in sync.

Changing the System Time

Next, truncate the event queue messages from the database and let me know if the problem is resolved.

Code: Select all

echo "truncate table xi_events; truncate table xi_meta; truncate table xi_eventqueue;" | mysql -uroot -pnagiosxi -h 127.0.0.1 nagiosxi

[vornado]

Nagios, PHP and MySQL timezones are now synchronized:

2019-12-04_9-57-07.png

I ran the SQL you provided and verified that the 3 tables were empty, but I still see the duplicate comments.

[lmiltchev]

Since now your system and PHP time has been synced, you can try removing ALL of the scheduled downtimes by going to:

Home > Incident Management > Scheduled Downtime > Toggle all = checked > With selected = Remove > Continue

After doing this, let the scheduled dowtimes be recreated automatically. You have all of these entries under the "Recurring Downtime", correct?

If you continue having issues, you may need to remove the recurring downtimes as well, and recreate them.

Let us know if this helped.

[vornado]

After 'Home | Incident Management - Scheduled Downtime', everything seems to be in order. Thanks to you and Benjamin for your help.

Steve

[lmiltchev]

I am glad your issue has been resolved! I am locking this topic. If you have any further questions/issues, please start a new thread.