Page 1 of 1
Multiple Comments For Services With Scheduled Down Time
Posted: Wed Nov 27, 2019 11:21 am
by vornado
Hello, all.
We have a good number of services that have scheduled downtime. Lately, when we view the details for any of these services, the comment under 'Acknowledgements and Comments' that the service has been scheduled for downtime is repeated many, many times (over 60 times, in some cases). Soon after I delete them (tedious doing one at a time and waiting between each one), they reappear.
I'm not sure when this started, but I recently upgraded php to version 7.2 and Nagios XI to version 5.6.8.
Please advise how to fix this so that the comment appears only once.
Thank you for your usual quick reply.
Steve
2019-11-27_11-14-36.png
Re: Multiple Comments For Services With Scheduled Down Time
Posted: Wed Nov 27, 2019 1:48 pm
by benjaminsmith
Hi Steve,
Thanks for uploading the screenshot. Let's tail the Apache log by running the following command when you load that page to capture any relevant error messages.
Also, if you got to the Home > Scheduled Downtime, is there just one entry for each service?
Lastly, please PM the system profile so we can review the system specs and logs files? Thank you.
To send us your system profile.
Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and share this in a private message and then
reply to this post to bring it up in the queue.
Re: Multiple Comments For Services With Scheduled Down Time
Posted: Mon Dec 02, 2019 3:25 pm
by vornado
Hi Benjamin.
[Home | Scheduled Downtime] lists 1,189 entries with many duplicates (see attached screenshot, 2019-12-02_15-13-10.png). I have also attached profile.zip and error-log-20191202.txt) When I sort by service, I can see that there are many duplicates for the same service.
If you need anything else, please let me know.
Steve
Moderator's Note: The profile has been shared with the support team but has been removed from the public forum.
Re: Multiple Comments For Services With Scheduled Down Time
Posted: Mon Dec 02, 2019 4:07 pm
by benjaminsmith
Hi Steve,
Thanks for sending over the system profile. The database log is showing crashed tables, please log in as root to the shell and run the following command.
Code: Select all
/usr/local/nagiosxi/scripts/repair_databases.sh
Let me know if the issue is resolved.
Re: Multiple Comments For Services With Scheduled Down Time
Posted: Tue Dec 03, 2019 1:19 pm
by vornado
I ran the command but it does not appear to have fixed the problem. I still have a ridiculous number of duplicate comments for services with scheduled downtime.
Yesterday, 'Home | Scheduled Downtime' reported there were 1,189 records. Today there are 2,095!!!
UPDATE
It looks like new records are generated every hour according to Entry Time in the Scheduled Downtime screen. In an example I'm looking at in the Service Status Detail screen, the first comment has the most recent time (14:01:02). All the comments that follow it have the same time (currently 13:21:05). Please see the attached file.
The number of comments matches the number of records in Scheduled Downtime for this service. Maybe I'm wrong, but I believe what should be shown on the Service Status Detail is only the most recent entry. If these records are being generated, can I just delete them all?
Re: Multiple Comments For Services With Scheduled Down Time
Posted: Tue Dec 03, 2019 4:26 pm
by benjaminsmith
Hello
@vornado,
The recurring downtime script is running as a cron job and the time settings are not in sync, and most likely causing the duplications.
set-timezone.png
Follow the article below to set the correct timezones and then go to Admin > System Config > System Profile to make the timezone, php time and system time are in sync.
Changing the System Time
Next, truncate the event queue messages from the database and let me know if the problem is resolved.
Code: Select all
echo "truncate table xi_events; truncate table xi_meta; truncate table xi_eventqueue;" | mysql -uroot -pnagiosxi -h 127.0.0.1 nagiosxi
Re: Multiple Comments For Services With Scheduled Down Time
Posted: Wed Dec 04, 2019 10:25 am
by vornado
Nagios, PHP and MySQL timezones are now synchronized:
2019-12-04_9-57-07.png
I ran the SQL you provided and verified that the 3 tables were empty, but I still see the duplicate comments.
Re: Multiple Comments For Services With Scheduled Down Time
Posted: Wed Dec 04, 2019 10:59 am
by lmiltchev
Since now your system and PHP time has been synced, you can try removing ALL of the scheduled downtimes by going to:
Home > Incident Management > Scheduled Downtime > Toggle all = checked > With selected = Remove > Continue
After doing this, let the scheduled dowtimes be recreated automatically. You have all of these entries under the "Recurring Downtime", correct?
If you continue having issues, you may need to remove the recurring downtimes as well, and recreate them.
Let us know if this helped.
Re: Multiple Comments For Services With Scheduled Down Time
Posted: Thu Dec 05, 2019 11:15 am
by vornado
After 'Home | Incident Management - Scheduled Downtime', everything seems to be in order. Thanks to you and Benjamin for your help.
Steve
Re: Multiple Comments For Services With Scheduled Down Time
Posted: Thu Dec 05, 2019 11:31 am
by lmiltchev
I am glad your issue has been resolved! I am locking this topic. If you have any further questions/issues, please start a new thread.