Page 1 of 2
NagiosXI UI 'Last Check' Lagging Behind
Posted: Sun Apr 21, 2019 11:57 pm
by azreenariff
Hi Nagios Support,
We are currently monitoring 1680 hosts and 46980 services on NagiosXI 5.4.13, running on RHEL 7.4 with 16 CPU, 32GB memory. DB is offloaded on a separate MySQL 5.7 server.
On the Host or Service Status table page, our NagiosXI UI shows the 'Last Check' 30-60min behind (check intervals are every 5min). If we see this from Nagios Core UI, the 'Last Check' shows correct updated time.
Is this normal? What is causing this lagging from NagiosXI UI? Is there any way we can get the NagiosXI UI to show updated real-time of 'Last Check' time?
Thanks.
Re: NagiosXI UI 'Last Check' Lagging Behind
Posted: Mon Apr 22, 2019 9:54 am
by benjaminsmith
Hi
@azreenariff,
You'll want to check your time settings to make sure they are in sync on the Nagios XI server and the host for the DB. I've attached a link to documentation for setting system time. Check the following
1. Timezone settings in Nagios XI
2. System time and timezone
3. PHP timezone
See:
Changing The System Time
To the time on the offloaded database, run the following:
Code: Select all
echo "SELECT NOW();" | mysql -u root -pnagiosxi
Let us know if you are able to get it resolved.
Re: NagiosXI UI 'Last Check' Lagging Behind
Posted: Tue Apr 23, 2019 5:09 am
by azreenariff
Hi Benjamin,
Thank you for your reply. The time are same on the servers. I believe you misunderstood my problem.
What I meant was that on the NagiosXI UI, the Host/Service Status table page is showing statuses of 30-60min behind (it is supposed to show status of about 5min behind from current time, since our check intervals are set for every 5min - this shows correctly when we switch to look this from its Nagios Core UI).
We need to know what could cause the lateness of the statuses in NagiosXI UI to get updated, whereas the Nagios Core UI is updated as expected? Is it because of the high number of hosts/services that makes NagiosXI UI slow to query the DB, or what might you think could be the cause?
Thanks.
Re: NagiosXI UI 'Last Check' Lagging Behind
Posted: Tue Apr 23, 2019 10:58 am
by benjaminsmith
Hi
@azreenariff,
Thanks for checking those. Timezone errors can cause unexpected behavior but if that checks out, I'd like to review the system profile for errors.
However, since you have an offloaded database the logs will not be in the system profile. Can you post the database logs as well (
/var/log/mysqld.log)?
Thank you.
To send us your system profile.
Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and share in a private message or upload it to the post/ticket.
Re: NagiosXI UI 'Last Check' Lagging Behind
Posted: Thu Apr 25, 2019 5:00 pm
by benjaminsmith
Hi
@azreenariff,
Looking at the database log file, you have some corrupted database tables. Please run the repair script as root from the terminal:
Code: Select all
/usr/local/nagiosxi/scripts/repair_databases.sh
Reference:
Repairing The Nagios XI Databases
Re: NagiosXI UI 'Last Check' Lagging Behind
Posted: Thu Apr 25, 2019 9:11 pm
by azreenariff
Hi Benjamin,
Yes, thank you for the advice. We have actually repaired the database tables even before we posted the log. Kindly ignore the this.
Just need to know how we can get the NagiosXI to be able to display on the UI interface up to the latest check statuses, and not lagging behind too much. Could it be due to the slowness of the php of the UI in getting the display updated because of the large number of hosts & services that we have?
Thanks.
Re: NagiosXI UI 'Last Check' Lagging Behind
Posted: Fri Apr 26, 2019 10:58 am
by benjaminsmith
Hello
@azreenariff,
The system load is high and that might causing the delay. Please start by following the guide below to optimize your php settings.
Nagios XI - Optimizing The PHP Settings File
If you have deadpool processor enabled, please disable it for now. Also, we'd like to review your etc hosts file (
/etc/hosts ).
***> One or more problems was encountered while processing the config files...
Check your configuration file(s) to ensure that they contain valid
directives and data defintions.
It looks like there are some errors in the configuration files. Go to Configure > Config File Management > Verify Files and correct any issues.
Lastly, let's check the nagios user account. What is the output of the following command?.
Thanks.
Re: NagiosXI UI 'Last Check' Lagging Behind
Posted: Fri Apr 26, 2019 11:21 pm
by azreenariff
Hi Benjamin,
Thank you so much with all the recommendations. I will be out of office until Thursday next week. Will get back to you again and update you on Friday.
Thanks.
Re: NagiosXI UI 'Last Check' Lagging Behind
Posted: Mon Apr 29, 2019 9:27 am
by benjaminsmith
Hi
@azreenariff,
Sounds good. I will wait for your reply.
Re: NagiosXI UI 'Last Check' Lagging Behind
Posted: Fri May 03, 2019 2:03 am
by azreenariff
Hi Benjamin,
Regarding the optimization settings, we have already performed them some time ago and increased accordingly. The deadpool processor is not enabled.
/etc/hosts as below:
127.0.0.1 localhost localhost.domain localhost4 localhost4.localdomain4
#::1 localhost localhost.domain localhost6 localhost6.localdomain6
10.103.12.94 plnagaphm0a.khpuo.khpu.my plnagaphm0a
Verify files as below:
Running pre-flight check on configuration data...
Checking objects...
Checked 46939 services.
Checked 1781 hosts.
Checked 68 hosts groups.
Checked 4 service groups.
Checked 83 contacts.
Checked 35 contact groups.
Checked 233 commands.
Checked 21 time periods.
Checked 0 host escalations.
Checked 0 service escalations.
Checking for circular paths...
Checked 1781 hosts
Checked 0 host dependencies
Checked 21 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...
Total Warnings: 0
Total Errors: 0
Things look okay - no serious problems were detected during the pre-flight check
chage -l nagios as below:
Last password change : Jul 16, 2018
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : -1
Maximum number of days between password change : -1
Number of days of warning before password expires : -1
Thanks.