Page 1 of 1
Report Generation extremely slow/failing.
Posted: Tue Jan 08, 2019 5:38 am
by biswajit.banerjee
All reports generation is very slow and some times it doesn't get generated at all.
The system has 8 cpus at the moment and 16GB RAM. load average: 3.92, 3.80, 4.11 & Host Status Summary 405.
Re: Report Generation extremely slow/failing.
Posted: Tue Jan 08, 2019 2:07 pm
by cdienger
Check out
https://support.nagios.com/kb/article/n ... s-829.html for some info on troubleshooting and settings to improve performance.
Re: Report Generation extremely slow/failing.
Posted: Wed Jan 09, 2019 1:38 am
by biswajit.banerjee
Hi ,
We do not observe any issue in the error logs with respect to the shared document link .
We have around 450 devices + 4500 services just to give you estimate .
If i run SLA or Availibilty report for 1 month it takes 2 + hours to complete . I hope that is not a normal behavior .
Please let me know what is required to fix this issue .
Thanks
Re: Report Generation extremely slow/failing.
Posted: Wed Jan 09, 2019 12:32 pm
by npolovenko
@biswajit.banerjee, Did you get a chance to look into apache error log while the report is hanging?
tail -50 /var/log/httpd/error_log
or
tail -50 /var/log/httpd/ssl_error_log
Run the following script as root to check the databases:
/usr/local/nagiosxi/scripts/repair_databases.sh
Have you modified the settings in your /etc/php.ini file? What's the current max_input_vars value and what's the memory limit?
Re: Report Generation extremely slow/failing.
Posted: Wed Jan 09, 2019 12:43 pm
by cdienger
Also, are you applying any filters to limit the report to host/hostgroup/service group? Configuring it so reports don't generate automatically(Admin > System Config > Performance Settings > Auto-Running) and then applying filters when you do need to run a report can help a lot.
Re: Report Generation extremely slow/failing.
Posted: Fri Jan 11, 2019 4:36 am
by biswajit.banerjee
Hi
We have done following
/usr/local/nagiosxi/scripts/repair_databases.sh
we have checked tail -50 /var/log/httpd/error_log and tail -50 /var/log/httpd/ssl_error_log , But No errors apart for few warnings
checked max_input_vars as 10000 in php.ini
memory_limit = 256M
max_execution_time = 120
max_input_time = 120
Admin > System Config > Performance Settings > Auto-Running Stopped
we are not applying any filters to limit the report to host/hostgroup/service group.
I hope i have answered you all question .
Please suggest the way ahead steps . Please let us know if any logs / Details are required for analysis , we will update on priority.
Thanks
Re: Report Generation extremely slow/failing.
Posted: Fri Jan 11, 2019 6:19 am
by biswajit.banerjee
Attaching the logs
error_log and ssl_error_log
for your reference
Re: Report Generation extremely slow/failing.
Posted: Fri Jan 11, 2019 12:27 pm
by cdienger
Can you provide us with a time frame of when the report was run so we can narrow down our review of the logs?
Also, try increasing the memory setting in php.ini to 1024M
Reports are generated by parsing nagios.log and the logs in /usr/local/nagios/var/archives, which can be time consuming. Allocating more memory as described can help but the reports are often limited to disk speed.
Re: Report Generation extremely slow/failing.
Posted: Wed Jan 23, 2019 1:03 pm
by biswajit.banerjee
I have rebuild Nagios xi,but following issues coming...
1.Scheduled reports that are attached as PDF files to emails report as being corrupted when I try to view them.
2,Availability is not generated,
System profile ,screenshot & report attached.
Re: Report Generation extremely slow/failing.
Posted: Wed Jan 23, 2019 5:55 pm
by npolovenko
@biswajit.banerjee, How many files do you currently have in the archives folder?
cd /usr/local/nagios/var/archives/
ls | wc -l
I've seen issues with reports where XI would have lots of logs in the archives from years in past. Many reports have to render even the oldest reports in that folder searching for downtime entries.
Please increase the following settings in the php.ini to 120, restart apache with service httpd restart and let me know if that fixes the problem.
default_socket_timeout = 60
mysql.connect_timeout = 60