avail.cgi - performance issues

[sutenag]

Hello,

We're currently unable to run the availability report, as when it is run, the CPU utilisation on the server hits 100% and the process never seems to return results. As a workaround, we have renamed avail.cgi to something else to avoid it being run upon opening the reports page. I have found that performance issues with this binary have been reported before here: https://github.com/NagiosEnterprises/na ... issues/280 and so there seems to be no fix until Nagios Core 5. I have two questions:

1. Is there a release date for Nagios Core 5?

2. Is there anything that we can do in the meantime to either get the availability reports to return in a more reasonable time, or extract the same information in another way?


Some information about the system in question:

Nagios XI Version : 5.4.13
monitoring-1 3.10.0-693.21.1.el7.x86_64 x86_64
Red Hat Enterprise Linux Server release 7.4 (Maipo)

CPU: Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
RAM: 16GB


Kind Regards,

Cuchulain

[benjaminsmith]

Hi Cuchulain,

Please send us you system profile (see below) so we can review you configuration settings and log files. The avail.cgi does parse the log files to generate reports and this is i/o intensive, so it's important that the debug mode is not enabled in the logging to help speed this up.

If the system is hanging, you may need to optimize the PHP settings. We have a step-by-step guide to change these settings:

Nagios XI - Optimizing The PHP Settings File

As far as Core 5 is concerned, it's in the planning/development stage right now, so I don't have release date ( more details at: roadmap ).

To Send Over a System ProfileTo send us your system profile.
Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and upload it to a cloud storage of your choice or attach it in a private message.

Thanks.

[sutenag]

Hi,

I have sent you the system profile in a PM. I will review the document regarding PHP settings and let you know if any changes are made.

Please let me know if the system profile reveals anything that could be causing the performance issues.

Kind Regards

Cuchulain

[sutenag]

Hi,

I have reviewed the suggestions outlined in the link you provided regarding optimising the PHP settings. The only setting which seems to require adjusting is the max_execution_time setting.

Code: Select all

[u][b]max_input_vars[/b][/u]

[root@monitoring-1 httpd]# grep -ic "Input variables" *
access_log:0
access_log-20190310:0
error_log:0
error_log-20190310:0
ssl_access_log:0
ssl_access_log-20190310:0
ssl_error_log:0
ssl_error_log-20190310:0
ssl_request_log:0
ssl_request_log-20190310:0

[root@monitoring-1 httpd]# grep -ic "max_input_vars" *
access_log:0
access_log-20190310:0
error_log:0
error_log-20190310:0
ssl_access_log:0
ssl_access_log-20190310:0
ssl_error_log:0
ssl_error_log-20190310:0
ssl_request_log:0
ssl_request_log-20190310:0


[u][b]memory_limit[/b][/u]

[root@monitoring-1 httpd]# grep -ic "Allowed memory size" *
access_log:0
access_log-20190310:0
error_log:0
error_log-20190310:0
ssl_access_log:0
ssl_access_log-20190310:0
ssl_error_log:0
ssl_error_log-20190310:0
ssl_request_log:0
ssl_request_log-20190310:0

[b]max_execution_time[/b]

[root@monitoring-1 httpd]# grep -ic "execution time" *
access_log:0
access_log-20190310:0
error_log:0
error_log-20190310:0
ssl_access_log:0
ssl_access_log-20190310:0
ssl_error_log:0
[b]ssl_error_log-20190310:128[/b]
ssl_request_log:0
ssl_request_log-20190310:0

suggest increasing max_execution_time = 90 as per document


[b][u]max_input_time[/u][/b]

[root@monitoring-1 httpd]# grep -ic "input time of" *
access_log:0
access_log-20190310:0
error_log:0
error_log-20190310:0
ssl_access_log:0
ssl_access_log-20190310:0
ssl_error_log:0
ssl_error_log-20190310:0
ssl_request_log:0
ssl_request_log-20190310:0

Kind Regards,

Cuchulain

[npolovenko]

@sutenag, Firstly, to prevent the availability report from automatically running instead of moving the CGI file you could check the box to disable reports from automatically running.

Untitled.png

Secondly, with the amount of host and service checks you have on your system, your IO wait is high and that impacts the report speed. You need to get a faster hard drive to improve IO.

Implementing a ramdisk could help with the IO wait. We have an easy automated script that installs it automatically.
https://assets.nagios.com/downloads/nag ... giosXI.pdf

You also have lots of logs coming to the syslog and by disabling some of them you will likely improve the system load and that should help speed up reports a little bit.

Stopping all third-party software that runs on the server should help the IO wait and lower the system load.

But generally speaking, on large systems, the reports are going to be slow no matter what. Unless you pick a very small interval for the report.