PHP Fatal error: Maximum execution time

[onegative]

G 'Day Support,

I am seeing these errors in my /etc/httpd/logs/ssl_error_log.

The errors are as follows:

[Fri Oct 25 11:04:55.655682 2019] [:error] [pid 17169] [client 205.175.112.7:63108] PHP Fatal error: Maximum execution time of 90 seconds exceeded in /usr/local/nagiosxi/html/includes/utils-objects.inc.php on line 0, referer: https://nagiosxi.mcis.washington.edu/na ... w=comments

[Fri Oct 25 11:05:25.270664 2019] [:error] [pid 17913] [client 205.175.112.7:63120] PHP Fatal error: Maximum execution time of 90 seconds exceeded in /usr/local/nagiosxi/html/includes/dbauth.inc.php on line 0, referer: https://nagiosxi.mcis.washington.edu/na ... w=comments

Can someone help me understand where I might start looking to troubleshoot this?

Any help is greatly appreciated.

And as always; thanks for your help,
Danny

[lmiltchev]

It seems to me this could be a database-related issue. Is your db offloaded to a remote server? Perhaps, there was a connectivity issues or db is just slow. PHP could be looping through a large number of objects too, e.g. users (running bunch of queries), and it's hitting the limit, defined in the /etc/php.ini.

You could try increasing the resource limits in the php.ini file:

Code: Select all

max_execution_time =
max_input_time = 
memory_limit =
max_input_vars =

but most importantly, you would need to eliminate the main issue, so that you won't be hitting the limit again. I would look into optimizing your db (if needed) and/or troubleshooting any possible connectivity issues to the db.

[onegative]

@lmiltchev

So I have been increasing the max_execution_time to 120, 180, 240 & 300 but with the same results. Still seeing the error in the ssl_error.log.

[Sun Oct 27 13:23:23.819726 2019] [:error] [pid 17388] [client 205.175.112.7:53361] PHP Fatal error: Maximum execution time of 180 seconds exceeded in /usr/local/nagiosxi/html/includes/dbauth.inc.php on line 0, referer: https://nagiosxi.mcis.washington.edu/na ... w=comments

[Sun Oct 27 13:32:11.445736 2019] [:error] [pid 30456] [client 205.175.112.7:53657] PHP Fatal error: Maximum execution time of 240 seconds exceeded in /usr/local/nagiosxi/html/includes/dbauth.inc.php on line 0, referer: https://nagiosxi.mcis.washington.edu/na ... w=comments

[Sun Oct 27 13:40:38.572725 2019] [:error] [pid 10208] [client 205.175.112.7:54076] PHP Fatal error: Maximum execution time of 300 seconds exceeded in /usr/local/nagiosxi/html/includes/dbauth.inc.php on line 0, referer: https://nagiosxi.mcis.washington.edu/na ... w=comments


I am wondering what is actually calling the specific URL that is timing out. I tired watching the access_log for the request but am not seeing it. Are there any internal php calls requesting the status.php page? I am seeing a very high CPU utilization on my httpd processes. The MySQL server is local to the web server and does not appear to be showing any distress with resources. I am at a loss here and really need help trying to figure what is actually happening and why.

Find attached a screen shot of top and I will send you PM of my system profile.

Thanks for your help,
Danny

[lmiltchev]

You database log is filled with errors like this one:

191027 13:41:45 [ERROR] mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed

First, we need to fix the crashed tables. This may or may not fix the original issue, but we need to do this first. See this:

https://support.nagios.com/kb/article/n ... es-24.html

You may also look into the db optimization documentation here:

https://assets.nagios.com/downloads/nag ... zation.pdf

After running the db repair script, check the mariadb.log for errors again:

Code: Select all

tail /var/log/mariadb/mariadb.log

[onegative]

@lmiltchev

Well I think I may have determined the issue. I went and modified the reaper settings and that seems to have solved the issue. I am seeing the mariadb increase in CPU utilization but the httpd processes seem much better and periodically go up but then back down. I am also no longer seeing the "PHP Fatal error: Maximum execution time" errors being reported in the ssl_error_log. I am going to watch this closely for the next several hours...I hope this has solved my problem....will let you know.

Thanks,
Danny

[lmiltchev]

I am glad we are moving to the right direction. We will keep the topic open for the time being. Let us know when it is OK to close it. Thanks!

[onegative]

This can be Locked.

The reaper settings seemed to have provided enough performance enhancement based on the number of checkresults files I process to stablize the system.

Averaging 20% total CPU
Averaging 1-2 for CPULoad (does jump to 4-7 during dbmaint.php runs but recovers quickly).
And my 1G /var/nagiosramdisk has yet to fill up and crater the system's ndo2db process.

Looking good even with 50,000+ objects being monitored.

I think if the Nagios was re-written in Java the OOP capabilities would greatly enhance each instance's ability to manage much larger environments...

But for now and the price I am satisfied...

Thanks for your help,
Danny

[scottwilkerson]

This can be Locked.

The reaper settings seemed to have provided enough performance enhancement based on the number of checkresults files I process to stablize the system.

Averaging 20% total CPU
Averaging 1-2 for CPULoad (does jump to 4-7 during dbmaint.php runs but recovers quickly).
And my 1G /var/nagiosramdisk has yet to fill up and crater the system's ndo2db process.

Looking good even with 50,000+ objects being monitored.

I think if the Nagios was re-written in Java the OOP capabilities would greatly enhance each instance's ability to manage much larger environments...

But for now and the price I am satisfied...

Thanks for your help,
Danny

Great!

Locking