Extremely slow load times for graphs after PHP upgrade

[Atria]

Due to a security remediation I had to upgrade the PHP on my Nagios box to 5.6.18 from the 5.4 version included. Since the upgrade I have noticed that the graphical elements of my deployment take upwards of 20-30 seconds to load. Initially I thought this was exclusive to nagvis, but all graphs and other elements in XI are taking the same amount of time. I notice when I run php -version I get the following error.

Code: Select all

PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/lib64/php/modules/ixed.5.4.lin' - /usr/lib64/php/modules/ixed.5.4.lin: undefined symbol: zend_get_hash_value in Unknown on line 0

According to what I've seen on the internet this should be trying to load from the php.ini located in /etc/php.ini, and should be pointed to the ixed.5.6.lin, however even after changing this value and restarting the VM, I still see this error. This may not be related to the slowdown, but it's all I've been able to really pin down.

[tgriep]

Nagios uses Source guardian for PHP code and if you upgrade PHP, you need to reinstall Source Guardian with the same PHP module version as PHP.

If you have a copy of the source of Nagios XI on your server, you can run this script to re-install it.
Login as root to your XI server and run the following

Code: Select all

cd /tmp/nagiosxi
./install-sourceguardian-extension.sh
service httpd restart

Try that out and see if this fixes the issue.

[Atria]

When I try that I get the following.

Code: Select all

[frack@monitoring nagiosxi]$ sudo ./install-sourceguardian-extension.sh 
[sudo] password for frack: 
PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/lib64/php/modules/ixed.5.4.lin' - /usr/lib64/php/modules/ixed.5.4.lin: undefined symbol: zend_get_hash_value in Unknown on line 0
Archive:  sourceguardian/ixed4.lin.x86-32.zip
checkdir:  cannot create extraction directory: /usr/lib/php/modules
           No such file or directory

[tgriep]

The install script uses PHP to get it's version and the bad source guardian config is causing it to fail.
Login as root to the server, sudo may not be enough, and run the following to remove the old config file.

Code: Select all

mv /etc/php.d/sourceguardian.ini /tmp

That will make a backup in case we need to go back.
Then try and run the procedure from the previous post.

[Atria]

Seem to have received the same result.

Code: Select all

[frack@monitoring nagiosxi]$ su root
Password: 
[root@monitoring nagiosxi]# mv /etc/php.d/sourceguardian.ini /tmp
[root@monitoring nagiosxi]# cd /tmp
[root@monitoring tmp]# cd nagiosxi/
[root@monitoring nagiosxi]# ./install-
install-2012-prereqs                 install-pnptemplates                 install-templates
install-html                         install-sourceguardian-extension.sh  
install-nagiosxi-init                install-sudoers                      
[root@monitoring nagiosxi]# ./install-sourceguardian-extension.sh 
Archive:  sourceguardian/ixed4.lin.x86-32.zip
checkdir:  cannot create extraction directory: /usr/lib/php/modules
           No such file or directory

[Atria]

Ok so upon further investigation there was no directory called /usr/lib/php/ so I made the folder, and ran the script again. This time getting

Code: Select all

[root@monitoring nagiosxi]# ./install-sourceguardian-extension.sh 
Archive:  sourceguardian/ixed4.lin.x86-32.zip
  inflating: /usr/lib/php/modules/ixed.5.6.lin  
Sourceguardian extension found for PHP version 5.6
Sourceguardian extension already in php.ini
[root@monitoring nagiosxi]# php -version
PHP 5.6.18 (cli) (built: Feb  3 2016 13:07:14) 
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies

Which appears to have cleared up that error, however I am still seeing 20-30 second load times for graphs. Any idea where I should look from here?

[tgriep]

In the /etc/php.ini settings to can edit the following to see if that helps.
They are examples but try increasing the memory_limit first.

Code: Select all

max_execution_time = 60     ; Maximum execution time of each script, in  seconds
max_input_time = 60     ; Maximum amount of time each script may spend parsing request data
memory_limit = 256M      ; Maximum amount of memory a script may consume 

After changing these you will have to restart Apache

Code: Select all

service httpd restart

[ssax]

Take a look in your /var/log/httpd/error_log and /var/log/httpd/ssl_error_log to see if that gives any indication.

Thank you

[Atria]

A few persistent errors

Code: Select all

[Fri Apr 15 09:16:09.559476 2016] [:error] [pid 21409] [client 10.0.219.118:62871] PHP Notice:  Undefined index: background in /usr/local/nagiosxi/html/includes/page-home-main.php on line 55, referer: http://monitoring.atriacom.com/nagiosxi/index.php

Code: Select all

[Fri Apr 15 05:51:36.158231 2016] [cgi:error] [pid 26390] [client 172.30.0.39:57314] script not found or unable to stat: /var/www/cgi-bin/test-cgi

Code: Select all

[Fri Apr 15 05:51:35.623900 2016] [:error] [pid 28245] [client 172.30.0.39:57207] script '/usr/local/nagiosxi/html/wp-login.php' not found or unable to stat

Code: Select all

[Thu Apr 14 16:02:56.955347 2016] [:error] [pid 12105] [client 10.0.219.126:18254] PHP Warning:  pg_pconnect(): PostgreSQL link lost, unable to reconnect in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-postgres64.inc.php on line 682, referer: http://monitoring.atriacom.com/nagiosxi//includes/page-home-main.php?&=

Code: Select all

[Thu Apr 14 16:02:55.922838 2016] [:error] [pid 12131] [client 10.0.219.106:65001] PHP Warning:  pg_pconnect(): Unable to connect to PostgreSQL server: could not connect to server: No such file or directory\n\tIs the server running locally and accepting\n\tconnections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"? in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-postgres64.inc.php on line 682, referer: http://monitoring.atriacom.com/nagiosxi/includes/components/xicore/status.php?show=hostdetail&host=cr10824-10824+Deer+Park

[Atria]

Also I modified the PHP script settings and saw no improvement.