White page trying to import many unconfigured objects

[f.dantoni]

Hallo,
I'm configuring a trial version of Nagios XI 5.7.2. (we're about to activate an enterprise license)
When importing a small number of unconfigured passive objects (from a Nagios core installation on another machine) using , everything works as expected, everything is configured as expected.
Trying to import ALL the unconfigured objects, I receive a white page, with the following httpd log:

Code: Select all

[Tue Aug 25 10:25:48.323071 2020] [proxy_fcgi:error] [pid 185812:tid 140593237833472] [client 10.255.219.151:63858] Premature end of script headers: missingobjects.php, referer: http://10.1.200.83/nagiosxi/admin/missingobjects.php
[Tue Aug 25 10:25:48.323153 2020] [proxy_fcgi:error] [pid 185812:tid 140593237833472] [client 10.255.219.151:63858] AH01070: Error parsing script headers, referer: http://10.1.200.83/nagiosxi/admin/missingobjects.php
[Tue Aug 25 10:25:48.323166 2020] [proxy_fcgi:error] [pid 185812:tid 140593237833472] (22)Invalid argument: [client 10.255.219.151:63858] AH01075: Error dispatching request to : , referer: http://10.1.200.83/nagiosxi/admin/missingobjects.php

I also tried increasing the memory_limit parameter into php.ini config file (from 256M to 2048M) without success.
I've not found much information about this particular problem into the forums and on the internet, could you provide some hints to solve this problems?

Thanks,
Fabrizio

[benjaminsmith]

Hi Fabrizio,

Thanks for trying out Nagios XI. About how many unconfigured objects are you trying to import? You may need to increase some of the other PHP limits, please follow the steps in the kb article below to optimize your PHP settings on this server and let me know if the issue is resolved.

Nagios XI - Optimizing The PHP Settings File

[f.dantoni]

HI,

About how many unconfigured objects are you trying to import?

About 2000hosts testing 4500 services this time.

I'll try to modify some of the other php limits and reporto back.

Thanks for your support.

[f.dantoni]

Hi again,
I tried increasing the suggested values in /etc/php.ini up to 10x restarting the httpd daemon after each modify without success..... The error in the Apache logs is the same and is triggered immediately (instantly) after selecting all the hosts and pressing the gear icon at the end of the page.

Code: Select all

 ;max_input_vars = 5000
max_input_vars = 50000

;memory_limit = 256M
memory_limit = 2560M

;max_input_time = 120
max_input_time = 1200

[Wed Aug 26 10:35:11.220547 2020] [core:notice] [pid 1110076:tid 139693651351872] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Wed Aug 26 16:08:28.982571 2020] [proxy_fcgi:error] [pid 1110081:tid 139692913706752] [client 10.255.219.175:57872] Premature end of script headers: missingobjects.php, referer: http://10.1.200.83/nagiosxi/admin/missingobjects.php
[Wed Aug 26 16:08:28.982631 2020] [proxy_fcgi:error] [pid 1110081:tid 139692913706752] [client 10.255.219.175:57872] AH01070: Error parsing script headers, referer: http://10.1.200.83/nagiosxi/admin/missingobjects.php
[Wed Aug 26 16:08:28.982637 2020] [proxy_fcgi:error] [pid 1110081:tid 139692913706752] (22)Invalid argument: [client 10.255.219.175:57872] AH01075: Error dispatching request to : , referer: http://10.1.200.83/nagiosxi/admin/missingobjects.php

Thanks Again.

[benjaminsmith]

Hi,
Hi,
Thanks for the update. Since it is working on the smaller number, I'm quite certain this is a php/apache issue. They system is hitting a limit. Are you running Cent 8, if so, please restart php-fpm.

Code: Select all

sudo systemctl restart php-fpm

Also, make sure that the execution time has been increase as well in the php.ini file.

Code: Select all

max_execution_time = 90

Try adding these to the bottom of your /etc/httpd/conf/httpd.conf:

Code: Select all

Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5

Restrat Apache and then test this once more and let me know-how it goes, and send us your system profile as well so we can review the other logs.

To send us your system profile.
Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and share in a private message or upload it to the post/ticket, and then reply to this post to bring it up in the queue.

[f.dantoni]

Good Morning,
I've tried adding those lines without any improvement, restarting all the daemons suggested.

Save the profile.zip file and share in a private message or upload it to the post/ticket, and then reply to this post to bring it up in the queue.

PM Sent with the requested profile.zip.

Another detail I noticed (probably unrelated): Centos8 comes with Python 3.6.8, and the script /usr/local/bin/snmptraphandling.py gives an error on line 21, making the SNMP Traps service unusable.
Correcting the error makes it work.

Running the script on another machine (Centos 6) with Python 2.6.6 works flawlessy.
I will probably post later on this subject.

Thanks.

[benjaminsmith]

Hi,

Thanks for the profile. Going over the logs I noticed a few issues that could be causing issues.

PHP Time: Thu, 27 Aug 2020 08:05:01 +0000
System Time: Thu, 27 Aug 2020 10:05:01 +0200

The time settings are off. This is not causing the current issue but Nagios is a scheduling engine at it's core, and any time issues can result in strange behaviors. The following cover syncing PHP time and the server time.

Changing The System Time

[core:notice] [pid 69384:tid 140306571999552] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0

It looks like SELinux is enabled. Nagios XI is not compatible with this policy package, so please disable that and try importing the configurations again. Thanks, Benjamin

See: How to Disable SELinux on CentOS 8

[f.dantoni]

Hi,
I've ho idea why this line is logged,

[core:notice] [pid 69384:tid 140306571999552] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0

because SELinux should be already disabled:

Code: Select all

[root@prnagxi01 ~]# sestatus
SELinux status:                 disabled


[root@prnagxi01 ~]# cat /etc/sysconfig/selinux
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted
# SETLOCALDEFS= Check local definition changes
SETLOCALDEFS=0
[root@prnagxi01 ~]#

Thanks for pointing me up the TZ mismatch.
Fabrizio

[ssax]

I labbed this up in EL8 and it's working properly.

Make sure your /etc/my.cnf.d/mysql-server.cnf has sql_mode="" set under the [mysqld] section:

Code: Select all

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysql/mysqld.log
pid-file=/run/mysqld/mysqld.pid
sql_mode=""

If you don't have it you'll need to add it then restart mysqld:

Code: Select all

systemctl restart mysqld

This is likely how low your max_execution_time is, please change it to 900 in your /etc/php.ini, then restart httpd:

Code: Select all

service httpd restart

If you still get a white screen, please increase it even higher to see if that resolves it, go crazy like 3600 because we're just seeing if that really is what the issue is.

Let us know the results.

[f.dantoni]

Hi,
Just tried all your suggestions without success.

1) added the line in /etc/my.cnf.d/mariadb-server.cnf and restarted mariadb
The DB is on another fresh Centos8 machine, as per "Offloading MySQL to Remote Server" document.

Code: Select all

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mariadb/mariadb.log
pid-file=/run/mariadb/mariadb.pid
sql_mode=""

2) increased max_execution_time to 900 (was 600), then to 3600, finally to 36000 restarting httpd and php-fpm services after every change.

Code: Select all

[Mon Aug 31 09:09:25.649596 2020] [suexec:notice] [pid 987953:tid 140240984234304] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using ::1. Set the 'ServerName' directive globally to suppress this message
[Mon Aug 31 09:09:25.663430 2020] [lbmethod_heartbeat:notice] [pid 987953:tid 140240984234304] AH02282: No slotmem from mod_heartmonitor
[Mon Aug 31 09:09:25.666022 2020] [mpm_event:notice] [pid 987953:tid 140240984234304] AH00489: Apache/2.4.37 (centos) OpenSSL/1.1.1c configured -- resuming normal operations
[Mon Aug 31 09:09:25.666041 2020] [core:notice] [pid 987953:tid 140240984234304] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Mon Aug 31 09:10:14.278240 2020] [proxy_fcgi:error] [pid 988271:tid 140240270386944] [client 10.255.219.182:51284] Premature end of script headers: missingobjects.php, referer: http://10.1.200.83/nagiosxi/admin/missingobjects.php
[Mon Aug 31 09:10:14.278293 2020] [proxy_fcgi:error] [pid 988271:tid 140240270386944] [client 10.255.219.182:51284] AH01070: Error parsing script headers, referer: http://10.1.200.83/nagiosxi/admin/missingobjects.php
[Mon Aug 31 09:10:14.278297 2020] [proxy_fcgi:error] [pid 988271:tid 140240270386944] (22)Invalid argument: [client 10.255.219.182:51284] AH01075: Error dispatching request to : , referer: http://10.1.200.83/nagiosxi/admin/missingobjects.php

but the errors remains the same in /var/log/httpd/error_log

Thanks.