Cannot access Cluster Status page after 2.1.0 update

[rferebee]

Like this?

Code: Select all

; log_errors
;   Default Value: On
;   Development Value: On
;   Production Value: On

; error_log
;   /var/log/php-errors.log

I think I'm doing it wrong, it's not creating the file.

[mbellerue]

Before we do the PHP logging portion, can we talk about that nlsrepcc mount point? I saw some errors related to it, and I'm wondering if it's storing Log Server data? Maybe it's not mounted correctly or something?

[rferebee]

/nlsrepcc is my Log Repository.

Here's what I'm using for the mounting command:

Code: Select all

//10.128.207.113/NLSREPCC /nlsrepcc cifs rw,username=nagios,password=password,uid=994,gid=994,file_mode=0770,dir_mode=0770 0 0

It seems to be working in that I can write to it from my Log Servers, not sure about anything else that could be wrong other than the snapshots aren't working.

[rferebee]

Also, I might be crazy, but these numbers seem a bit low to me... I don't remember ever seeing this much space taken up. I think it's usually around 1.5-1.8TB free.

[mbellerue]

Here's what I'm seeing in your Apache error log.

/usr/bin/du: cannot read directory '/nlsrepcc': Permission denied

Looks like it's just being thrown from the du command, but it may be worth double checking that both the Nagios and Apache users are able to work with that share.

[rferebee]

Ok, I'm looking at the users and groups on my Log Server and it doesn't look like the apache user is a member of the apache group. That doesn't seem correct.

The only member of the apache group is the nagios user.

Also, when I look at the permissions for the items in /nlsrepcc it shows the owner as an account named 'gluster' and the group 'nagios'. The nagios group has nagio, apache and nrpe as members.

I don't remember ever having to explicitly grant apache rights to the Log Repository. I have screen shots from prior configs and it looks like the only account that needs access is the nagios account.

[mbellerue]

That looks about right. I can't speak to the gluster user, but as long as the nagios group has write access, and both nagios and the Apache user are in the nagios group, you should be good to go there. Let's go back to PHP error logging.

In your /etc/php.ini file, right around line 500 you should see the uncommented log_errors = On line. At least it should be On. If it's not On, make it On. Below that line add error_log = /var/log/php_errors.log In the example you posted, they have semicolons at the beginning of each line. Those are commented out in the final configuration. Then go through the httpd restart process, load the problem pages, and let's see if we get an error.

[rferebee]

Yeah, I don't know. It's not working. I think I have the syntax correct, but it's not creating the log file.

Code: Select all

; html_errors
;   Default Value: On
;   Development Value: On
;   Production value: On

log_errors = On
error_log = /var/log/php-errors.log
;   Default Value: Off
;   Development Value: On
;   Production Value: On

; max_input_time
;   Default Value: -1 (Unlimited)
;   Development Value: 60 (60 seconds)
;   Production Value: 60 (60 seconds)

[rferebee]

Is this helpful? I found it here /var/log/httpd/error_log

Code: Select all

[Thu Oct 03 13:05:49.498337 2019] [mpm_prefork:notice] [pid 515] AH00170: caught SIGWINCH, shutting down gracefully
[Thu Oct 03 13:05:50.575789 2019] [suexec:notice] [pid 1032] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using fe80::250:56ff:fea0:2854. Set the 'ServerName' directive globally to suppress this message
[Thu Oct 03 13:05:50.608556 2019] [lbmethod_heartbeat:notice] [pid 1032] AH02282: No slotmem from mod_heartmonitor
[Thu Oct 03 13:05:50.622758 2019] [mpm_prefork:notice] [pid 1032] AH00163: Apache/2.4.6 (CentOS) PHP/5.4.16 configured -- resuming normal operations
[Thu Oct 03 13:05:50.622784 2019] [core:notice] [pid 1032] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Thu Oct 03 13:06:06.746023 2019] [:error] [pid 1035] [client 10.131.32.33:1959] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 77 bytes) in /var/www/html/nagioslogserver/application/libraries/Elasticsearch.php on line 0, referer: http://10.128.207.111/nagioslogserver/admin/
[Thu Oct 03 13:06:13.317828 2019] [:error] [pid 1037] [client 10.131.32.33:1962] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 79 bytes) in /var/www/html/nagioslogserver/application/libraries/Elasticsearch.php on line 0
[Thu Oct 03 13:08:10.330882 2019] [mpm_prefork:notice] [pid 1032] AH00170: caught SIGWINCH, shutting down gracefully
[Thu Oct 03 13:08:11.420339 2019] [suexec:notice] [pid 1754] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using fe80::250:56ff:fea0:2854. Set the 'ServerName' directive globally to suppress this message
[Thu Oct 03 13:08:11.452256 2019] [lbmethod_heartbeat:notice] [pid 1754] AH02282: No slotmem from mod_heartmonitor
[Thu Oct 03 13:08:11.467736 2019] [mpm_prefork:notice] [pid 1754] AH00163: Apache/2.4.6 (CentOS) PHP/5.4.16 configured -- resuming normal operations
[Thu Oct 03 13:08:11.467764 2019] [core:notice] [pid 1754] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Thu Oct 03 13:08:53.838379 2019] [:error] [pid 1757] [client 10.131.32.33:1997] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 32 bytes) in /var/www/html/nagioslogserver/application/libraries/Elasticsearch.php on line 0, referer: http://10.128.207.111/nagioslogserver/admin/instance_status
[Thu Oct 03 13:09:46.642805 2019] [:error] [pid 1758] [client 10.131.32.33:2022] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 78 bytes) in /var/www/html/nagioslogserver/application/libraries/Elasticsearch.php on line 0, referer: http://10.128.207.111/nagioslogserver/admin/
[Thu Oct 03 13:14:36.339450 2019] [mpm_prefork:notice] [pid 1754] AH00170: caught SIGWINCH, shutting down gracefully
[Thu Oct 03 13:14:37.427056 2019] [suexec:notice] [pid 3739] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using fe80::250:56ff:fea0:2854. Set the 'ServerName' directive globally to suppress this message
[Thu Oct 03 13:14:37.456973 2019] [lbmethod_heartbeat:notice] [pid 3739] AH02282: No slotmem from mod_heartmonitor
[Thu Oct 03 13:14:37.471025 2019] [mpm_prefork:notice] [pid 3739] AH00163: Apache/2.4.6 (CentOS) PHP/5.4.16 configured -- resuming normal operations
[Thu Oct 03 13:14:37.471056 2019] [core:notice] [pid 3739] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Thu Oct 03 13:14:57.606368 2019] [:error] [pid 3741] [client 10.131.32.33:2117] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 78 bytes) in /var/www/html/nagioslogserver/application/libraries/Elasticsearch.php on line 0
[Thu Oct 03 13:16:20.342685 2019] [:error] [pid 3825] [client 10.131.32.33:2160] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 85 bytes) in /var/www/html/nagioslogserver/application/libraries/Elasticsearch.php on line 0, referer: http://10.128.207.111/nagioslogserver/admin/
[Thu Oct 03 13:16:26.588027 2019] [:error] [pid 3830] [client 10.131.32.33:2166] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 72 bytes) in /var/www/html/nagioslogserver/application/libraries/Elasticsearch.php on line 0

[scottwilkerson]

Is this helpful? I found it here /var/log/httpd/error_log

Yes, lets edit /etc/php.ini and change this

Code: Select all

memory_limit = 128M

to this

Code: Select all

memory_limit = 512M

Then restart httpd

Code: Select all

systemctl restart httpd