Nagios Support Forum

Posted: **Mon Jul 26, 2021 8:33 am**

Both my NLS hosts are coming up with Gateway Timeout errors. I restarted httpd and it has made no difference.

Posted: **Mon Jul 26, 2021 9:10 am**

This is critical on both hosts:

Screenshot 2021-07-26 101000.jpg

Posted: **Mon Jul 26, 2021 12:40 pm**

Please PM me a copy of your profile.zip from each node, you can download it from Admin > System Status by clicking the Download System Profile button.

What OS/version are they running?

Code: Select all

uname -a
cat /etc/*release

Posted: **Mon Jul 26, 2021 2:41 pm**

Wish I could. Since my post one started working periodically but not enough for me to get to the admin page. The other started loading the "waiting on elasticsearch" page but is stuck on that.

Both systems are identical:

Code: Select all

[root@cms-chi-nls02 ~]# uname -a
Linux cms-chi-nls02.itciss.com 4.18.0-240.el8.x86_64 #1 SMP Wed Sep 23 05:13:10 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux
[root@cms-chi-nls02 ~]# cat /etc/*release
NAME="Red Hat Enterprise Linux"
VERSION="8.3 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.3"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
Red Hat Enterprise Linux release 8.3 (Ootpa)
Red Hat Enterprise Linux release 8.3 (Ootpa)

Posted: **Tue Jul 27, 2021 9:02 am**

I've rebooted both nodes and now on the Cluster Status screen I get http 500 error.

EDIT: I updated the memory limit in php.ini and that fixed the 500 error.

EDIT2: The 500 error is back, the php.ini change only "fixed" it for a minute. I see no errors in the httpd error_log.

Posted: **Tue Jul 27, 2021 12:17 pm**

You should be able to get the profile through SSH/WinSCP like so:

Code: Select all

rm -rf /tmp/system-profile*
/usr/local/nagioslogserver/scripts/profile.sh

Then send me the resulting

Code: Select all

/tmp/system-profile.tar.gz

If that doesn't work, run this tail command as root and leave it running:

Code: Select all

tail -Fn0 /var/log/httpd/*

Then replicate the 500 error and send the output.

Are you seeing any errors in /var/log/messages or anything?

Posted: **Wed Jul 28, 2021 8:50 am**

I sent you the profile. I rebooted the boxes one more time and the 500 error is no longer there. I restarted httpd but the php memory limit didnt seem to fix it until a reboot.

That said, I am continuing to get the gateway timeout on the nodes and I PM'd the profile from one of them. I am not seeing any errors in any log files (messages, http error log, etc).

Posted: **Wed Jul 28, 2021 6:01 pm**

Edit this file:

Code: Select all

/etc/httpd/conf.d/php.conf

Change this specific part (around line 46):

Code: Select all

<IfModule !mod_php5.c>
  <IfModule !mod_php7.c>
    # Enable http authorization headers
    SetEnvIfNoCase ^Authorization$ "(.+)" HTTP_AUTHORIZATION=$1

    <FilesMatch \.(php|phar)$>
        SetHandler "proxy:unix:/run/php-fpm/www.sock|fcgi://localhost"
    </FilesMatch>
  </IfModule>
</IfModule>

Add this:

Code: Select all

    Timeout 600
    RequestReadTimeout header=600,minrate=50
    RequestReadTimeout body=600,minrate=50
    <Proxy "unix:/run/php-fpm/www.sock|fcgi://localhost">
        ProxySet timeout=600
    </Proxy>

So it looks like this:

Code: Select all

<IfModule !mod_php5.c>
  <IfModule !mod_php7.c>
    # Enable http authorization headers
    SetEnvIfNoCase ^Authorization$ "(.+)" HTTP_AUTHORIZATION=$1

    Timeout 600
    RequestReadTimeout header=600,minrate=50
    RequestReadTimeout body=600,minrate=50
    <Proxy "unix:/run/php-fpm/www.sock|fcgi://localhost">
        ProxySet timeout=600
    </Proxy>

    <FilesMatch \.(php|phar)$>
        SetHandler "proxy:unix:/run/php-fpm/www.sock|fcgi://localhost"
    </FilesMatch>
  </IfModule>
</IfModule>

Then edit this file:

Code: Select all

/etc/php-fpm.d/www.conf

Set:

Code: Select all

pm.max_requests = 50

Then restart apache/php-fpm:

Code: Select all

systemctl restart httpd php-fpm

Let me know if that resolves it.

Posted: **Thu Jul 29, 2021 3:25 pm**

I don't get the gateway timeout issue anymore but any clicks take forever to load the webpage. The hosts are not under any sort of load.

Posted: **Fri Jul 30, 2021 9:24 am**

Can you PM a fresh copy of your profile so I can look at the latest logs?

What is top showing?

Code: Select all

top -c -n3

Nagios Support Forum

Gateway timeout

Gateway timeout

Re: Gateway timeout

Re: Gateway timeout

Re: Gateway timeout

Re: Gateway timeout

Re: Gateway timeout

Re: Gateway timeout

Re: Gateway timeout

Re: Gateway timeout

Re: Gateway timeout