check_http odd behavior

[lcroce]

Hey everyone,

New to the forums but not new nagios user here. I recently run into an odd problem that I can't figure out. I just set up a new nagios server on CentOS 7 (my old one was on CentOS6) and several of my website checks have started returning a 400 Bad Request whereas the same exact checks work on my old server. The configs are identical and I migrated the IP address of the old server to the new one so it can't be a network based acl that's causing this. The only difference between the two servers other than potential software versions (nagios version is the same) is that my old CentOS 6 box was 32 bit whereas the new server is 64 bit. I'm posting the command definition and the results of running check_http from the terminal. Any ideas?

Code: Select all

define command {
                command_name                          check_cert_https_port_days
                command_line                          $USER1$/check_http -I $HOSTADDRESS$ -p $ARG1$ -C $ARG2$
}

NEW
./check_http -H 10.0.10.1 -p 4433 -C 45
SSL OK - Certificate '<removed info>' will expire on ****. HTTP WARNING: HTTP/1.1 400 Bad Request - 903 bytes in 0.014 second response time |time=0.013582s;;;0.000000 size=903B;;;0

OLD
 ./check_http -H 10.0.10.1 -p 4433 -C 45
SSL OK - Certificate '<removed info>' will expire on ****. HTTP OK: HTTP/1.1 302 Found - 908 bytes in 0.023 second response time |time=0.023273s;;;0.000000 size=908B;;;0

[mcapra]

I'd be interested in which versions of the check_http plugin are in-use on the old and new machines. check_http -V should print the version number.

Also if any sort of proxying/NATing is in place.

[lcroce]

2.2.1 on both as provided by the EPEL repo. No proxying or NATing that would affect these checks that are failing. Interestingly enough, curl's output mirrors the results of the check_http plugin. The old server returns a 302 while the new one returns a 400.

[tmcdonald]

The fact that curl is also having issues tells me it is in one of the system libraries, likely openssl. What is the output of openssl version on both the old and new servers? For reference, my Cent 6 is OpenSSL 1.0.1e-fips 11 Feb 2013 and Cent 7 is OpenSSL 1.0.2k-fips 26 Jan 2017.

[lcroce]

Yes, those versions match what I have. I started suspecting openssl this morning although I'm not sure how it would cause a different http return code unless there's a bug in that version of openssl. None of the known issues stand out to me. My Fedora 26 box has OpenSSL 1.1.0g-fips 2 Nov 2017 which works fine with curl so it's unlikely that this is some change in behavior by design.

[dwhitfield]

What the output if you add a -v to the end of those check_http commands?

[lcroce]

Included below.

Code: Select all

/check_http -H 10.0.10.1 -p 4433 -C 45 -v
SSL initialized
SSL OK - Certificate '<removed>' will expire on 2019-06-03 19:59 -0400/EDT. GET / HTTP/1.1
User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)
Connection: close
Host: 10.0.10.1:4433
Accept: */*


https://10.0.10.1:4433/ is 903 characters
STATUS: HTTP/1.1 400 Bad Request
**** HEADER ****
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Proxy-Connection: close
Connection: close
Content-Length: 727
**** CONTENT ****

<content removed>

HTTP WARNING: HTTP/1.1 400 Bad Request - 903 bytes in 0.012 second response time |time=0.012289s;;;0.000000 size=903B;;;0

[tacolover101]

what is the output if you run curl http://10.0.10.1:4433 -v?

[lcroce]

See below.

Code: Select all

curl https://10.0.10.1:4433 --insecure -vvv
* About to connect() to 10.0.10.1 port 4433 (#0)
*   Trying 10.0.10.1...
* Connected to 10.0.10.1 (10.0.10.1) port 4433 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
*       subject: CN=<content removed>
*       start date: Mar 04 00:00:00 2016 GMT
*       expire date: Jun 03 23:59:59 2019 GMT
*       common name: <content removed>
*       issuer: CN=RapidSSL SHA256 CA - G2,O=GeoTrust Inc.,C=US
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.0.10.1:4433
> Accept: */*
>
< HTTP/1.1 400 Bad Request
< Cache-Control: no-cache
< Pragma: no-cache
< Content-Type: text/html; charset=utf-8
< Proxy-Connection: Keep-Alive
< Connection: Keep-Alive
< Content-Length: 727
<

<content removed>

* Connection #0 to host 10.0.10.1 left intact

[npolovenko]

@lcroce, Can you open a web browser on a new server and try to access this web page:

Code: Select all

https://10.0.10.1:4433/ 

Would you be able to see the contents or would it take you to a nonexisting page?
Could there be some kind of a login page that automatically authenticates the old server but denies the access to a new server? Or is it more of a static web page?