check_http - Cannot make SSL connection

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
Locked
as300182
Posts: 36
Joined: Tue May 17, 2016 8:09 am

check_http - Cannot make SSL connection

Post by as300182 »

For years the following command worked just fine:

Code: Select all

check_http -f follow -H -S "xxxxxxxxxxxxx" -J /usr/local/nagios/libexec/cert/cert.pem -K /usr/local/nagios/libexec/cert/privatekey.pem -s "Home Page"
Then the certificate used for authentication changed. Now when I try this from the CLI with -v switch I get:

CRITICAL - Cannot make SSL connection.
6315:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:583:

The certificate that I used to extract the client cert and private key from works just fine in my browser. So I know that's good. This error is symptomatic of something missing in the certificate chain. However, I've added the CA cert to the Nagios server, and I've tried adding it to the cert.pem file. However, I don't think check_http looks at the servers certs, and I'm pretty sure only one cert can be in a pem file. I kind of proved this by changing the order of the client cert and CA cert in the cert.pem file. By putting the CA cert first I get a 'Private key does not match certificate' error. So I guess check_http stops reading the pem file when it gets to the first '-----END CERTIFICATE-----'!!

Does anyone have any suggestions for resolving this please?
rkennedy
Posts: 6579
Joined: Mon Oct 05, 2015 11:45 am

Re: check_http - Cannot make SSL connection

Post by rkennedy »

You can add multiple certificates to the same PEM file (root CA -> certificate).

Can you try to run this through curl, with -v to see if we're able to get more of an output to what's wrong? Post the full input / output for us to take a look at.

Additionally, are the permissions proper on your cert files so that Nagios can access them?
Former Nagios Employee
as300182
Posts: 36
Joined: Tue May 17, 2016 8:09 am

Re: check_http - Cannot make SSL connection

Post by as300182 »

You can add multiple certificates to the same PEM file (root CA -> certificate).
I tried this. The file format looks like this:

-----BEGIN CERTIFICATE-----
--
--
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
--
--
-----END CERTIFICATE-----

Does that seem right to you or is that my issue? I'm pretty sure check_http is stopping at the first -----END CERTIFICATE----- it comes to. That would be consistent with OpenSSL and other tools that can work with certs.
Additionally, are the permissions proper on your cert files so that Nagios can access them?
I'm pretty sure the permissions are correct otherwise Nagios couldn't have reported that the the key doesn't match the certificate. How would it know if it couldn't read the files? :-) So I think that's good. In any case they are at least identical to the old pem files that used to work.
Can you try to run this through curl?
I'll be honest. I have curl on the box but I have no idea how to run the check_http command using curl. Can you give me an example or some tips please?
bwallace
Posts: 1146
Joined: Tue Nov 17, 2015 1:57 pm

Re: check_http - Cannot make SSL connection

Post by bwallace »

I'm not sure what rkennedy had in mind with curl, but in an attempt to keep momentum going here, what happens when you run a basic openssl test from the nagios machine? I 'm curious about this as you've confirmed all is well via a browser. We're obviously removing the check itself from the equation here, but at least we'll confirm how this transaction unfolds with out the check cmd muddying things:

openssl s_client -connect yourServer:443
Be sure to check out the Knowledgebase for helpful articles and solutions!
as300182
Posts: 36
Joined: Tue May 17, 2016 8:09 am

Re: check_http - Cannot make SSL connection

Post by as300182 »

I tried this

Code: Select all

openssl s_client -connect yourServer:443
and got this result.

CONNECTED(00000003)
27676:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:583:

I then tried it with the certs one needs to login.

Code: Select all

openssl s_client -host xxxxxxxxx -port 443 -cert /usr/local/nagios/libexec/cert/cert.pem -key /usr/local/nagios/libexec/cert/privatekey.pem
this produced an almost identical result but with a slightly different error number.

CONNECTED(00000003)
29121:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:583:
rkennedy
Posts: 6579
Joined: Mon Oct 05, 2015 11:45 am

Re: check_http - Cannot make SSL connection

Post by rkennedy »

If that's the case through the command through openssl then something is actually wrong with your certificate. Can you try recreating both of them, and see if it works the second time around?

Certificates can be a pain at times if things aren't lining up exact.
Former Nagios Employee
as300182
Posts: 36
Joined: Tue May 17, 2016 8:09 am

Re: check_http - Cannot make SSL connection

Post by as300182 »

I seem to be getting closer to a solution but I'm still not there. I tried recreating the certificate but that made no difference.

I then added the complete certificate chain to the pem file and that made no difference. Then I discovered something interesting. I've been using OpenSSL to test the pem file as follows:

Code: Select all

openssl verify -verbose /usr/local/nagios/libexec/cert/test.pem
The pem file contains the following:

-----BEGIN CERTIFICATE-----
Client cert
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
Intermediate cert
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
Root CA cert
-----END CERTIFICATE-----

and at first this was giving me the all too familiar; error 2 at 1 depth lookup:unable to get issuer certificate

So then I decided to remove the the Root CA cert from the pem file and add it to the certificate store on the Nagios server.

Hey presto. I now get; /usr/local/nagios/libexec/cert/test.pem: OK

But that's as far as it goes. In OpenSSL with

Code: Select all

openssl s_client -host xxxxxxxxx -port 443 -cert /usr/local/nagios/libexec/cert/test.pem -key /usr/local/nagios/libexec/cert/privatekey.pem
I still get:
CONNECTED(00000003)
15780:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:583:

And in nagios, not much better.

Code: Select all

check_http -f follow -H -S "xxxxxxxxxxxxx" -J /usr/local/nagios/libexec/cert/test.pem -K /usr/local/nagios/libexec/cert/privatekey.pem -s "Home Page"
Still gets me;
CRITICAL - Cannot make SSL connection.
16210:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:583:

I can't see where I'm going wrong. The same certs work on a browser and OpenSSL confirms the files I'm using for authorisation are OK.

Where do I go from here?
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: check_http - Cannot make SSL connection

Post by ssax »

Is this going through a web proxy? If you're not sure, please ask you're network team if they know, I've seen issues with check_http, SSL, and proxies.
as300182
Posts: 36
Joined: Tue May 17, 2016 8:09 am

Re: check_http - Cannot make SSL connection

Post by as300182 »

Don't you want to kick yourself for missing something so obvious.

YES it was the proxy. If I changed the host from the domain name for the server (which goes via proxy) to the IP address of the server, it now works. :-)

I also discovered a couple of things that might be useful to others.

1. When passing certificates through check_http the pem file can contain the whole chain up to but not including the root CA. That has to go into the certificate store of the Nagios server. check_http appears to ignore it in the pem file and seems insistent on looking for it in the local cert store.
2. check_http seems to stop checking at the first match for a root CA that it finds in the servers certificate store. So any old and expired root CA’s should be removed from the certificate store of the Nagios server. You could re-organise certs so that the latest comes first, but why bother. They are of no use any more.

Well these observations were certainly true for me.

Many thanks to everyone for your patience and guidance with this one.
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: check_http - Cannot make SSL connection

Post by mcapra »

Glad you were able to find a solution! Locking this up
Former Nagios employee
https://www.mcapra.com/
Locked