NRPE: SSL Socket Shutdown.

[Koja]

Hello everybody!

I happened to notice that one of my CentOS 8 servers' NRPE checks was failing:



On the servers console, using the following command, I can see that there is an SSL error occurring: journalctl -eu nrpe -f

Code: Select all

Oct 15 10:39:04 AWX.verkko.local nrpe[3151]: Error: (use_ssl == true): Request packet version was invalid!
Oct 15 10:39:04 AWX.verkko.local nrpe[3151]: Could not read request from client 192.4.61.36, bailing out...
Oct 15 10:39:04 AWX.verkko.local nrpe[3151]: INFO: SSL Socket Shutdown.
Oct 15 10:39:07 AWX.verkko.local nrpe[3214]: Error: (use_ssl == true): Request packet version was invalid!
Oct 15 10:39:07 AWX.verkko.local nrpe[3214]: Could not read request from client 192.4.61.36, bailing out...
Oct 15 10:39:07 AWX.verkko.local nrpe[3214]: INFO: SSL Socket Shutdown.
Oct 15 10:39:08 AWX.verkko.local nrpe[3229]: Error: (use_ssl == true): Request packet version was invalid!
Oct 15 10:39:08 AWX.verkko.local nrpe[3229]: Could not read request from client 192.4.61.36, bailing out...
Oct 15 10:39:08 AWX.verkko.local nrpe[3229]: INFO: SSL Socket Shutdown.

At first, I tried restarting nrpe.service and rebooting the server. Momentarily, the issue went away, but then came back a few minutes later. As I looked more closely at NRPE's logs, I could see that this had been happening for a course of weeks.

For some reason, this particular NRPE service check gets randomly timed out because of a supposed SSL error.

Here is that service check on the server in question (/etc/nagios/nrpe.cfg):
command[check_updates]=/usr/lib64/nagios/plugins/check_updates --clean -t 120 -w 20 -c 30

I got the script from here: https://github.com/matteocorti/check_updates

This is really strange because, for example, right now I'm not getting any errors with the service, but fifteen minutes later I might. Any help on this would be greatly appreciated!

[Koja]

Both servers have NRPE Version: 4.0.3

[benjaminsmith]

Hi @Koja,

Thanks for posting the Nagios Community Forum! Normally, when there is the invalid packet error it's because the host is running an older version of nrpe than the XI/Core server.

Try adding the -n argument to your check command to rule out if this is related to SSL.

Let us know what you find out.

Benjamin

[Koja]

Hello @benjaminsmith,

When I run the command with the -n switch, I get the following output:

Code: Select all

CHECK_NRPE: Receive header underflow - only -1 bytes received (4 expected).

When using journalctl -eu nrpe on the target server, I can see that when I run the command without SSL, I get the following entry repeatedly on NRPE's logs:

Code: Select all

Oct 22 10:51:17 hostname.domain.corp nrpe[440]: Error: (!log_opts) Could not complete                                                                                                                                    SSL handshake with 10.10.10.100: 1

[Koja]

Is there any solution to this issue? I haven't been able to solve with my Google-fu.

[benjaminsmith]

Hi,

That's a common error message. Take a look at the following knowledgebase article for troubleshooting that one:

NRPE - CHECK_NRPE: Error - Could Not Complete SSL Handshake

And let us know if you're able to resolve it.
Benajmin

[Koja]

Hello @benjaminsmith,

I've tried the methods described in your link, I was pretty sure that compiling NRPE with SSL would solve the issue, but it didn't.

The strange thing is, only two services for this host are apparently getting the SSL error, all the other ones work fine:



When I, for example, run the Uptime check command locally, I get a more spesific error:

Code: Select all

[root@netflow ~]# /usr/lib64/nagios/plugins/check_uptime
/usr/lib64/nagios/plugins/check_uptime: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory

I found multiple solutions online on how to fix the libssl.so issue, but for some reason they're not working for this particular server.

[Koja]

Hello,

I have now figured out the issue. The problem can be resolved on both CentOS 7.8.2003 and CentOS 8.2.2004 by installing some missing packages.

On CentOS 8:
yum -y install compat-openssl10

On CentOS 7:
rpm -Uvh https://download-ib01.fedoraproject.org ... x86_64.rpm

I hope this answer will be of some help for others battling with the same issue!

This post can now be archived.

[benjaminsmith]

Hi @Koja,

Thanks for following up with your solution!

We'll lock this one up.

Benjamin
Nagios Support Team