Page 2 of 3
Re: errors on cron and ssh checks after centos update
Posted: Mon Dec 10, 2018 10:05 am
by amunter
Hi,
A few posts up I slipped in that I had downloaded the linux-nrpe-client tarball and re-ran the fullinstall script to do a new install, so that's already done, but the errors persist.
Re: errors on cron and ssh checks after centos update
Posted: Mon Dec 10, 2018 4:42 pm
by npolovenko
Please run these commands from the remote nrpe server and show me the output:
/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -t 30 -c check_init_service -a 'crond'
/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -t 30 -c check_init_service -a 'sshd'
Also, on the npre server make sure that SELinux is not enabled.
sestatus
getenforce
https://support.nagios.com/kb/article.php?id=25
Re: errors on cron and ssh checks after centos update
Posted: Mon Dec 10, 2018 4:48 pm
by amunter
Well, it's enabled, but permissive. It shouldn't be blocking anything.
Re: errors on cron and ssh checks after centos update
Posted: Mon Dec 10, 2018 4:54 pm
by npolovenko
@amunter, Please disable it temporarily and then rerun the nrpe commands:
setenforce 0
Re: errors on cron and ssh checks after centos update
Posted: Mon Dec 10, 2018 5:13 pm
by amunter
OK. Here's what I get on the NRPE machine running those commands. Connection reset by peer.
# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -t 30 -c check_init_service -a 'sshd'
CHECK_NRPE: Error - Could not connect to 127.0.0.1: Connection reset by peer
# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -t 30 -c check_init_service -a 'crond'
CHECK_NRPE: Error - Could not connect to 127.0.0.1: Connection reset by peer
Same result after turning off firewalld via systemctl.
Here is firewall policy:
# firewall-cmd --list-all
drop (active)
target: DROP
icmp-block-inversion: no
interfaces: ens160
sources:
services: dhcpv6-client http ssh https
ports: 5666/tcp
protocols:
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
Re: errors on cron and ssh checks after centos update
Posted: Mon Dec 10, 2018 5:25 pm
by npolovenko
@amunter, Can you make sure that you have 127.0.0.1 under allowed hosts in the xinetd.d/nrpe script? If you don't, add it to the list. IP addresses are separated by space. Then restart the xinetd service.
Re: errors on cron and ssh checks after centos update
Posted: Tue Dec 11, 2018 10:03 am
by amunter
Here is the contents of the file. Localhost was already allowed to connect through xinetd.
Code: Select all
# more /etc/xinetd.d/nrpe
# default: off
# description: NRPE (Nagios Remote Plugin Executor)
service nrpe
{
disable = no
per_source = 25
socket_type = stream
port = 5666
wait = no
user = nagios
group = nagios
server = /usr/local/nagios/bin/nrpe
server_args = -c /usr/local/nagios/etc/nrpe.cfg --inetd
only_from = 127.0.0.1 192.168.1.100
log_on_success =
}
Re: errors on cron and ssh checks after centos update
Posted: Tue Dec 11, 2018 10:33 am
by amunter
More info!
I finally found this in /var/log/messages. Something about SSL errors.
Code: Select all
Dec 11 10:30:39 nrpeserver xinetd[19492]: libwrap refused connection to nrpe (libwrap=nrpe) from ::ffff:127.0.0.1
Dec 11 10:30:39 nrpeserver xinetd[19492]: FAIL: nrpe libwrap from=::ffff:127.0.0.1
Dec 11 10:30:39 nrpeserver check_nrpe: Error: (nerrs = 0)(!log_opts) Could not complete SSL handshake with 127.0.0.1: rc=-1 SSL-error=5
Dec 11 10:30:39 nrpeserver check_nrpe: Error: (nerrs = 0)(!log_opts) Could not complete SSL handshake with 127.0.0.1: rc=-1 SSL-error=5
In the yum update that caused the issues I got the following updates:
Code: Select all
# yum history info 88 | grep ssl
Updated openssl-1:1.0.2k-12.el7.x86_64 @base
Updated openssl-devel-1:1.0.2k-12.el7.x86_64 @base
Updated openssl-libs-1:1.0.2k-12.el7.x86_64 @base
Re: errors on cron and ssh checks after centos update
Posted: Tue Dec 11, 2018 11:02 am
by amunter
OK. So I think that error was because TCP wrappers was blocking localhost. I added 127.0.0.1 to hosts.allow and now I see the following in /var/log/messages:
Code: Select all
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=SYSCALL msg=audit(1544543909.412:343): arch=c000003e syscall=2 success=yes exit=5 a0=556ca84eba90 a1=2 a2=180 a3=3 items=1 ppid=5908 pid=5909 auid=4294967295 uid=1002 gid=1002 euid=0 suid=0 fsuid=0 egid=1002 sgid=1002 fsgid=1002 tty=(none) ses=4294967295 comm="sudo" exe="/usr/bin/sudo" key="logins"
Dec 11 10:58:29 nrpeserveraudispd: node=nrpeserver type=SYSCALL msg=audit(1544543909.412:343): arch=c000003e syscall=2 success=yes exit=5 a0=556ca84eba90 a1=2 a2=180 a3=3 items=1 ppid=5908 pid=5909 auid=4294967295 uid=1002 gid=1002 euid=0 suid=0 fsuid=0 egid=1002 sgid=1002 fsgid=1002 tty=(none) ses=4294967295 comm="sudo" exe="/usr/bin/sudo" key="logins"
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=CWD msg=audit(1544543909.412:343): cwd="/"
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=CWD msg=audit(1544543909.412:343): cwd="/"
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=PATH msg=audit(1544543909.412:343): item=0 name="/var/run/faillock/nagios" inode=33632 dev=00:13 mode=0100600 ouid=1002 ogid=1002 rdev=00:00 objtype=NORMAL cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=PATH msg=audit(1544543909.412:343): item=0 name="/var/run/faillock/nagios" inode=33632 dev=00:13 mode=0100600 ouid=1002 ogid=1002 rdev=00:00 objtype=NORMAL cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=PROCTITLE msg=audit(1544543909.412:343): proctitle=7375646F002F7573722F6C6F63616C2F6E6167696F732F6C6962657865632F636865636B5F696E69745F736572766963650073736864
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=PROCTITLE msg=audit(1544543909.412:343): proctitle=7375646F002F7573722F6C6F63616C2F6E6167696F732F6C6962657865632F636865636B5F696E69745F736572766963650073736864
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=USER_ACCT msg=audit(1544543909.414:344): pid=5909 uid=1002 auid=4294967295 ses=4294967295 msg='op=PAM:accounting grantors=? acct="nagios" exe="/usr/bin/sudo" hostname=? addr=? terminal=? res=failed'
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=USER_ACCT msg=audit(1544543909.414:344): pid=5909 uid=1002 auid=4294967295 ses=4294967295 msg='op=PAM:accounting grantors=? acct="nagios" exe="/usr/bin/sudo" hostname=? addr=? terminal=? res=failed'
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=USER_CMD msg=audit(1544543909.415:345): pid=5909 uid=1002 auid=4294967295 ses=4294967295 msg='cwd="/" cmd=2F7573722F6C6F63616C2F6E6167696F732F6C6962657865632F636865636B5F696E69745F736572766963652073736864 terminal=? res=success'
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=USER_CMD msg=audit(1544543909.415:345): pid=5909 uid=1002 auid=4294967295 ses=4294967295 msg='cwd="/" cmd=2F7573722F6C6F63616C2F6E6167696F732F6C6962657865632F636865636B5F696E69745F736572766963652073736864 terminal=? res=success'
Re: errors on cron and ssh checks after centos update
Posted: Tue Dec 11, 2018 11:47 am
by npolovenko
@amunter, Still the same output from these checks after you added 127.0.0.1 to hosts.allow?
# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -t 30 -c check_init_service -a 'sshd'
CHECK_NRPE: Error - Could not connect to 127.0.0.1: Connection reset by peer
# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -t 30 -c check_init_service -a 'crond'
CHECK_NRPE: Error - Could not connect to 127.0.0.1: Connection reset by peer