Page 2 of 3

Re: errors on cron and ssh checks after centos update

Posted: Mon Dec 10, 2018 10:05 am
by amunter
Hi,

A few posts up I slipped in that I had downloaded the linux-nrpe-client tarball and re-ran the fullinstall script to do a new install, so that's already done, but the errors persist.

Re: errors on cron and ssh checks after centos update

Posted: Mon Dec 10, 2018 4:42 pm
by npolovenko
Please run these commands from the remote nrpe server and show me the output:
/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -t 30 -c check_init_service -a 'crond'
/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -t 30 -c check_init_service -a 'sshd'
Also, on the npre server make sure that SELinux is not enabled.
sestatus
getenforce

https://support.nagios.com/kb/article.php?id=25

Re: errors on cron and ssh checks after centos update

Posted: Mon Dec 10, 2018 4:48 pm
by amunter
Well, it's enabled, but permissive. It shouldn't be blocking anything.

Re: errors on cron and ssh checks after centos update

Posted: Mon Dec 10, 2018 4:54 pm
by npolovenko
@amunter, Please disable it temporarily and then rerun the nrpe commands:
setenforce 0

Re: errors on cron and ssh checks after centos update

Posted: Mon Dec 10, 2018 5:13 pm
by amunter
OK. Here's what I get on the NRPE machine running those commands. Connection reset by peer.
# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -t 30 -c check_init_service -a 'sshd'
CHECK_NRPE: Error - Could not connect to 127.0.0.1: Connection reset by peer
# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -t 30 -c check_init_service -a 'crond'
CHECK_NRPE: Error - Could not connect to 127.0.0.1: Connection reset by peer
Same result after turning off firewalld via systemctl.

Here is firewall policy:
# firewall-cmd --list-all
drop (active)
target: DROP
icmp-block-inversion: no
interfaces: ens160
sources:
services: dhcpv6-client http ssh https
ports: 5666/tcp
protocols:
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:

Re: errors on cron and ssh checks after centos update

Posted: Mon Dec 10, 2018 5:25 pm
by npolovenko
@amunter, Can you make sure that you have 127.0.0.1 under allowed hosts in the xinetd.d/nrpe script? If you don't, add it to the list. IP addresses are separated by space. Then restart the xinetd service.

Re: errors on cron and ssh checks after centos update

Posted: Tue Dec 11, 2018 10:03 am
by amunter
Here is the contents of the file. Localhost was already allowed to connect through xinetd.

Code: Select all

# more /etc/xinetd.d/nrpe
# default: off
# description: NRPE (Nagios Remote Plugin Executor)
service nrpe
{
    disable         = no
    per_source      = 25
    socket_type     = stream
    port            = 5666
    wait            = no
    user            = nagios
    group           = nagios
    server          = /usr/local/nagios/bin/nrpe
    server_args     = -c /usr/local/nagios/etc/nrpe.cfg --inetd
    only_from       = 127.0.0.1 192.168.1.100
    log_on_success  =
}


Re: errors on cron and ssh checks after centos update

Posted: Tue Dec 11, 2018 10:33 am
by amunter
More info!

I finally found this in /var/log/messages. Something about SSL errors.

Code: Select all

Dec 11 10:30:39 nrpeserver xinetd[19492]: libwrap refused connection to nrpe (libwrap=nrpe) from ::ffff:127.0.0.1
Dec 11 10:30:39 nrpeserver xinetd[19492]: FAIL: nrpe libwrap from=::ffff:127.0.0.1
Dec 11 10:30:39 nrpeserver check_nrpe: Error: (nerrs = 0)(!log_opts) Could not complete SSL handshake with 127.0.0.1: rc=-1 SSL-error=5
Dec 11 10:30:39 nrpeserver check_nrpe: Error: (nerrs = 0)(!log_opts) Could not complete SSL handshake with 127.0.0.1: rc=-1 SSL-error=5

In the yum update that caused the issues I got the following updates:

Code: Select all

# yum history info 88 | grep ssl
    Updated     openssl-1:1.0.2k-12.el7.x86_64                        @base
    Updated     openssl-devel-1:1.0.2k-12.el7.x86_64                  @base
    Updated     openssl-libs-1:1.0.2k-12.el7.x86_64                   @base

Re: errors on cron and ssh checks after centos update

Posted: Tue Dec 11, 2018 11:02 am
by amunter
OK. So I think that error was because TCP wrappers was blocking localhost. I added 127.0.0.1 to hosts.allow and now I see the following in /var/log/messages:

Code: Select all

Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=SYSCALL msg=audit(1544543909.412:343): arch=c000003e syscall=2 success=yes exit=5 a0=556ca84eba90 a1=2 a2=180 a3=3 items=1 ppid=5908 pid=5909 auid=4294967295 uid=1002 gid=1002 euid=0 suid=0 fsuid=0 egid=1002 sgid=1002 fsgid=1002 tty=(none) ses=4294967295 comm="sudo" exe="/usr/bin/sudo" key="logins"
Dec 11 10:58:29 nrpeserveraudispd: node=nrpeserver type=SYSCALL msg=audit(1544543909.412:343): arch=c000003e syscall=2 success=yes exit=5 a0=556ca84eba90 a1=2 a2=180 a3=3 items=1 ppid=5908 pid=5909 auid=4294967295 uid=1002 gid=1002 euid=0 suid=0 fsuid=0 egid=1002 sgid=1002 fsgid=1002 tty=(none) ses=4294967295 comm="sudo" exe="/usr/bin/sudo" key="logins"
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=CWD msg=audit(1544543909.412:343):  cwd="/"
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=CWD msg=audit(1544543909.412:343):  cwd="/"
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=PATH msg=audit(1544543909.412:343): item=0 name="/var/run/faillock/nagios" inode=33632 dev=00:13 mode=0100600 ouid=1002 ogid=1002 rdev=00:00 objtype=NORMAL cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=PATH msg=audit(1544543909.412:343): item=0 name="/var/run/faillock/nagios" inode=33632 dev=00:13 mode=0100600 ouid=1002 ogid=1002 rdev=00:00 objtype=NORMAL cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=PROCTITLE msg=audit(1544543909.412:343): proctitle=7375646F002F7573722F6C6F63616C2F6E6167696F732F6C6962657865632F636865636B5F696E69745F736572766963650073736864
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=PROCTITLE msg=audit(1544543909.412:343): proctitle=7375646F002F7573722F6C6F63616C2F6E6167696F732F6C6962657865632F636865636B5F696E69745F736572766963650073736864
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=USER_ACCT msg=audit(1544543909.414:344): pid=5909 uid=1002 auid=4294967295 ses=4294967295 msg='op=PAM:accounting grantors=? acct="nagios" exe="/usr/bin/sudo" hostname=? addr=? terminal=? res=failed'
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=USER_ACCT msg=audit(1544543909.414:344): pid=5909 uid=1002 auid=4294967295 ses=4294967295 msg='op=PAM:accounting grantors=? acct="nagios" exe="/usr/bin/sudo" hostname=? addr=? terminal=? res=failed'
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=USER_CMD msg=audit(1544543909.415:345): pid=5909 uid=1002 auid=4294967295 ses=4294967295 msg='cwd="/" cmd=2F7573722F6C6F63616C2F6E6167696F732F6C6962657865632F636865636B5F696E69745F736572766963652073736864 terminal=? res=success'
Dec 11 10:58:29 nrpeserver audispd: node=nrpeserver type=USER_CMD msg=audit(1544543909.415:345): pid=5909 uid=1002 auid=4294967295 ses=4294967295 msg='cwd="/" cmd=2F7573722F6C6F63616C2F6E6167696F732F6C6962657865632F636865636B5F696E69745F736572766963652073736864 terminal=? res=success'


Re: errors on cron and ssh checks after centos update

Posted: Tue Dec 11, 2018 11:47 am
by npolovenko
@amunter, Still the same output from these checks after you added 127.0.0.1 to hosts.allow?
# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -t 30 -c check_init_service -a 'sshd'
CHECK_NRPE: Error - Could not connect to 127.0.0.1: Connection reset by peer
# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -t 30 -c check_init_service -a 'crond'
CHECK_NRPE: Error - Could not connect to 127.0.0.1: Connection reset by peer