Page 1 of 3
errors on cron and ssh checks after centos update
Posted: Tue Dec 04, 2018 3:17 pm
by amunter
I just updated a CentOS host today that I monitor with Nagios XI. Lots of yum updates were pending. It's now running CentOS Linux release 7.6.1810 (Core). It is also happening on a Redhat 7 machine that was also updated today. Same issues.
After the update I started getting Nagios warnings on checks for cron and ssh. Each hour I get a warning email "NRPE: Unable to read output". I also get an error for Jenkins CI service monitoring. It is working fine on other RHEL 7 hosts.
I hadn't updated the Nagios XI server, but I'm doing that now.
Any ideas what would have caused the issue with just those monitored services on those two and where else I can look for more error text?
Re: errors on cron and ssh checks after centos update
Posted: Tue Dec 04, 2018 4:48 pm
by npolovenko
@amunter, Can you run the following commands on the nrpe server and show me the output?
sudo /usr/local/nagios/libexec/check_init_service sshd
sudo /usr/local/nagios/libexec/check_init_service crond
Re: errors on cron and ssh checks after centos update
Posted: Thu Dec 06, 2018 10:30 am
by amunter
Code: Select all
$ sudo /usr/local/nagios/libexec/check_init_service sshd
active
$ sudo /usr/local/nagios/libexec/check_init_service crond
active
Re: errors on cron and ssh checks after centos update
Posted: Thu Dec 06, 2018 1:12 pm
by npolovenko
@amunter, Interesting. Appears that both services are actually running. Can you run these commands from the nagios server and show me the output?
/usr/local/nagios/libexec/check_nrpe -H nrpe_ip_address -t 30 -c check_init_service -a 'crond'
/usr/local/nagios/libexec/check_nrpe -H nrpe_ip_address -t 30 -c check_init_service -a 'sshd'
Re: errors on cron and ssh checks after centos update
Posted: Thu Dec 06, 2018 2:12 pm
by amunter
Code: Select all
[root@nagiosxi ~]# /usr/local/nagios/libexec/check_nrpe -H 192.168.1.102 -t 30 -c check_init_service -a 'crond'
NRPE: Unable to read output
[root@nagiosxi ~]# /usr/local/nagios/libexec/check_nrpe -H 192.168.1.102 -t 30 -c check_init_service -a 'sshd'
NRPE: Unable to read output
Other checks run ok:
Code: Select all
[root@nagiosxi ~]# /usr/local/nagios/libexec/check_nrpe -H 192.168.1.102 -t 30
NRPE v3.2.1
[root@nagiosxi ~]# /usr/local/nagios/libexec/check_nrpe -H 192.168.1.102 -t 30 -c check_procs -a '-w 150 -c 250'
PROCS OK: 121 processes | procs=121;150;250;0;
but the check_init_service ones fail remotely even though they succeed locally.
Re: errors on cron and ssh checks after centos update
Posted: Thu Dec 06, 2018 2:26 pm
by npolovenko
@amunter, Can you show me how the check_init service is defined in the nrpe.cfg file on the nrpe server?
Re: errors on cron and ssh checks after centos update
Posted: Thu Dec 06, 2018 2:49 pm
by amunter
Here it is with all of the comments and whitespace lines removed. I notice that it doesn't have my nagios server listed in allowed hosts, but the other checks still complete, and I did give it the IP address of my nagios server when running the "fullinstall" script recently on the installer. That IP is in the /etc/xinetd.d/nrpe file as an "only_from" variable.
When these warnings started I updated my NagiosXI install, then got the linux-nrpe-client tarball from it and reinstalled nrpe, specifying the nagios server's IP at the end of the install process.
Code: Select all
# cat /usr/local/nagios/etc/nrpe.cfg | grep -v "#" | uniq
log_facility=daemon
pid_file=/var/run/nrpe.pid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127.0.0.1
dont_blame_nrpe=1
debug=0
command_timeout=60
connection_timeout=300
include_dir=/usr/local/nagios/etc/nrpe
Re: errors on cron and ssh checks after centos update
Posted: Thu Dec 06, 2018 3:51 pm
by npolovenko
@amunter, If nrpe was installed under the xinetd it's normal that allowed hosts would not be in the nrpe.cfg, but in the /etc/xinetd.d/nrpe. Can you upload all the files that you have in this folder?
/usr/local/nagios/etc/nrpe
Re: errors on cron and ssh checks after centos update
Posted: Fri Dec 07, 2018 10:50 am
by amunter
Sure, here ya go. Two files are in the directory: asterisk.cfg and common.cfg.
Re: errors on cron and ssh checks after centos update
Posted: Fri Dec 07, 2018 4:49 pm
by ssax
I would recompile NRPE OR re-install the linux NRPE agent, some packages probably got updated from what NRPE binary was compiled with and is not working properly now. A recompile/re-install should likely pick them up, it depends on how you installed the agent though.
https://assets.nagios.com/downloads/nag ... _Agent.pdf