Page 1 of 2
NRPE: Command 'check_init_service' not defined
Posted: Tue Oct 27, 2015 1:14 pm
by linuser
I am able to get this to work from the remote system but not the nagios server. The error in the subject line is what I am getting. From the remote system I can run this from command line:
Code: Select all
/usr/lib64/nagios/plugins/check_init_service bgpd
and get this:
Code: Select all
Redirecting to /bin/systemctl status bgpd.service
bgpd.service - BGP routing daemon
Loaded: loaded (/usr/lib/systemd/system/bgpd.service; enabled)
Active: active (running) since Tue 2015-10-27 10:40:44 CDT; 2h 18min ago
Process: 2081 ExecStart=/usr/sbin/bgpd -u quagga -g quagga -f /etc/quagga/bgpd.conf -d -P 17500 (code=exited, status=0/SUCCESS)
Main PID: 2267 (bgpd)
CGroup: /system.slice/bgpd.service
└─2267 /usr/sbin/bgpd -u quagga -g quagga -f /etc/quagga/bgpd.conf -d -P 17500
So far so good. Here are the command arguments from the nrpe.cfg file on the remote server.
Code: Select all
command[check_init1]=/usr/lib64/nagios/plugins/check_init_service bgpd
command[check_init2]=/usr/lib64/nagios/plugins/check_init_service zebra
However when I go to set all this up on the nagios server to check it remotely it fails. From the command line I run:
Code: Select all
./check_nrpe -n -H 192.168.255.70 -t 30 -c check_init_service bgpd
I have tried this with single quotes around bgpd, double quotes, no quotes, appending -a before the argument. Nothing works and it returns:
Code: Select all
NRPE: Command 'check_init_service' not defined
Problem is- I am not sure what its wanting here. There is also nothing helpful in any logs. My nagios server config is setup as follows:
from commands.cfg:
Code: Select all
# 'check_init_service' command definition
define command{
command_name check_init_service
command_line $USER1$/check_nrpe -n -H $HOSTADDRESS$ -t 30 -c check_init_service -a $ARG1$
}
and In the host file config:
Code: Select all
define service{
use unix-service ; Name of service template to use
host_name <my_domain>
service_description Quagga-bgpd
check_command check_nrpe-2.24!check_init1
notification_period workhours
notifications_enabled 1
}
define service{
use unix-service ; Name of service template to use
host_name <my_domain>
service_description Quagga-zebra
check_command check_nrpe-2.24!check_init2
notification_period workhours
notifications_enabled 1
}
I think if I can get this to work from command line with nrpe It may work in nagios. But I am completely stuck at this point and not sure what to do from here. Any help is greatly appreciated.
Re: NRPE: Command 'check_init_service' not defined
Posted: Tue Oct 27, 2015 2:47 pm
by hsmith
Hate to ask the obvious, but did you restart xinetd after editing the configuration on the remote server?
Re: NRPE: Command 'check_init_service' not defined
Posted: Tue Oct 27, 2015 2:54 pm
by linuser
hsmith wrote:Hate to ask the obvious, but did you restart xinetd after editing the configuration on the remote server?
No, I'm not using the config that way. Nrpe runs as a standalone daemon, and nrpe listens on 5666, not xinetd.
Thanks!
Re: NRPE: Command 'check_init_service' not defined
Posted: Tue Oct 27, 2015 3:03 pm
by hsmith
All right, wasn't sure which way you had it. At any rate, was it restarted so the new config can take effect?
Re: NRPE: Command 'check_init_service' not defined
Posted: Tue Oct 27, 2015 3:25 pm
by linuser
hsmith wrote:All right, wasn't sure which way you had it. At any rate, was it restarted so the new config can take effect?
Yes, many times.
Re: NRPE: Command 'check_init_service' not defined
Posted: Tue Oct 27, 2015 4:29 pm
by rkennedy
On your nrpe.cfg on the remote server, you have defined the commands as -
Code: Select all
command[check_init1]=/usr/lib64/nagios/plugins/check_init_service bgpd
command[check_init2]=/usr/lib64/nagios/plugins/check_init_service zebra
When you reference these from your command on the Nagios server you need to use those as your -c variable.
Example -
Code: Select all
./check_nrpe -n -H 192.168.255.70 -t 30 -c check_init1
Can you let me know if that works for you or not?
Re: NRPE: Command 'check_init_service' not defined
Posted: Tue Oct 27, 2015 4:55 pm
by linuser
rkennedy wrote:On your nrpe.cfg on the remote server, you have defined the commands as -
Code: Select all
command[check_init1]=/usr/lib64/nagios/plugins/check_init_service bgpd
command[check_init2]=/usr/lib64/nagios/plugins/check_init_service zebra
When you reference these from your command on the Nagios server you need to use those as your -c variable.
Example -
Code: Select all
./check_nrpe -n -H 192.168.255.70 -t 30 -c check_init1
Can you let me know if that works for you or not?
Yes, that better. But it still does not work. Now I get:
As a test I ran another command for NTP which uses "check_time1" as its listed in the cfg file. That worked and returned output.
On the server in /var/log/nagios/nagios.log I see this:
Code: Select all
[1445980219] SERVICE NOTIFICATION: nagios;<hostname>;Quagga-zebra;CRITICAL;notify-service-by-email;NRPE: Unable to read output
And in /var/log/audit/audit.log this pops in every time I run the command:
Code: Select all
type=AVC msg=audit(1445982801.776:1442): avc: denied { search } for pid=6120 comm="mountpoint" name="/" dev="tmpfs" ino=11272 scontext=system_u:system_r:nrpe_t:s0 tcontext=system_u:object_r:cgroup_t:s0 tclass=dir
type=SYSCALL msg=audit(1445982801.776:1442): arch=c000003e syscall=4 success=no exit=-13 a0=7fffee895f12 a1=7fffee893bb0 a2=7fffee893bb0 a3=7fffee893920 items=0 ppid=6118 pid=6120 auid=4294967295 uid=997 gid=995 euid=997 suid=997 fsuid=997 egid=995 sgid=995 fsgid=995 tty=(none) ses=4294967295 comm="mountpoint" exe="/usr/bin/mountpoint" subj=system_u:system_r:nrpe_t:s0 key=(null)
type=AVC msg=audit(1445982801.783:1443): avc: denied { execute } for pid=6117 comm="service" name="systemctl" dev="dm-1" ino=2101040 scontext=system_u:system_r:nrpe_t:s0 tcontext=system_u:object_r:systemd_systemctl_exec_t:s0 tclass=file
type=SYSCALL msg=audit(1445982801.783:1443): arch=c000003e syscall=59 success=no exit=-13 a0=1f17280 a1=1f409d0 a2=1f18a90 a3=7fffbdd79f30 items=0 ppid=6115 pid=6117 auid=4294967295 uid=997 gid=995 euid=997 suid=997 fsuid=997 egid=995 sgid=995 fsgid=995 tty=(none) ses=4294967295 comm="service" exe="/usr/bin/bash" subj=system_u:system_r:nrpe_t:s0 key=(null)
type=AVC msg=audit(1445982801.783:1444): avc: denied { getattr } for pid=6117 comm="service" path="/usr/bin/systemctl" dev="dm-1" ino=2101040 scontext=system_u:system_r:nrpe_t:s0 tcontext=system_u:object_r:systemd_systemctl_exec_t:s0 tclass=file
type=SYSCALL msg=audit(1445982801.783:1444): arch=c000003e syscall=4 success=no exit=-13 a0=1f17280 a1=7fffbdd7a100 a2=7fffbdd7a100 a3=7fffbdd79f30 items=0 ppid=6115 pid=6117 auid=4294967295 uid=997 gid=995 euid=997 suid=997 fsuid=997 egid=995 sgid=995 fsgid=995 tty=(none) ses=4294967295 comm="service" exe="/usr/bin/bash" subj=system_u:system_r:nrpe_t:s0 key=(null)
type=AVC msg=audit(1445982801.783:1445): avc: denied { getattr } for pid=6117 comm="service" path="/usr/bin/systemctl" dev="dm-1" ino=2101040 scontext=system_u:system_r:nrpe_t:s0 tcontext=system_u:object_r:systemd_systemctl_exec_t:s0 tclass=file
type=SYSCALL msg=audit(1445982801.783:1445): arch=c000003e syscall=4 success=no exit=-13 a0=1f17280 a1=7fffbdd7a0e0 a2=7fffbdd7a0e0 a3=7fffbdd79f30 items=0 ppid=6115 pid=6117 auid=4294967295 uid=997 gid=995 euid=997 suid=997 fsuid=997 egid=995 sgid=995 fsgid=995 tty=(none) ses=4294967295 comm="service" exe="/usr/bin/bash" subj=system_u:system_r:nrpe_t:s0 key=(null)
type=AVC msg=audit(1445982801.783:1446): avc: denied { getattr } for pid=6117 comm="service" path="/usr/bin/systemctl" dev="dm-1" ino=2101040 scontext=system_u:system_r:nrpe_t:s0 tcontext=system_u:object_r:systemd_systemctl_exec_t:s0 tclass=file
type=SYSCALL msg=audit(1445982801.783:1446): arch=c000003e syscall=4 success=no exit=-13 a0=1f17280 a1=7fffbdd7a1b0 a2=7fffbdd7a1b0 a3=2 items=0 ppid=6115 pid=6117 auid=4294967295 uid=997 gid=995 euid=997 suid=997 fsuid=997 egid=995 sgid=995 fsgid=995 tty=(none) ses=4294967295 comm="service" exe="/usr/bin/bash" subj=system_u:system_r:nrpe_t:s0 key=(null)
I thought this could be SELInux so I put it in permissive mode but still getting the same error.
Re: NRPE: Command 'check_init_service' not defined
Posted: Tue Oct 27, 2015 5:33 pm
by linuser
As a test I bare-boned the check_init_script to something like this:
And on the nagios server I can run the command with success.
Code: Select all
[root@labnagios ~]# /usr/lib64/nagios/plugins/check_nrpe -n -H 192.168.255.70 -c check_init1
Hi
Soooooo....seems its the output from the script that NRPE can't handle. I *think* its due to the way the older init scripts returned output. (you get a lot more with systemctl/systemd)
Re: NRPE: Command 'check_init_service' not defined
Posted: Wed Oct 28, 2015 8:50 am
by linuser
I changed the script again to a very basic one just to see how it looks in the nagios UI. I have added the file to this post. For some reason NRPE can read and process this. If I change up the script though to use systemctl with the full output NRPE can't read it. Here is what I did:
I changed this: (original)
Code: Select all
if [ $# -eq 1 ]; then
/sbin/service $1 status
ret=$?
case "$ret" in
to this: (modified)
Code: Select all
if [ $# -eq 1 ]; then
/bin/systemctl status $1 | awk 'NR==3'
ret=$?
case "$ret" in
The "awk 'NR==3' only gives me the 3rd line output, which is all I am interested in and it looks like this when you run the script from the remote host.
Code: Select all
[root@lab plugins]# /usr/lib64/nagios/plugins/check_init_service bgpd
Active: active (running) since Tue 2015-10-27 10:40:44 CDT; 21h ago
One would think that NRPE would be able to process this like the barebones script but it can't. On the nagios server I get the same error as before.
Code: Select all
[root@labnagios plugins]# ./check_nrpe -n -H 192.168.255.70 -c check_init1
NRPE: Unable to read output
Here is the contents of the original full script for context.
Code: Select all
#!/bin/sh
PROGNAME=`basename $0`
print_usage() {
echo "Usage: $PROGNAME"
}
print_help() {
echo ""
print_usage
echo ""
echo "This plugin checks the status of services normally started by the init process."
echo ""
support
exit 0
}
case "$1" in
--help)
print_help
exit 0
;;
-h)
print_help
exit 0
;;
*)
if [ $# -eq 1 ]; then
/sbin/service $1 status
ret=$?
case "$ret" in
0)
exit $ret
;;
*)
exit 2
;;
esac
else
echo "ERROR: No service name specified on command line"
exit 3
fi
;;
esac
At this point I'd like to know a few things:
1) Why NRPE can process a basic "This is a test" script but it can't process the modified one that uses systmectl with a one line output?
2) What can I do to get this working? I can only see 2 options - either modify the system to use /sbin/service for these two services, or modify the script again to allow NRPE to properly read it.
Help is greatly appreciated.
Re: NRPE: Command 'check_init_service' not defined
Posted: Wed Oct 28, 2015 9:51 am
by rkennedy
Hm, this may be related to permissions. It would explain why your simple script worked as well. Can you try adding sudo before your command?
Code: Select all
command[check_init1]=sudo /usr/lib64/nagios/plugins/check_init_service bgpd
command[check_init2]=sudo /usr/lib64/nagios/plugins/check_init_service zebra