Return Code: 3, Output: NRPE: Unable to read output

[jenglish]

Thanks.

Not SELinux:

Code: Select all

(TEST-BOYERS-3|jenglish@coralsea ~)$ getenforce
Permissive
(TEST-BOYERS-3|jenglish@coralsea ~)$ /usr/lib64/nagios/plugins/check_nrpe -H localhost -c check_multipath
NRPE: Unable to read output

Sudoers File:

Code: Select all

(TEST-BOYERS-3|jenglish@coralsea ~)$ sudo grep nrpe /etc/sudoers
nrpe ALL=(ALL) NOPASSWD: /usr/sbin/hpacucli, /bin/ping, /usr/sbin/dmidecode, /usr/bin/openssl, /bin/egrep, /sbin/ethtool -S *, /sbin/service * restart, /usr/lib/nagios/plugins/check_fts_t1.sh, /usr/lib/nagios/plugins/check_zero_length_files.sh, /usr/lib/nagios/plugins/check_file_age, /usr/bin/radwho, /usr/bin/radlast, /usr/sbin/postqueue, /bin/cat, /bin/ls, /usr/bin/find, /usr/lib/nagios/plugins/check_dir, /sbin/multipath -ll, /sbin/service iptables status, /usr/sbin/hpacucli, /bin/ping, /usr/sbin/dmidecode, /usr/bin/openssl, /bin/egrep, /usr/sbin/ethtool -S, /usr/openv/volmgr/bin/vmquery, /usr/openv/volmgr/bin/tpconfig, /var/ossec/bin/list_agents, /usr/openv/netbackup/bin/admincmd/bpplclients, /usr/openv/volmgr/bin/vmoprcmd, /usr/lib/nagios/plugins/check_multipath.sh *, /usr/lib64/nagios/plugins/, /usr/lib/nagios/plugins/

Getting the same results.

I ran strace with it this time. I'm not a developer, but not sure if this is expected:

Code: Select all

shutdown(3, SHUT_WR)                    = -1 ENOTCONN (Transport endpoint is not connected)
select(4, [3], NULL, NULL, {1, 0})      = 1 (in [3], left {0, 999998})
recvfrom(3, "\25\3\3\0\32\270\332\377F\240\233$\0254\364<\27\301\342lG#\356\221\270_6\236dtG", 1000, 0, NULL, NULL) = 31
select(4, [3], NULL, NULL, {1, 0})      = 1 (in [3], left {0, 999999})
recvfrom(3, "", 1000, 0, NULL, NULL)    = 0
close(3)                                = 0
fstat(1, {st_mode=S_IFREG|0640, st_size=41883, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f83ae7d1000
write(1, "NRPE: Unable to read output\n", 28NRPE: Unable to read output

I wonder if that strace line goes with this:

Code: Select all

Oct 29 14:42:57 coralsea.eqip.opm nrpe[37665]: socket: Address family not supported by protocol

[npolovenko]

@jenglish, Could you run these 4 commands and show me the output?

/usr/lib64/nagios/plugins/check_nrpe -4 -H localhost -c check_multipath
/usr/lib64/nagios/plugins/check_nrpe -n -H localhost -c check_multipath
/usr/lib64/nagios/plugins/check_nrpe -H localhost
/usr/lib64/nagios/plugins/check_multipath.sh

[jenglish]

Sure.

Code: Select all

(TEST-BOYERS-3|jenglish@coralsea ~)$ /usr/lib64/nagios/plugins/check_nrpe -4 -H localhost -c check_multipath
NRPE: Unable to read output
(TEST-BOYERS-3|jenglish@coralsea ~)$ /usr/lib64/nagios/plugins/check_nrpe -n -H localhost -c check_multipath
CHECK_NRPE: Receive header underflow - only -1 bytes received (4 expected).
(TEST-BOYERS-3|jenglish@coralsea ~)$ /usr/lib64/nagios/plugins/check_nrpe -H localhost
NRPE v3.2.1
(TEST-BOYERS-3|jenglish@coralsea ~)$ /usr/lib64/nagios/plugins/check_multipath.sh
(TEST-BOYERS-3|jenglish@coralsea ~)$

[npolovenko]

@jenglish, Looks like it's not the NRPE issue but rather the script is not returning any output on this server.

(TEST-BOYERS-3|jenglish@coralsea ~)$ /usr/lib64/nagios/plugins/check_multipath.sh
(TEST-BOYERS-3|jenglish@coralsea ~)$

Please make sure that all of the following files are present on the server:

MULTIPATH=/sbin/multipath
SUDO=/usr/bin/sudo
LSMOD=/sbin/lsmod

ls -l /sbin/multipath
ls -l /usr/bin/sudo
ls -l /sbin/lsmod

[jenglish]

All 3 are present:

Code: Select all

TEST-BOYERS-3|jenglish@coralsea ~)$ which multipath ; which sudo ; which lsmod
/usr/sbin/multipath
/usr/bin/sudo
/usr/sbin/lsmod

[npolovenko]

@jenglish, Please put the attached script in the /usr/lib64/nagios/plugins/ folder and run chmod +x:

chmod +x check_multipath_1.sh

And then run it:

./check_multipath_1.sh

Also, please send me the permissions:

ls -l /sbin/multipath
ls -l /usr/bin/sudo
ls -l /sbin/lsmod

[jenglish]

Thanks. Here is the output:

Code: Select all

(TEST-BOYERS-3|jenglish@coralsea plugins)$ ./check_multipath_1.sh
MULTIPATH: OK - No failed paths
(TEST-BOYERS-3|jenglish@coralsea plugins)$ ls -l /sbin/multipath
-rwxr-xr-x. 1 root root 23632 Jan 31  2018 /sbin/multipath
(TEST-BOYERS-3|jenglish@coralsea plugins)$ ls -l /usr/bin/sudo
---s--x--x. 1 root root 143248 May 28 04:03 /usr/bin/sudo
(TEST-BOYERS-3|jenglish@coralsea plugins)$ ls -l /sbin/lsmod
lrwxrwxrwx. 1 root root 11 Sep 25 10:01 /sbin/lsmod -> ../bin/kmod

I made a backup of our current check_multipath script and put the one you supplied in place. Looks like it worked.

Code: Select all

Oct 30 09:05:11 coralsea nrpe[14398]: Host 10.0.130.70 is asking for command 'check_multipath' to be run...
Oct 30 09:05:11 coralsea nrpe[14398]: Running command: sudo /usr/lib/nagios/plugins/check_multipath.sh
Oct 30 09:05:11 coralsea nrpe[14400]: WARNING: my_system() seteuid(0): Operation not permitted
Oct 30 09:05:12 coralsea nrpe[14398]: Command completed with return code 0 and output: MULTIPATH: OK - No failed paths
Oct 30 09:05:12 coralsea nrpe[14398]: Return Code: 0, Output: MULTIPATH: OK - No failed paths

It's just odd because that script is used on other RHEL7 physical servers and works as expected:

Code: Select all

Oct 30 09:17:36 boy-opprddb02 nrpe[14007]: Host 172.27.17.133 is asking for command 'check_generic' to be run...
Oct 30 09:17:36 boy-opprddb02 nrpe[14007]: Running command: /usr/lib64/nagios/plugins/check_multipath_active.sh
Oct 30 09:17:36 boy-opprddb02 nrpe[14008]: WARNING: my_system() seteuid(0): Operation not permitted
Oct 30 09:17:37 boy-opprddb02 nrpe[14007]: Command completed with return code 0 and output: OK
Oct 30 09:17:37 boy-opprddb02 nrpe[14007]: Return Code: 0, Output: OK

[npolovenko]

@jenglish, Looks like it worked. The return code of 0 corresponds to OK state in Nagios. Can you confirm that it works through NRPE now? Are we good to close this as resolved?

[jenglish]

@npolovenko

Yes. We can close. Thank you very much for your patience in tackling this problem. Sadly, it seems our own script needs tweaking for this to work. Your support is much appreciated.

[npolovenko]

@jenglish, You're welcome! Thanks for using the support forum.