Page 1 of 1

check_nrpe remote "uses"/checks wrong disk

Posted: Tue Feb 19, 2019 9:08 am
by greinick
Hi,

I have tried to find a solution/answer to my problem, but had no luck; may be someone on the forum can help.

I have a group of identical servers, same hardware, disks, memory same nagios/nrpe config and version. On all but one I can check the disks usage with check_nrpe.

On the problem-server I can local execute check_nrpe and do get the right values regarding usage/free. Doing the check remotely from the nagios server gets different disk and therefore wrong values.

e.g.
local:
/usr/lib64/nagios/plugins/check_disk -w 1000 -c 500 -p /dev/mapper/centos-root -m
DISK OK - free space: / 154398 MB (97,84% inode=100%);| /=3400MB;156798;157298;0;157798

remote:a
/usr/local/nagios/libexec/check_nrpe -H node-01 -c check_disk1
DISK CRITICAL - free space: /var/lib/ceph/osd/ceph-01 0 GB (0.00% inode=-9900%);| /var/lib/ceph/osd/ceph-01=3149679245GB;3150858731;3150858751;0;3150858761

Any ideas? Regards

Re: check_nrpe remote "uses"/checks wrong disk

Posted: Tue Feb 19, 2019 4:32 pm
by scottwilkerson
In your "remote" example it is going to run what is defined for the check_disk1 check in your nrpe.cfg on the remote server node-01

What is that defined as?

Re: check_nrpe remote "uses"/checks wrong disk

Posted: Wed Feb 20, 2019 3:54 am
by greinick
on the node-01 I have:

grep check_disk1 /etc/nagios/nrpe.cfg

command[check_disk1]=/usr/lib64/nagios/plugins/check_disk -w 1000 -c 500 -p /dev/mapper/centos-root -m

running: /usr/lib64/nagios/plugins/check_disk -w 1000 -c 500 -p /dev/mapper/centos-root -m

shows

DISK OK - free space: / 154400 MB (97,84% inode=100%);| /=3397MB;156798;157298;0;157798

Re: check_nrpe remote "uses"/checks wrong disk

Posted: Wed Feb 20, 2019 7:52 am
by scottwilkerson
Can you verify that /etc/nagios/nrpe.cfg is the config node01 is using

Code: Select all

ps -ef|grep nrpe
If so, is it possible that /etc/nagios/nrpe.cfg is including another config file that could be overriding this setting?

is it possible that on the Nagios server it is sending node-01 to the wrong IP?

from the Nagios server

Code: Select all

ping node-01

Re: check_nrpe remote "uses"/checks wrong disk

Posted: Wed Feb 20, 2019 8:49 am
by greinick
ps -ef|grep nrpe
nrpe 2338735 1 0 Feb19 ? 00:00:00 /usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f


ping node-01
PING node-01 (172.17.10.15) 56(84) bytes of data.
64 bytes from node-01 (172.17.10.15): icmp_seq=1 ttl=64 time=0.447 ms

Re: check_nrpe remote "uses"/checks wrong disk

Posted: Wed Feb 20, 2019 9:20 am
by scottwilkerson
Lets run the following on 172.17.10.15

Code: Select all

grep include_dir /etc/nagios/nrpe.cfg
grep check_disk1 /etc/nagios/nrpe.cfg 
ip add

Re: check_nrpe remote "uses"/checks wrong disk

Posted: Wed Feb 20, 2019 9:26 am
by greinick

Code: Select all

# grep include_dir /etc/nagios/nrpe.cfg
include_dir=/etc/nrpe.d/

Code: Select all

# grep check_disk1 /etc/nagios/nrpe.cfg 
command[check_disk1]=/usr/lib64/nagios/plugins/check_disk -w 1000 -c 500 -p /dev/mapper/centos-root -m

Code: Select all

# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: enp4s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master team0 state UP group default qlen 1000
    link/ether 0c:c4:7a:de:92:74 brd ff:ff:ff:ff:ff:ff
3: enp4s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master team0 state UP group default qlen 1000
    link/ether 0c:c4:7a:de:92:74 brd ff:ff:ff:ff:ff:ff
4: ens4f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master team0 state UP group default qlen 1000
    link/ether 0c:c4:7a:de:92:74 brd ff:ff:ff:ff:ff:ff
5: ens4f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master team0 state UP group default qlen 1000
    link/ether 0c:c4:7a:de:92:74 brd ff:ff:ff:ff:ff:ff
7: team0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 0c:c4:7a:de:92:74 brd ff:ff:ff:ff:ff:ff
    inet 172.17.10.15/24 brd 172.17.252.255 scope global noprefixroute team0
       valid_lft forever preferred_lft forever

Re: check_nrpe remote "uses"/checks wrong disk

Posted: Wed Feb 20, 2019 10:15 am
by scottwilkerson
Ok, so you have an included dir

let's run the following

Code: Select all

grep check_disk1 -R /etc/nrpe.d/

Re: check_nrpe remote "uses"/checks wrong disk

Posted: Wed Feb 20, 2019 10:27 am
by greinick
Kotau! :D

This was the only server with a special config; there was a older config for checking other disk, with the same command name. :geek:

Case closed, problem solved. Thanks a lot for your help.

Re: check_nrpe remote "uses"/checks wrong disk

Posted: Wed Feb 20, 2019 10:29 am
by scottwilkerson
greinick wrote:Kotau! :D

This was the only server with a special config; there was a older config for checking other disk, with the same command name. :geek:

Case closed, problem solved. Thanks a lot for your help.
Great!

Locking thread