Different result on check_disk when use nrpe

[DavidPose]

Hi.

First sorry for my english. I am spanish.

A have updated Nagios Core, Nagios Plugins and NRPE to the last versions compiled from source. I have a problem with the comand check_disk when I use it with nrpe.

In the client machine with "df -h":

Code: Select all

S.ficheros     Tamaño Usados  Disp Uso% Montado en
/dev/dm-0         19G   1,4G   17G   8% /

If I use check_disk directly:

Code: Select all

./check_disk -w 20% -c 10% -p /dev/dm-0
DISK OK - free space: / 16577 MB (92% inode=95%);| /=1366MB;15141;17034;0;18927

In my nrpe.cfg i have a harcoded comand like this:

Code: Select all

command[check_dm-0]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/dm-0

If I use it through nrpe:

Code: Select all

./check_nrpe -H 127.0.0.1 -c check_dm-0
DISK OK - free space: /var/tmp 16577 MB (92% inode=95%);| /var/tmp=1366MB;15141;17034;0;18927

The result seems identical, but the route generated is diferent through nrpe. The corect route is / and nrpe show /var/tmp.

This is exactly the same command that i was using with nrpe-server and nagios plugins in debian repositories. What is wrong?

Thanks

[tgriep]

Can you run this on the remote server and post the output?

Code: Select all

/usr/local/nagios/libexec/check_disk --version

Also can you post your full nrpe.cfg file and the common.cfg file from that server so we can view it?

[DavidPose]

Second attempt to respond....

All the commands posted in my question were runned in the client (monitored) machine.

Code: Select all

/usr/local/nagios/libexec/check_disk --version
check_disk v2.1.3 (nagios-plugins 2.1.3)

For more info:

Code: Select all

/usr/local/nagios/libexec/check_nrpe
Incorrect command line arguments supplied

NRPE Plugin for Nagios
Copyright (c) 1999-2008 Ethan Galstad (nagios@nagios.org)
Version: 3.0.1
Last Modified: 09-08-2016
License: GPL v2 with exemptions (-l for more info)
SSL/TLS Available: OpenSSL 0.9.6 or higher required

nrpe.cfg (cleanned commented lines):

Code: Select all

log_facility=daemon
debug=0
pid_file=/usr/local/nagios/var/nrpe.pid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127.0.0.1,192.168.0.81,192.168.0.82,192.168.0.55
dont_blame_nrpe=1
allow_bash_command_substitution=0
command_timeout=60
connection_timeout=300

command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_load]=/usr/local/nagios/libexec/check_load -w 15,10,5 -c 30,25,20
command[check_dm-0]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/dm-0
command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/local/nagios/libexec/check_procs -w 150 -c 200

I obtain the same result if I run check_nrep from the nagios core machine or ir I run the command from the client (monitored) machine.

[lmiltchev]

I obtain the same result if I run check_nrep from the nagios core machine or ir I run the command from the client (monitored) machine.

So, you are getting the same results when you are running the check locally:

Code: Select all

./check_disk -w 20% -c 10% -p /dev/dm-0

or via check_nrpe on the Nagios Core server:

Code: Select all

./check_nrpe -H <client ip> -c check_dm-0

The only issue is when you are running check_nrpe locally on the client, correct?

Code: Select all

./check_nrpe -H 127.0.0.1 -c check_dm-0

Are you running nrpe under xinetd or as a "standalone" daemon? Have you tried restarting nrpe/xinetd on the client? Do you get the same output as you run the check as nagios user?

Code: Select all

su nagios
./check_nrpe -H 127.0.0.1 -c check_dm-0

[tgriep]

Try using the mount name in the command definition instead in the device name and see if that works the way you want.
Edit the nrpe.cfg file and change this line from

Code: Select all

command[check_dm-0]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/dm-0

to

Code: Select all

command[check_dm-0]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /

Save the file and restart nrpe and see in that is what you are looking for.

[DavidPose]

Hi.

So, you are getting the same results when you are running the check locally:

Code: Select all

./check_disk -w 20% -c 10% -p /dev/dm-0

or via check_nrpe on the Nagios Core server:

Code: Select all

./check_nrpe -H <client ip> -c check_dm-0

The only issue is when you are running check_nrpe locally on the client, correct?

Code: Select all

./check_nrpe -H 127.0.0.1 -c check_dm-0

Are you running nrpe under xinetd or as a "standalone" daemon? Have you tried restarting nrpe/xinetd on the client? Do you get the same output as you run the check as nagios user?

Code: Select all

su nagios
./check_nrpe -H 127.0.0.1 -c check_dm-0

I am running nrpe as a standalone daemon.

Localy in the monitored machine (correct result):

Code: Select all

./check_disk -w 20% -c 10% -p /dev/dm-0
DISK OK - free space: / 16579 MB (92% inode=95%);| /=1364MB;15141;17034;0;18927

Localy in the monitored machine through nrpe (wrong result):

Code: Select all

./check_nrpe -H 127.0.0.1 -c check_dm-0
DISK OK - free space: /var/tmp 16579 MB (92% inode=95%);| /var/tmp=1364MB;15141;17034;0;18927

Remotely from nagios core server through nrpe (same wrong result):

Code: Select all

./check_nrpe -H 192.168.0.40 -c check_dm-0
DISK OK - free space: /var/tmp 16579 MB (92% inode=95%);| /var/tmp=1364MB;15141;17034;0;18927

The hardcoded command in the nrpe.cfg of the remote (monitored) machine:

Code: Select all

command[check_dm-0]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/dm-0

Try using the mount name in the command definition instead in the device name and see if that works the way you want.
Edit the nrpe.cfg file and change this line from

Code: Select all

command[check_dm-0]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/dm-0

to

Code: Select all

command[check_dm-0]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /

Save the file and restart nrpe and see in that is what you are looking for.

It seems that works right.

I have created a new hardcoded command for testing it:

Code: Select all

command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /

And the results:

Code: Select all

./check_disk -w 20% -c 10% -p /
DISK OK - free space: / 16579 MB (92% inode=95%);| /=1364MB;15141;17034;0;18927

Code: Select all

./check_nrpe -H 127.0.0.1 -c check_hda1
DISK OK - free space: / 16579 MB (92% inode=95%);| /=1364MB;15141;17034;0;18927

It´s a piti. The old one is the command that I am using in the production environment. I will have to change it in all the machines.
I can not understand why with the new version of nrpe the old command don´t work correctly. Maybe a bug?

[tgriep]

Yes, it does look like it is a bug in NRPE. You may want to post your findings here and see if the developer has a fix.
https://github.com/NagiosEnterprises/nrpe/issues

[santosh048]

Hi David,

I had the same issue and was able to replicate it.

The issue seems to be in the nrpe daemon . When I use the nrpe daemon to control the nrpe.cfg, the behaviour is the one as reported by you.

But when I use xinetd.d to control the nrpe.cfg file, the output is ok.

Below is the output :

$ sudo systemctl status nrpe.service
● nrpe.service - Nagios Remote Plugin Executor
Loaded: loaded (/usr/lib/systemd/system/nrpe.service; disabled; vendor preset: disabled)
Active: active (running) since Tue 2018-03-20 13:20:19 CET; 4s ago
Docs: http://www.nagios.org/documentation
Process: 3727 ExecStopPost=/bin/rm -f /var/spool/nagios/nrpe.pid (code=exited, status=0/SUCCESS)
Main PID: 3731 (nrpe)
CGroup: /system.slice/nrpe.service
└─3731 /usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -f

Mar 20 13:20:19 localhost systemd[1]: Started Nagios Remote Plugin Executor.
Mar 20 13:20:19 localhost systemd[1]: Starting Nagios Remote Plugin Executor...
Mar 20 13:20:19 localhost nrpe[3731]: Starting up daemon
Mar 20 13:20:19 localhost nrpe[3731]: Server listening on 0.0.0.0 port 5666.
Mar 20 13:20:19 localhost nrpe[3731]: Server listening on :: port 5666.
Mar 20 13:20:19 localhost nrpe[3731]: Listening for connections on port 5666
Mar 20 13:20:19 localhost nrpe[3731]: Allowing connections from: 127.0.0.1,::1

netstat -tnlp |grep 5666

tcp 0 0 0.0.0.0:5666 0.0.0.0:* LISTEN 3731/nrpe
tcp6 0 0 :::5666 :::* LISTEN 3731/nrpe

$/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -c check_disk
DISK OK - free space: /var/tmp 12363 MB (64% inode=99%);| /var/tmp=6882MB;19166;19156;0;19246


##############################################################################

Now stopping nrpe daemon and configuring nrpe to be served by Xinetd:
##############################################################################

$ sudo systemctl status nrpe.service
● nrpe.service - Nagios Remote Plugin Executor
Loaded: loaded (/usr/lib/systemd/system/nrpe.service; disabled; vendor preset: disabled)
Active: inactive (dead)
Docs: http://www.nagios.org/documentation

Mar 20 13:20:19 localhost systemd[1]: Started Nagios Remote Plugin Executor.
Mar 20 13:20:19 localhost systemd[1]: Starting Nagios Remote Plugin Executor...
Mar 20 13:20:19 localhost nrpe[3731]: Starting up daemon
Mar 20 13:20:19 localhost nrpe[3731]: Server listening on 0.0.0.0 port 5666.
Mar 20 13:20:19 localhost nrpe[3731]: Server listening on :: port 5666.
Mar 20 13:20:19 localhost nrpe[3731]: Listening for connections on port 5666
Mar 20 13:20:19 localhost nrpe[3731]: Allowing connections from: 127.0.0.1,::1
Mar 20 13:24:12 localhost nrpe[3731]: Caught SIGTERM - shutting down...
Mar 20 13:24:12 localhost systemd[1]: Stopping Nagios Remote Plugin Executor...
Mar 20 13:24:12 localhost systemd[1]: Stopped Nagios Remote Plugin Executor.

$ sudo systemctl status xinetd.service
● xinetd.service - Xinetd A Powerful Replacement For Inetd
Loaded: loaded (/usr/lib/systemd/system/xinetd.service; disabled; vendor preset: enabled)
Active: active (running) since Tue 2018-03-20 13:25:03 CET; 29s ago
Process: 3843 ExecStart=/usr/sbin/xinetd -stayalive -pidfile /var/run/xinetd.pid $EXTRAOPTIONS (code=exited, status=0/SUCCESS)
Main PID: 3844 (xinetd)
CGroup: /system.slice/xinetd.service
└─3844 /usr/sbin/xinetd -stayalive -pidfile /var/run/xinetd.pid

Mar 20 13:25:03 localhost xinetd[3844]: removing daytime
Mar 20 13:25:03 localhost xinetd[3844]: removing discard
Mar 20 13:25:03 localhost xinetd[3844]: removing discard
Mar 20 13:25:03 localhost xinetd[3844]: removing echo
Mar 20 13:25:03 localhost xinetd[3844]: removing echo
Mar 20 13:25:03 localhost xinetd[3844]: removing tcpmux
Mar 20 13:25:03 localhost xinetd[3844]: removing time
Mar 20 13:25:03 localhost xinetd[3844]: removing time
Mar 20 13:25:03 localhost xinetd[3844]: xinetd Version 2.3.15 started with libwrap loadavg labeled-networking options compiled in.
Mar 20 13:25:03 localhost xinetd[3844]: Started working: 2 available services

$ sudo netstat -tnlp |grep 5666
tcp6 0 0 :::5666 :::* LISTEN 3844/xinetd

$ /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -c check_disk
DISK OK - free space: / 12363 MB (64% inode=99%);| /=6882MB;19166;19156;0;19246

[tgriep]

@santosh048 , what are the versions of the NRPE Agent and the check_disk plugin that you used for your testing?