Page 1 of 1

Check NRPE does not accept arguments after restart

Posted: Sun Jun 18, 2017 11:13 am
by partnernet
Dear all,

I am facing an issue with NRPE and I would like your help.
I have two nagios core servers installed. The primary nagios is on Centos 6.5 and the backup is on Centos 7.2.
The servers were installed almost 2 years ago and everything worked fine. I had setup NRPE checks from one nagios server to the other in order to make sure that everything runs smoothly.

Yesterday I performed a restart on the primary nagios server and after this, it doesn't accept arguments in the NRPE checks.
The same happens when I run the commands from the remote nagios as well as the machine itself.
e.g.

Code: Select all

# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1
NRPE v2.15
BUT

Code: Select all

# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -p 5666 -t 30 -c check_disk -a 5% 10% /dev/mapper/vg_cloudnagios-lv_root
CHECK_NRPE: Received 0 bytes from daemon.  Check the remote server logs for error messages.
In var/log/messages I have the following:

Code: Select all

 nagios: wproc: Core Worker 1648: job 32216 (pid=21930): Dormant child reaped
 xinetd[21894]: START: nrpe pid=22042 from=::ffff:127.0.0.1
 nrpe[22042]: Error: Request contained command arguments!
 nrpe[22042]: Client request was invalid, bailing out...
 xinetd[21894]: EXIT: nrpe status=0 pid=22042 duration=0(sec)
I have checked that dont_blame_nrpe is set to 1. It is like it is being ignored.
As NRPE is running under xinetd, I have restarted xinetd service.

Any help would be greatly appreciated.
Thanks,
Anna

Re: Check NRPE does not accept arguments after restart

Posted: Mon Jun 19, 2017 12:45 pm
by scottwilkerson
can you attach your whole nrpe.cfg and any included files.

Thanks

Re: Check NRPE does not accept arguments after restart

Posted: Tue Jun 20, 2017 6:47 am
by partnernet
Hi,
I have uploaded the file.

Please let me know if you find anything.
Thank you in advance.
Anna

Re: Check NRPE does not accept arguments after restart

Posted: Tue Jun 20, 2017 3:21 pm
by tgriep
The nrpe.cfg file seems to be correct.
For some tests, can you run the following on the Nagios server and post the output?

Code: Select all

/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -p 5666 -t 30 -c check_disk -a '5% 10% /dev/mapper/vg_cloudnagios-lv_root'
/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -p 5666 -t 30 -c check_disk -a '-w 5% -c 10% -p /dev/mapper/vg_cloudnagios-lv_root'
/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -p 5666 -t 30 -c check_disk
Then run the following commands as root and post the output as well.

Code: Select all

ps -ef |grep nrep
netstat -anp |grep 5666
cat /etc/xinetd.d/nrpe

Re: Check NRPE does not accept arguments after restart

Posted: Fri Jun 23, 2017 9:13 am
by partnernet
Hi,

Please find the output below:

Code: Select all

# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -p 5666 -t 30 -c check_disk -a '5% 10% /dev/mapper/vg_cloudnagios-lv_root'
CHECK_NRPE: Received 0 bytes from daemon.  Check the remote server logs for error messages.
You have new mail in /var/spool/mail/root

Code: Select all

# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -p 5666 -t 30 -c check_disk -a '-w 5% -c 10% -p /dev/mapper/vg_cloudnagios-lv_root'
CHECK_NRPE: Received 0 bytes from daemon.  Check the remote server logs for error messages.

Code: Select all

# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -p 5666 -t 30 -c check_disk
Unknown argument
Usage:
 check_disk -w limit -c limit [-W limit] [-K limit] {-p path | -x device}
[-C] [-E] [-e] [-f] [-g group ] [-k] [-l] [-M] [-m] [-R path ] [-r path ]
[-t timeout] [-u unit] [-v] [-X type] [-N type] [-n]

Code: Select all

# ps -ef |grep nrep
root     49116 47589  0 17:03 pts/0    00:00:00 grep nrep

Code: Select all

# netstat -anp |grep 5666
tcp        0      0 10.2.10.116:57634           172.16.10.253:5666          TIME_WAIT   -
tcp        0      0 127.0.0.1:43034             127.0.0.1:5666              TIME_WAIT   -
tcp        0      0 127.0.0.1:43027             127.0.0.1:5666              TIME_WAIT   -
tcp        0      0 10.2.10.116:55390           10.2.1.28:5666              TIME_WAIT   -
tcp        0      0 127.0.0.1:43019             127.0.0.1:5666              TIME_WAIT   -
tcp        0      0 :::5666                     :::*                        LISTEN      24019/xinetd

Code: Select all

# cat /etc/xinetd.d/nrpe
# default: on
# description: NRPE (Nagios Remote Plugin Executor)
service nrpe
{
        flags           = REUSE
        socket_type     = stream
        port            = 5666
        wait            = no
        user            = nagios
        group           = nagios
        server          = /usr/local/nagios/bin/nrpe
        server_args     = -c /usr/local/nagios/etc/nrpe.cfg --inetd
        log_on_failure  += USERID
        disable         = no
        only_from       = 127.0.0.1 <backup_nagios_IP>
}

Thank you very much for your help.
Anna

Re: Check NRPE does not accept arguments after restart

Posted: Fri Jun 23, 2017 10:55 am
by tgriep
There is a command line argument for when the NRPE Agent is compiled that enables the Agent to accept arguments. It it is not compiled that way, it will never accept arguments.
I don't know how the reboot did that but maybe when the agent was initially installed, it was enabled and loaded in to memory but the one on the hard drive was reverted back to a copy that is not enabled.
My suggestion is to remove the NRPE agent and them reinstall the new one by following this KB article.
https://support.nagios.com/kb/article/n ... ource.html

Re: Check NRPE does not accept arguments after restart

Posted: Wed Sep 06, 2017 10:40 am
by partnernet
Hello again,

I managed to uninstall and reinstall the nrpe service and now it received arguments without any issue.
It remains a mystery to me as of why this happened and also what will happen when the server is rebooted again.
I will try it as soon as I have the chance.

Thank you all for your suggestions.
Anna

Re: Check NRPE does not accept arguments after restart

Posted: Wed Sep 06, 2017 2:29 pm
by tgriep
Your welcome. Let us know if you still need any further help on this.