Page 2 of 3

Re: Unable to connect to host error

Posted: Tue Sep 22, 2020 9:49 am
by userman73
Yes, it was.
I am not sure about that - how would I be able to tell?

Code: Select all

#############################################################################
#
#  Sample NRPE Config File
#
#  Notes:
#
#  This is a sample configuration file for the NRPE daemon.  It needs to be
#  located on the remote host that is running the NRPE daemon, not the host
#  from which the check_nrpe client is being executed.
#
#############################################################################


# LOG FACILITY
# The syslog facility that should be used for logging purposes.

log_facility=daemon



# LOG FILE
# If a log file is specified in this option, nrpe will write to
# that file instead of using syslog.

#log_file=/var/run/nrpe.log



# DEBUGGING OPTION
# This option determines whether or not debugging messages are logged to the
# syslog facility.
# Values: 0=debugging off, 1=debugging on

debug=0



# PID FILE
# The name of the file in which the NRPE daemon should write it's process ID
# number.  The file is only written if the NRPE daemon is started by the root
# user and is running in standalone mode.

pid_file=/var/run/nrpe/nrpe.pid



# PORT NUMBER
# Port number we should wait for connections on.
# NOTE: This must be a non-privileged port (i.e. > 1024).
# NOTE: This option is ignored if NRPE is running under either inetd
# or xinetd or via systemd. [In systemd please use
# systemctl edit nrpe.service
# to set up the port.

server_port=5666



# SERVER ADDRESS
# Address that nrpe should bind to in case there are more than one interface
# and you do not want nrpe to bind on all interfaces.
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd
#       or with systemd. Please start by hand.

#server_address=127.0.0.1



# LISTEN QUEUE SIZE
# Listen queue size (backlog) for serving incoming connections.
# You may want to increase this value under high load.

#listen_queue_size=5



# NRPE USER
# This determines the effective user that the NRPE daemon should run as.
# You can either supply a username or a UID.
#
# NOTE: This option is ignored if NRPE is running under either inetd
# or xinetd or via systemd. [In systemd please use
# systemctl edit nrpe.service
# to set up the group.

nrpe_user=nrpe



# NRPE GROUP
# This determines the effective group that the NRPE daemon should run as.
# You can either supply a group name or a GID.
#
# NOTE: This option is ignored if NRPE is running under either inetd
# or xinetd or via systemd. [In systemd please use
# systemctl edit nrpe.service
# to set up the user.

nrpe_group=nrpe



# ALLOWED HOST ADDRESSES
# This is an optional comma-delimited list of IP address or hostnames
# that are allowed to talk to the NRPE daemon. Network addresses with a bit mask
# (i.e. 192.168.1.0/24) are also supported. Hostname wildcards are not currently
# supported.
#
# Note: The daemon only does rudimentary checking of the client's IP
# address.  I would highly recommend adding entries in your /etc/hosts.allow
# file to allow only the specified host to connect to the port
# you are running this daemon on.
#
# NOTE: This option is ignored if NRPE is running under either inetd
# or xinetd or systemd

allowed_hosts=127.0.0.1,::1,<ip>



# COMMAND ARGUMENT PROCESSING
# This option determines whether or not the NRPE daemon will allow clients
# to specify arguments to commands that are executed.  This option only works
# if the daemon was configured with the --enable-command-args configure script
# option.
#
# *** ENABLING THIS OPTION IS A SECURITY RISK! ***
# Read the SECURITY file for information on some of the security implications
# of enabling this variable.
#
# Values: 0=do not allow arguments, 1=allow command arguments

dont_blame_nrpe=1



# BASH COMMAND SUBSTITUTION
# This option determines whether or not the NRPE daemon will allow clients
# to specify arguments that contain bash command substitutions of the form
# $(...).  This option only works if the daemon was configured with both
# the --enable-command-args and --enable-bash-command-substitution configure
# script options.
#
# *** ENABLING THIS OPTION IS A HIGH SECURITY RISK! ***
# Read the SECURITY file for information on some of the security implications
# of enabling this variable.
#
# Values: 0=do not allow bash command substitutions,
#         1=allow bash command substitutions

allow_bash_command_substitution=0



# COMMAND PREFIX
# This option allows you to prefix all commands with a user-defined string.
# A space is automatically added between the specified prefix string and the
# command line from the command definition.
#
# *** THIS EXAMPLE MAY POSE A POTENTIAL SECURITY RISK, SO USE WITH CAUTION! ***
# Usage scenario:
# Execute restricted commmands using sudo.  For this to work, you need to add
# the nagios user to your /etc/sudoers.  An example entry for allowing
# execution of the plugins from might be:
#
# nagios          ALL=(ALL) NOPASSWD: /usr/lib/nagios/plugins/
#
# This lets the nagios user run all commands in that directory (and only them)
# without asking for a password.  If you do this, make sure you don't give
# random users write access to that directory or its contents!

# command_prefix=/usr/bin/sudo


# MAX COMMANDS
# This specifies how many children processes may be spawned at any one
# time, essentially limiting the fork()s that occur.
# Default (0) is set to unlimited
# max_commands=0



# COMMAND TIMEOUT
# This specifies the maximum number of seconds that the NRPE daemon will
# allow plugins to finish executing before killing them off.

command_timeout=60



# CONNECTION TIMEOUT
# This specifies the maximum number of seconds that the NRPE daemon will
# wait for a connection to be established before exiting. This is sometimes
# seen where a network problem stops the SSL being established even though
# all network sessions are connected. This causes the nrpe daemons to
# accumulate, eating system resources. Do not set this too low.

connection_timeout=300



# WEAK RANDOM SEED OPTION
# This directive allows you to use SSL even if your system does not have
# a /dev/random or /dev/urandom (on purpose or because the necessary patches
# were not applied). The random number generator will be seeded from a file
# which is either a file pointed to by the environment valiable $RANDFILE
# or $HOME/.rnd. If neither exists, the pseudo random number generator will
# be initialized and a warning will be issued.
# Values: 0=only seed from /dev/[u]random, 1=also seed from weak randomness

#allow_weak_random_seed=1



# SSL/TLS OPTIONS
# These directives allow you to specify how to use SSL/TLS.

# SSL VERSION
# This can be any of: SSLv2 (only use SSLv2), SSLv2+ (use any version),
#        SSLv3 (only use SSLv3), SSLv3+ (use SSLv3 or above), TLSv1 (only use
#        TLSv1), TLSv1+ (use TLSv1 or above), TLSv1.1 (only use TLSv1.1),
#        TLSv1.1+ (use TLSv1.1 or above), TLSv1.2 (only use TLSv1.2),
#        TLSv1.2+ (use TLSv1.2 or above)
# If an "or above" version is used, the best will be negotiated. So if both
# ends are able to do TLSv1.2 and use specify SSLv2, you will get TLSv1.2.
# If you are using openssl 1.1.0 or above, the SSLv2 options are not available.

#ssl_version=SSLv2+

# SSL USE ADH
# This is for backward compatibility and is DEPRECATED. Set to 1 to enable
# ADH or 2 to require ADH. 1 is currently the default but will be changed
# in a later version.

#ssl_use_adh=1

# SSL CIPHER LIST
# This lists which ciphers can be used. For backward compatibility, this
# defaults to 'ssl_cipher_list=ALL:!MD5:@STRENGTH' for < OpenSSL 1.1.0,
# and 'ssl_cipher_list=ALL:!MD5:@STRENGTH:@SECLEVEL=0' for OpenSSL 1.1.0 and
# greater. 

#ssl_cipher_list=ALL:!MD5:@STRENGTH
#ssl_cipher_list=ALL:!MD5:@STRENGTH:@SECLEVEL=0
#ssl_cipher_list=ALL:!aNULL:!eNULL:!SSLv2:!LOW:!EXP:!RC4:!MD5:@STRENGTH

# SSL Certificate and Private Key Files

#ssl_cacert_file=/etc/ssl/servercerts/ca-cert.pem
#ssl_cert_file=/etc/ssl/servercerts/nagios-cert.pem
#ssl_privatekey_file=/etc/ssl/servercerts/nagios-key.pem

# SSL USE CLIENT CERTS
# This options determines client certificate usage.
# Values: 0 = Don't ask for or require client certificates (default)
#         1 = Ask for client certificates
#         2 = Require client certificates

#ssl_client_certs=0

# SSL LOGGING
# This option determines which SSL messages are send to syslog. OR values
# together to specify multiple options.

# Values: 0x00 (0)  = No additional logging (default)
#         0x01 (1)  = Log startup SSL/TLS parameters
#         0x02 (2)  = Log remote IP address
#         0x04 (4)  = Log SSL/TLS version of connections
#         0x08 (8)  = Log which cipher is being used for the connection
#         0x10 (16) = Log if client has a certificate
#         0x20 (32) = Log details of client's certificate if it has one
#         -1 or 0xff or 0x2f = All of the above

#ssl_logging=0x00



# NASTY METACHARACTERS
# This option allows you to override the list of characters that cannot
# be passed to the NRPE daemon.

# nasty_metachars="|`&><'\\[]{};\r\n"



# COMMAND DEFINITIONS
# Command definitions that this daemon will run.  Definitions
# are in the following format:
#
# command[<command_name>]=<command_line>
#
# When the daemon receives a request to return the results of <command_name>
# it will execute the command specified by the <command_line> argument.
#
# Unlike Nagios, the command line cannot contain macros - it must be
# typed exactly as it should be executed.
#
# Note: Any plugins that are used in the command lines must reside
# on the machine that this daemon is running on!  The examples below
# assume that you have plugins installed in a /usr/local/nagios/libexec
# directory.  Also note that you will have to modify the definitions below
# to match the argument format the plugins expect.  Remember, these are
# examples only!


# The following examples use hardcoded command arguments...
# This is by far the most secure method of using NRPE

command[check_users]=/usr/lib64/nagios/plugins/check_users -w 5 -c 10
command[check_load]=/usr/lib64/nagios/plugins/check_load -r -w .15,.10,.05 -c .30,.25,.20
command[check_hda1]=/usr/lib64/nagios/plugins/check_disk -w 20% -c 10% -p /dev/hda1
command[check_zombie_procs]=/usr/lib64/nagios/plugins/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/lib64/nagios/plugins/check_procs -w 150 -c 200


# The following examples allow user-supplied arguments and can
# only be used if the NRPE daemon was compiled with support for
# command arguments *AND* the dont_blame_nrpe directive in this
# config file is set to '1'.  This poses a potential security risk, so
# make sure you read the SECURITY file before doing this.

### MISC SYSTEM METRICS ###
#command[check_users]=/usr/lib64/nagios/plugins/check_users $ARG1$
#command[check_load]=/usr/lib64/nagios/plugins/check_load $ARG1$
#command[check_disk]=/usr/lib64/nagios/plugins/check_disk $ARG1$
#command[check_swap]=/usr/lib64/nagios/plugins/check_swap $ARG1$
#command[check_cpu_stats]=/usr/lib64/nagios/plugins/check_cpu_stats.sh $ARG1$
#command[check_mem]=/usr/lib64/nagios/plugins/custom_check_mem -n $ARG1$

### GENERIC SERVICES ###
#command[check_init_service]=sudo /usr/lib64/nagios/plugins/check_init_service $ARG1$
#command[check_services]=/usr/lib64/nagios/plugins/check_services -p $ARG1$

### SYSTEM UPDATES ###
#command[check_yum]=/usr/lib64/nagios/plugins/check_yum
#command[check_apt]=/usr/lib64/nagios/plugins/check_apt

### PROCESSES ###
#command[check_all_procs]=/usr/lib64/nagios/plugins/custom_check_procs
command[check_procs]=/usr/lib64/nagios/plugins/check_procs $ARG1$

### OPEN FILES ###
#command[check_open_files]=/usr/lib64/nagios/plugins/check_open_files.pl $ARG1$

### NETWORK CONNECTIONS ###
#command[check_netstat]=/usr/lib64/nagios/plugins/check_netstat.pl -p $ARG1$ $ARG2$

### ASTERISK ###
#command[check_asterisk]=/usr/lib64/nagios/plugins/check_asterisk.pl $ARG1$
#command[check_sip]=/usr/lib64/nagios/plugins/check_sip $ARG1$
#command[check_asterisk_sip_peers]=sudo /usr/lib64/nagios/plugins/check_asterisk_sip_peers.sh $ARG1$
#command[check_asterisk_version]=/usr/lib64/nagios/plugins/nagisk.pl -c version
#command[check_asterisk_peers]=/usr/lib64/nagios/plugins/nagisk.pl -c peers
#command[check_asterisk_channels]=/usr/lib64/nagios/plugins/nagisk.pl -c channels 
#command[check_asterisk_zaptel]=/usr/lib64/nagios/plugins/nagisk.pl -c zaptel 
#command[check_asterisk_span]=/usr/lib64/nagios/plugins/nagisk.pl -c span -s 1



# INCLUDE CONFIG FILE
# This directive allows you to include definitions from an external config file.

#include=<somefile.cfg>



# INCLUDE CONFIG DIRECTORY
# This directive allows you to include definitions from config files (with a
# .cfg extension) in one or more directories (with recursion).

#include_dir=<somedirectory>
#include_dir=<someotherdirectory>

include_dir=/etc/nrpe.d/


There is no common.cfg.

Re: Unable to connect to host error

Posted: Tue Sep 22, 2020 12:20 pm
by lmiltchev
I am not sure about that - how would I be able to tell?
Let's do some troubleshooting. Run the following commands from the command line, and show the output:

On the client (remote host):

Code: Select all

ip addr
netstat -apn | grep 5666
/path/to/nrpe -H 127.0.0.1 -c check_procs -a '-w 400 -c 500'
cat /etc/xinetd.d/nrpe
On the Nagios XI server:

Code: Select all

ip addr
nmap <client ip> -p 5666
/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1
/usr/local/nagios/libexec/check_nrpe -H <client ip>
/usr/local/nagios/libexec/check_nrpe -H <client ip> -c check_users
/usr/local/nagios/libexec/check_nrpe -H <client ip> -c check_procs -a '-w 400 -c 500'
Substitute "/path/to/nrpe" and "<client ip>" and the actual values in the above commands.

Re: Unable to connect to host error

Posted: Thu Sep 24, 2020 9:03 am
by userman73
lmiltchev wrote:
I am not sure about that - how would I be able to tell?
Let's do some troubleshooting. Run the following commands from the command line, and show the output:

On the client (remote host):

Code: Select all

ip addr
netstat -apn | grep 5666
/path/to/nrpe -H 127.0.0.1 -c check_procs -a '-w 400 -c 500'
cat /etc/xinetd.d/nrpe
10.0.6.46
tcp 0 0 0.0.0.0:5666 0.0.0.0:* LISTEN -
tcp6 0 0 :::5666 :::* LISTEN -
I tried ./nrpe -H 127.0.0.1 -c check_procs -a '-w 400 -c 500' but got:
./nrpe: invalid option -- 'H'
./nrpe: invalid option -- 'a'
./nrpe: invalid option -- 'w'
./nrpe: invalid option -- ' '
./nrpe: invalid option -- '0'
./nrpe: invalid option -- '0'
./nrpe: invalid option -- ' '
./nrpe: invalid option -- '-'
cat: /etc/xinetd.d/nrpe: No such file or directory


On the Nagios XI server:

Code: Select all

ip addr
nmap <client ip> -p 5666
/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1
/usr/local/nagios/libexec/check_nrpe -H <client ip>
/usr/local/nagios/libexec/check_nrpe -H <client ip> -c check_users
/usr/local/nagios/libexec/check_nrpe -H <client ip> -c check_procs -a '-w 400 -c 500'
10.0.23.158
Starting Nmap 6.47 ( http://nmap.org ) at 2020-09-24 09:58 EDT
Nmap scan report for <vm-name> (10.0.6.46)
Host is up (0.00022s latency).
PORT STATE SERVICE
5666/tcp open nrpe
MAC Address: <mac> (Unknown)
Nmap done: 1 IP address (1 host up) scanned in 0.07 seconds
NRPE v4.0.3
NRPE v3.2.1
USERS OK - 0 users currently logged in |users=0;5;10;0
PROCS OK: 142 processes | procs=142;400;500;0;

Substitute "/path/to/nrpe" and "<client ip>" and the actual values in the above commands.

Re: Unable to connect to host error

Posted: Thu Sep 24, 2020 9:41 am
by lmiltchev
I am confused - in one of your older posts you said:
This seems to have fixed it for this one host. I then applied the fix for another host, and it is still broken. Here is the output of the command:

CHECK_NRPE: Receive header underflow - only 0 bytes received (4 expected).
but now you are showing that all these three commands worked:

Code: Select all

/usr/local/nagios/libexec/check_nrpe -H <client ip>
NRPE v3.2.1

/usr/local/nagios/libexec/check_nrpe -H <client ip> -c check_users
USERS OK - 0 users currently logged in |users=0;5;10;0

/usr/local/nagios/libexec/check_nrpe -H <client ip> -c check_procs -a '-w 400 -c 500'
PROCS OK: 142 processes | procs=142;400;500;0;
Are you running your checks against the host that was "failing"? How's your service configured on the Nagios XI side of things?

Re: Unable to connect to host error

Posted: Thu Sep 24, 2020 10:21 am
by userman73
Sorry, wrong host.

Code: Select all

10.0.1.65
tcp        0      0 0.0.0.0:5666            0.0.0.0:*               LISTEN      1388/nrpe
tcp6       0      0 :::5666                 :::*                    LISTEN      1388/nrpe






Starting Nmap 6.47 ( http://nmap.org ) at 2020-09-24 11:16 EDT
Nmap scan report for <vm-name> (10.0.1.65)
Host is up (0.00018s latency).
PORT     STATE SERVICE
5666/tcp open  nrpe
MAC Address: <mac> (Unknown)
Nmap done: 1 IP address (1 host up) scanned in 0.06 seconds

NRPE v4.0.3
USERS OK - 0 users currently logged in |users=0;5;10;0
CHECK_NRPE: Receive header underflow - only 0 bytes received (4 expected).

Re: Unable to connect to host error

Posted: Thu Sep 24, 2020 10:56 am
by lmiltchev
OK, it seems that commands with "hard-coded" arguments work, but the ones that you need to pass args to the command line - don't. I suspect the issue is caused by one of the two things:

1. You don't have this line set in the nrpe.cfg file on the remote host:

Code: Select all

dont_blame_nrpe=1
or you set it up but forgot to restart the NRPE server after saving the file.

Code: Select all

service nrpe restart
2. You do have this directive set properly and restarted the service, but NRPE was not compiled with the "--enable-command-args" flag.
# COMMAND ARGUMENT PROCESSING
# This option determines whether or not the NRPE daemon will allow clients
# to specify arguments to commands that are executed. This option only works
# if the daemon was configured with the --enable-command-args configure script
# option.

#
# *** ENABLING THIS OPTION IS A SECURITY RISK! ***
# Read the SECURITY file for information on some of the security implications
# of enabling this variable.
#
# Values: 0=do not allow arguments, 1=allow command arguments

dont_blame_nrpe=1
You can fix this by recompiling NRPE on the client.

https://support.nagios.com/kb/article.php?id=515

The issue has been described here:

https://github.com/NagiosEnterprises/nrpe/issues/92

Re: Unable to connect to host error

Posted: Thu Sep 24, 2020 2:08 pm
by userman73
For some reason I see two configuration files:
/etc/nagios/nrpe.cfg
/etc/nrpe/nrpe.cfg

Which one do I want to update? Why are there two?

Re: Unable to connect to host error

Posted: Thu Sep 24, 2020 2:51 pm
by lmiltchev
I don't know why you have two files... Most likely, you installed NRPE twice using different methods. Try to find out which one you are using by running:

Code: Select all

ps -ef | grep nrpe
If NRPE is running as a standalone daemon (as I believe it is on your system), you should see which config is used.

Example from one of my test boxes:

Code: Select all

[root@speedking etc]# ps -ef | grep nrpe
nagios    1133     1  0 Sep17 ?        00:00:22 /usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -f
root     15725   695  0 14:49 pts/0    00:00:00 grep nrpe

Re: Unable to connect to host error

Posted: Tue Sep 29, 2020 7:42 am
by userman73
In the instructions for compile+ install, one of the steps is as follows: "./tools/setup". But I don't see the setup file in the tar.gz in the tools folder.

Re: Unable to connect to host error

Posted: Tue Sep 29, 2020 9:15 am
by lmiltchev
I believe "./tools/setup" is run when you are installing nagios plugins, not NRPE. You have nagios plugins already installed on this machine, don't you? If you do, you won't need to go through these steps.

Having said that, I just downloaded nagios plugins tarball, and the setup script is in the tools directory as expected...

Code: Select all

[root@main-nagios-xi nagios-plugins-release-2.3.3]# pwd
/tmp/nagios-plugins-release-2.3.3
[root@main-nagios-xi nagios-plugins-release-2.3.3]# ls -la tools/
total 100
drwxrwxr-x  2 root root  4096 Mar 11  2020 .
drwxrwxr-x 17 root root  4096 Mar 11  2020 ..
-rwxrwxr-x  1 root root 10010 Mar 11  2020 build_perl_modules.in
-rwxrwxr-x  1 root root   515 Mar 11  2020 devmode
-rwxrwxr-x  1 root root  1409 Mar 11  2020 distclean
-rwxrwxr-x  1 root root  1225 Mar 11  2020 find_authors
-rwxrwxr-x  1 root root  2042 Mar 11  2020 generate-change-log
-rwxrwxr-x  1 root root 21641 Mar 11  2020 git-notify
-rwxrwxr-x  1 root root   535 Mar 11  2020 git-post-receive-hook
-rw-rw-r--  1 root root  3598 Mar 11  2020 mini_epn.c
-rw-rw-r--  1 root root   698 Mar 11  2020 oneliners
-rw-rw-r--  1 root root  3787 Mar 11  2020 p1.pl
-rw-rw-r--  1 root root   339 Mar 11  2020 README
-rwxrwxr-x  1 root root  1283 Mar 11  2020 setup
-rwxrwxr-x  1 root root  4719 Mar 11  2020 tango
-rwxrwxr-x  1 root root  7074 Mar 11  2020 tinderbox_build
Is your check working now, after recompiling NRPE?