NRPE monitor Windows Clients connected to the Nagios Remote

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
xinxolHH
Posts: 14
Joined: Mon Feb 18, 2019 4:47 am

NRPE monitor Windows Clients connected to the Nagios Remote

Post by xinxolHH »

I have two Nagios server, A (Nagios Core 4.3.4) - Local Nagios, and B the Remote Nagios Server (NRPE, Xinetd, Nagios plugins).
I have configured them and installed with NRPE, (following the manual here) so now I can monitor my B Remote Server where the plugins are also installed-

Now the complex Second step:
The Remote Server B has Windows clients which are only reachable from the location B, from that Remote Server B; (only when accesesing Remote Server B). So there is no connection from Nagios Server A to Clients at the location B directly, it has to be connected/reported through the Remote Server B. PC Clients on the Location B, have the NSCP Client (NSClient++) installed with the NRPE module enabled. I want to configure them to get them monitored reported from server A.

When executing the NRPE command on the Nagios Server A - I get the SSL Error for the Remote PC Client 1B
(just omiting IP addresses, and writing the generic names)

Code: Select all

    
/usr/lib64/nagios/plugins/check_nrpe -H "PC Client 1B"
 CHECK_NRPE: (ssl_err != 5) Error - Could not complete SSL handshake with "PC Client 1B"
when I do it for the Remote Server B, this is working fine, and I can monitor resource at Remote Server B, with no issues.

Code: Select all

/usr/lib64/nagios/plugins/check_nrpe -H "Remote Server B"
 NRPE v3.2.1
The configuration on the Client PC 1B is as follows (nsclient.ini):

Code: Select all

 [/settings/NRPE/client]

    channel = NRPE

    [/settings/NRPE/client/targets/default]

    timeout = 30

    verify mode = peer-cert

    use ssl = 1

    insecure = false

    [/settings/NRPE/server]

    allowed hosts =localhost,"Nagios Server A"

    port = 5666

    allow arguments=1

    use ssl = 1

    ssl options = no-sslv2,no-sslv3

    verify mode = peer-cert

    insecure = false


Has anyone any example of the configuration /syntax of the nrpe commands to monitor the PC Clients at the remote location B, or any direction which I should look into it. Should I define a new "check_nrpe_external" command

The interest is to monitor the Local Disk of the remote PC Clients, their CPU usage, the numbers of users connected, etc, so we could know if the IT system are used on location B. When it is working for location B, it would be more locations to be added to the monitoring system.

thanks & Best Regards
Attachments
Structure of the Set-Up - remote Location  B
Structure of the Set-Up - remote Location B
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: NRPE monitor Windows Clients connected to the Nagios Rem

Post by ssax »

Please PM me your /usr/local/nagios/etc/nrpe.cfg and /usr/local/nagios/etc/nrpe/common.cfg files from Nagios Server B.

Please post the output of this command on Nagios Server A, it could be a version mismatch, you likely need to upgrade Nagios Server A's check_nrpe command to NRPE 3.X for it to work:

Code: Select all

/usr/lib64/nagios/plugins/check_nrpe -V
Which versions of NSClient++ is the Remote PC Client B1 machine running?

As long as check_nrpe is on server A and server B it's definitely doable, we just need to make sure we're passing the correct stuff and fix what is likely an SSL/version issue.

So essentially you're going to do something like this:

Nagios A runs:

Code: Select all

/usr/lib64/nagios/plugins/check_nrpe -H 'NAGIOSB' -c check_nrpe_external -a "'Remote PC Client B1 IP/DNS Name' 'PCB1_nsclient_command_to_run'"
Nagios B would have this in its nrpe.cfg:

Code: Select all

command[check_nrpe_external]=/usr/local/nagios/libexec/check_nrpe -H $ARG1$ -c $ARG2$
Which would then reach out to Remote PC Client B1 and check what it needs to check.

There could be other methods but we need to know the complete configuration and versions to recommend the best approach.
xinxolHH
Posts: 14
Joined: Mon Feb 18, 2019 4:47 am

Re: NRPE monitor Windows Clients connected to the Nagios Rem

Post by xinxolHH »

Hello ssax,

Thanks for the reply, I have sent you the nrpe.cfg file, but I have no common.cfg file installed in Nagios B server.

Nagios NRPE installed on Server A is:

Code: Select all

[root@TestNagiosA ]# /usr/lib64/nagios/plugins/check_nrpe -V
NRPE Plugin for Nagios
Version: 3.2.1
it is also the same version on Server B.

Windows Clients on Location B have the following NSClient+++ version, 0.5.235 from last year in 2018

Code: Select all

NSClient++ 0.5.2.35 2018-01-28 
I have also disabled the Xinetd and nrpe deamon is running on standalone mode on Server B.

I am insterested about the syntax written below, as my understanding on how to run the check_nrpe commands is unclear
I mainly want to execute the check_load, check_users, check_disk. check_men kind of commands on the PC windows clients in localtion B.

When executing the commands as stated above, I am getting the following error

Code: Select all


[root@NagiosServerA]# /usr/local/nagios/libexec/check_nrpe -t 30 -H "IPNagiosB"  -c check_nrpe_external  -a "'IPCLient1B' 'alias_cpu'"
CHECK_NRPE: Receive header underflow - only 0 bytes received (4 expected).


Best regards
X.
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: NRPE monitor Windows Clients connected to the Nagios Rem

Post by ssax »

You'll need to define the commands in your nrpe.cfg on server B, the commands I sent were just an example of how it needs to be done.

You could do it multiple ways but the end result is the same: Server B is doing the actual checking, Server A is just kicking off the check and receiving the results.

So for that to work, you would need to define a command on Server B called check_nrpe_external in order to use it, it would be something like this:
- It will differ depending on how you want to do it

Code: Select all

command[check_nrpe_external]=/usr/local/nagios/libexec/check_nrpe -t 30 -H $ARG1$ -c $ARG2$ $ARG3$
Then you would call it from Server A like this:

Code: Select all

[root@NagiosServerA]# /usr/local/nagios/libexec/check_nrpe -t 30 -H SERVERB -c check_nrpe_external -a CLIENT1B check_cpu '-a show-all'
So that would have SERVERA reaching out to SERVERB which then reaches out to CLIENT1B.

That should work
xinxolHH
Posts: 14
Joined: Mon Feb 18, 2019 4:47 am

Re: NRPE monitor Windows Clients connected to the Nagios Rem

Post by xinxolHH »

Hello

thanks for the reply, but I am still not get it to work- I would need an example which works,

When passing the commands as mentioned below, I get "Unable to read output"

Code: Select all

[root@TestNagiosA]# /usr/local/nagios/libexec/check_nrpe -t 30 -H ServerB -c check_nrpe_external -a PCClient1B check_users '-a -w 3 -c 5'
NRPE: Unable to read output
when check_users has not arguments, it is hardcoded in the nrpe.cfg in the Server B, I also get same "Unable to read output"

Code: Select all

[root@TestNagiosA]# /usr/local/nagios/libexec/check_nrpe -t 30 -H ServerB -c check_nrpe_external -a PCClient1B check_users
NRPE: Unable to read output
when changing the arguments to -H, I do not know if this is a correct option in the second command, I get connection refused

Code: Select all

[root@TestNagiosA]# /usr/local/nagios/libexec/check_nrpe -t 30 -H ServerB -c check_nrpe_external -H PCClient1B
connect to address PCClient1B port 5666: Connection refused
connect to host PCClient1B port 5666: Connection refused

ServerB and PCClient1B are given with IP addresses, I just "renamed" them here for general purpose. I am executing the commands at the terminal as root. So I really do not get it.

A bit more of trouble shooting, provide the following hint, about the DH key being too small, but I am not unware how to change it
from the remote NagiosServer B, to the PCClients inside the remote location B.

Code: Select all

check_nrpe: Error: (!log_opts) Could not complete SSL handshake with "PCCLient1B": dh key too small
and those are my settings on the remote NSClient++, as the servers (Server A, Server B) are configured with TLSv1.2+ options, and enabling TLS options in the config file of the NRPE 3.2.1 version (./configure --enable-command-args --enable-ssl)

Code: Select all

certificate = ${certificate-path}/certificate.pem,10.
ssl options = no-sslv2,no-sslv3
allowed ciphers = ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH

Thanks in advanced for any further suggestion,
X.
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: NRPE monitor Windows Clients connected to the Nagios Rem

Post by tgriep »

I think there are a few changes that have to be done on Server B.

In the nrpe.cfg file create a command like the following.

Code: Select all

command[check_nrpe_external]=/usr/local/nagios/libexec/check_nrpe $ARG1$
Also, make sure the NRPE agent accepts Arguments in the commands by setting the following to a 1.

Code: Select all

dont_blame_nrpe=1
Save the change and restart NRPE.

Make sure you have the Windows server's nsclient.ini file setup for NRPE access and that Server B's IP address is in the allowed hosts section.

Then on Server A, this is an example command to return the version of the NRPE agent on Server B to verify that you can connect.
Replace xxx.xxx.xxx.xxx with the IP address of Server B

Code: Select all

/usr/local/nagios/libexec/check_nrpe -H xxx.xxx.xxx.xxx
If it returns the version, that is a good thing.

The next command will try to connect to the Windows Host and return the version.
Replace xxx.xxx.xxx.xxx with the IP address of Server B and yyy.yyy.yyy.yyy with the IP address of the Windows server.

Code: Select all

/usr/local/nagios/libexec/check_nrpe -H xxx.xxx.xxx.xxx -c check_nrpe_forward -a '-H yyy.yyy.yyy.yyy'
If that returns the NSClient++ version, then you connected to the Windows server.

If NSClient is enabled to run checks, the following will run the check_cpu on the Windows system.

Code: Select all

/usr/local/nagios/libexec/check_nrpe -H xxx.xxx.xxx.xxx -c check_nrpe_forward -a '-H yyy.yyy.yyy.yyy -c check_cpu'
The biggest thing is to setup the the NRPE agent to allow arguments as well as NSClient, that gives most people problems.

Also, the -a argument, wrapping them in single quotes works for most commands but you will have to adjust it if there are any special characters.
Be sure to check out our Knowledgebase for helpful articles and solutions!
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: NRPE monitor Windows Clients connected to the Nagios Rem

Post by ssax »

First, let's test NRPE communication between NAGIOSA and NAGIOSB:

Run this command on NAGIOSA:
-- This will test communication between NAGIOSA and NAGIOSB

Code: Select all

/usr/local/nagios/libexec/check_nrpe -t 30 -H B.B.B.B
If that is not working, you need to get at least that to work before moving onto the next step.

If it is working (should return the NRPE version), we know it's communicating from NAGIOSA to NAGIOSB just fine so it's not communication that's the problem.

Once that's working, now test NRPE communication between NAGIOSB and PCLIENT1C (calling it C is easier in this example):

Run this command on NAGIOSB:
-- This will test communication between NAGIOSB and PCLIENT1C

Code: Select all

/usr/local/nagios/libexec/check_nrpe -t 30 -H C.C.C.C
Once those are both working and returning version numbers then we can move onto the next steps.
xinxolHH
Posts: 14
Joined: Mon Feb 18, 2019 4:47 am

Re: NRPE monitor Windows Clients connected to the Nagios Rem

Post by xinxolHH »

Hello ssax and tgriep

I have done more operations and troubleshooting according your recommendations, and checking the NSClient++

from server A - I can reach Server B with no Issues,

Code: Select all

[root@TestNagiosA]# /usr/local/nagios/libexec/check_nrpe -H ServerB
NRPE v3.2.1
also the option for arguments, "dont_blame_nrpe=1" is now enabled.

I have the following definition for check_nrpe_external on the server B, as mentioned in this post earlier (important, adding the -n option):

Code: Select all


command[check_nrpe_external]=/usr/local/nagios/libexec/check_nrpe -t 90 -H $ARG1$ -n -c $ARG2$ $ARG3$

and then on the NSClient.ini file, all setting regarding TLS are now disabled (Server B - PCs Clients insidde Location B)

Code: Select all


[/settings/NRPE/server]
insecure = true
verify mode = none
allow arguments = true
allow nasty characters = 1
allowed host =SERVERA,SERVERB,sub-net_from_serverB,internal_subnet_LocationB
extended response = 0
use ssl = false
; PORT NUMBER - Port to use for NRPE.
port = 5666

with those seeting, now finally works, from Server A, when executing the commads on the SERVER A, so I get the expected response, using the -n option in the check_nrpe_external defintion, and the following syntax

Code: Select all


[root@TestNagiosA ]# /usr/local/nagios/libexec/check_nrpe -t 90 -H "IPofServerB"  -c check_nrpe_external -a "IPofClientPC1B"  check_cpu '-a show-all'

OK: 5m: 0%, 1m: 0%, 5s: 0%|'total 5m'=0%;80;90 'total 1m'=0%;80;90 'total 5s'=0%;80;90

Just also to notice that PCClient1B on Location B, has two NIC cards, (additional one for the internal LAN) and both IP addresses are used for the monitoring, so both are IP ranges are included in the allowed host settings.

Now that works in the command line, I have the smaller issues of how to "translate" the syntax to the cfg files, with the current definition of the check_nrpe_external on SERVERB

Code: Select all

command[check_nrpe_external]=/usr/local/nagios/libexec/check_nrpe -t 90 -H $ARG1$ -n -c $ARG2$ $ARG3$
and the LocationB.cfg file, may contain the following syntax on the SERVER A at /etc/nagios/LocationB/ directory

Code: Select all

define service {
        use                                    generic-service
        host_name                         ClientPC1B
        service_description            Remote_CPU
        check_command                check_nrpe_ext!check_nrpe_external!check_cpu '-a show-all'
}
and where check_nrpe_ext is the local check_nrpe on SERVERA

Code: Select all


define command {
  command_name check_nrpe_ext
  command_line $USER1$/check_nrpe -t 90 -H IPofSERVERB -c $ARG1$
}

and whith that syntax I do not get it right, I get status UNKNOWN on the monitoring interface

Code: Select all


 UNKNOWN  

NRPE Plugin for Nagios Version: 3.2.1

how to pass the arguments in the LocationB.cfg file, it is the last step, any hint will help me. I am also a bit not understanding the complete set-up, as when I do the same commands on the SERVER A, with a different plugin, i am getting different anwers, not desired answers. I am doing something wrong, or is this a bug in the NRPE

For example when checking the amount of users on the remote PCClien1B from Server A in the command line:

Code: Select all

[root@TestNagiosA]# /usr/local/nagios/libexec/check_nrpe -t 90 -H IPofServerB  -c check_nrpe_external -a IPofPCClien1B -n check_users -w 2 -c 3
I get the following not desired output:

Code: Select all

NRPE Plugin for Nagios
Version: 3.2.1

Copyright (c) 2009-2017 Nagios Enterprises
              1999-2008 Ethan Galstad (nagios@nagios.org)

Last Modified: 2017-09-01

License: GPL v2 with exemptions (-l for more info)

SSL/TLS Available: OpenSSL 0.9.6 or higher required

Usage: check_nrpe -H <host> [-2] [-4] [-6] [-n] [-u] [-V] [-l] [-d <dhopt>]
       [-P <size>] [-S <ssl version>]  [-L <cipherlist>] [-C <clientcert>]
       [-K <key>] [-A <ca-certificate>] [-s <logopts>] [-b <bindaddr>]
       [-f <cfg-file>] [-p <port>] [-t <interval>:<state>] [-g <log-file>]
       [-c <command>] [-E] [-a <arglist...>]

Options:
 -H, --host=HOST              The address of the host running the NRPE daemon
 -2, --v2-packets-only        Only use version 2 packets, not version 3
 -4, --ipv4                   Bind to ipv4 only
 -6, --ipv6                   Bind to ipv6 only
 -n, --no-ssl                 Do no use SSL
 -u, --unknown-timeout        Make connection problems return UNKNOWN instead of CRITICAL
 -V, --version                Print version info and quit
 -l, --license                Show license
 -E, --stderr-to-stdout       Redirect stderr to stdout
 -d, --use-dh=DHOPT           Anonymous Diffie Hellman use:
                              0         Don't use Anonymous Diffie Hellman
                                        (This will be the default in a future release.)
                              1         Allow Anonymous Diffie Hellman (default)
                              2         Force Anonymous Diffie Hellman
 -P, --payload-size=SIZE      Specify non-default payload size for NSClient++
 -S, --ssl-version=VERSION    The SSL/TLS version to use. Can be any one of:
                              SSLv2     SSL v2 only
                              SSLv2+    SSL v2 or above
                              SSLv3     SSL v3 only
                              SSLv3+    SSL v3 or above
                              TLSv1     TLS v1 only
                              TLSv1+    TLS v1 or above (DEFAULT)
                              TLSv1.1   TLS v1.1 only
                              TLSv1.1+  TLS v1.1 or above
                              TLSv1.2   TLS v1.2 only
                              TLSv1.2+  TLS v1.2 or above
 -L, --cipher-list=LIST       The list of SSL ciphers to use (currently defaults
                              to "ALL:!MD5:@STRENGTH". THIS WILL change in a future release.)
 -C, --client-cert=FILE       The client certificate to use for PKI
 -K, --key-file=FILE          The private key to use with the client certificate
 -A, --ca-cert-file=FILE      The CA certificate to use for PKI
 -s, --ssl-logging=OPTIONS    SSL Logging Options
 -b, --bind=IPADDR            Local address to bind to
 -f, --config-file=FILE       Configuration file to use
 -g, --log-file=FILE          Log file to write to
 -p, --port=PORT              The port on which the daemon is running (default=5666)
 -c, --command=COMMAND        The name of the command that the remote daemon should run
 -a, --args=LIST              Optional arguments that should be passed to the command,
                              separated by a space. If provided, this must be the last
                              option supplied on the command line.

 NEW TIMEOUT SYNTAX
 -t, --timeout=INTERVAL:STATE
                              INTERVAL  Number of seconds before connection times out (default=10)
                              STATE     Check state to exit with in the event of a timeout (default=CRITICAL)
                              Timeout STATE must be a valid state name (case-insensitive) or integer:
                              (OK, WARNING, CRITICAL, UNKNOWN) or integer (0-3)

Note:
This plugin requires that you have the NRPE daemon running on the remote host.
You must also have configured the daemon to associate a specific plugin command
with the [command] option you are specifying here. Upon receipt of the
[command] argument, the NRPE daemon will run the appropriate plugin command and
send the plugin output and return code back to *this* plugin. This allows you
to execute plugins on remote hosts and 'fake' the results to make Nagios think
the plugin is being run locally.


Many thanks for your support.

Best Regards
X.
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: NRPE monitor Windows Clients connected to the Nagios Rem

Post by ssax »

Here is what worked for me:

NAGIOSA:

Code: Select all

define service {
    use                    generic-service
    host_name              ClientPC1B
    service_description    Remote_CPU
    check_command          check_nrpe_ext!192.168.5.232!check_cpu!'-a show-all'!!!!!
    register               1
}

define command {
    command_name    check_nrpe_ext
    command_line    $USER1$/check_nrpe -t 90 -H $HOSTADDRESS$ -c check_nrpe_external -a $ARG1$ $ARG2$ $ARG3$
}
NAGIOSB /usr/local/nagios/etc/nrpe.cfg:

Code: Select all

command[check_nrpe_external]=/usr/local/nagios/libexec/check_nrpe -t 90 -H $ARG1$ -c $ARG2$ $ARG3$
Since your ClientPC1B NSClient++ agent doesn't use SSL, yours would be:

Code: Select all

command[check_nrpe_external]=/usr/local/nagios/libexec/check_nrpe -t 90 -H $ARG1$ -n -c $ARG2$ $ARG3$
xinxolHH
Posts: 14
Joined: Mon Feb 18, 2019 4:47 am

Re: NRPE monitor Windows Clients connected to the Nagios Rem

Post by xinxolHH »

Hello ssax,

Thanks for your reply, and for the help in the syntax, but I am not fully undersdanding it yet. I have now another additional question which I do not understand. It does not work for me as you said, and I may think 192.168.532 is the equivalent to my NagiosB server, and the command check_cpu always worked, but not with the syntax specificied below:

Code: Select all

define service {
    use                    generic-service
    host_name              ClientPC1B
    service_description    Remote_CPU
    check_command          check_nrpe_ext!192.168.5.232!check_cpu!'-a show-all'!!!!!
    register               1
}
When executing the "part" of the command on NagiosB server on the terminal it works fine, with the check_cpu example, but not with the others - I get the message the commands are not found as for example, check_users - command not known - which it can be found on the defined directory /usr/lib64/nagios/plugins/, but It does not find the check_cpu on my NagiosB server.

Code: Select all

[root@TestNagiosB]# find / -name check_cpu
[root@TestNagiosB]#
and also check_cpu is not defined in the nrpe.cfg, from where it execute then ?

Code: Select all

[root@TestNagiosB]# /usr/local/nagios/libexec/check_nrpe -t 90  -H IPofPCClient1B -n -c  check_cpu 'show-all'
OK: CPU load is ok.|'total 5m'=0%;80;90 'total 1m'=0%;80;90 'total 5s'=0%;80;90

Code: Select all

[root@TestNagiosB]# /usr/local/nagios/libexec/check_nrpe -t 90  -H IPofClient1B -n -c  check_users
Unknown command(s): check_users

Code: Select all

[root@TestNagiosB]# /usr/local/nagios/libexec/check_nrpe -t 90  -H IPofClient1B -n -c check_users '-w 2 -c 3'
/usr/local/nagios/libexec/check_nrpe: invalid option -- 'w'
/usr/local/nagios/libexec/check_nrpe: invalid option -- ' '
/usr/local/nagios/libexec/check_nrpe: invalid option -- ' '
/usr/local/nagios/libexec/check_nrpe: invalid option -- '-'
NRPE Plugin for Nagios
Version: 3.2.1

Copyright (c) 2009-2017 Nagios Enterprises
              1999-2008 Ethan Galstad (nagios@nagios.org)

Last Modified: 2017-09-01

License: GPL v2 with exemptions (-l for more info)

SSL/TLS Available: OpenSSL 0.9.6 or higher required

Usage: check_nrpe -H <host> [-2] [-4] [-6] [-n] [-u] [-V] [-l] [-d <dhopt>]
       [-P <size>] [-S <ssl version>]  [-L <cipherlist>] [-C <clientcert>]
       [-K <key>] [-A <ca-certificate>] [-s <logopts>] [-b <bindaddr>]
       [-f <cfg-file>] [-p <port>] [-t <interval>:<state>] [-g <log-file>]
       [-c <command>] [-E] [-a <arglist...>]

....etc

Code: Select all

[root@TestNagiosB]# find / -name check_users
/usr/lib64/nagios/plugins/check_users
and of course in the file *.cfg I get an error as well, any help will be appreciate, and not only a recipe, I need to understand it and to know how it works.

thanks,
X.
Locked