Custom check from check_procs

[Tampa]

Been tinkering with this for a while now and I am running out of things to try. I'm trying to create a custom check using the check_procs as basis.

I have added the following to the commands.cfg on the nagios-core:

Code: Select all

# 'check_nrpe' command definition
define command{
       command_name    check_nrpe
       command_line    $USER1$/check_nrpe -H $HOSTADDRESS$ -t 30 -c $ARG1$ $ARG2$
       }

# 'check_opensim' command definition
define command{
       command_name    check_opensim
       command_line    $USER1$/check_opensim -C $ARG1$
       }

Inside the configuration for the monitor I appended the following:

Code: Select all

define service {
      host_name tampaserver
      service_description OpenSim tampashop
      check_command check_nrpe!check_opensim!"mono OpenSim.exe -inidirectory=/home/simulators/tampashop/"
      max_check_attempts 2
      check_interval 2
      retry_interval 2
      check_period 24x7
      check_freshness 1
      contact_groups admins
      notification_interval 2
      notification_period 24x7
      notifications_enabled 1
      register 1
}

In the nrpe.cfg on the monitor I added:

Code: Select all

command[check_opensim]=/usr/lib/nagios/plugins/check_procs -C $ARG1$

Nagios itself is giving me: (No output on stdout) stderr: execvp(/usr/local/nagios/libexec/check_nrpe, ...) failed. errno is 2: No such file or directory

I have installed nrpe-plugins on both host and monitor and nrpe-server on the monitor using apt.
My goal is to be able to define further services in the hosts monitor configuration for different processes.
The process on the monitor using ps -aux:

Code: Select all

simulat+ 25703  1.2  0.2 4468204 146928 pts/5  Ssl+ Feb26   3:15 mono OpenSim.exe -inidirectory=/home/simulators/tampashop/

EDIT:

After changing the path for check_nrpe to where nagios tries to find it I am now getting: (Return code of 255 is out of bounds : (No output on stdout) stderr: connect to address xx.xx.xx.xx port 5666: Connection refused))
There are no firewalls on either machine, but nmap still claims the port to be closed. I added the port to iptables, but that had no effect.

[mcapra]

If nmap from the Nagios Core machine to the NRPE machine says the port is closed, it's probably not wrong. We can't provide much assistance with firewall/networking related issues though.

That aside, lets say I want to monitor the qmgr process with specific arguments:

Code: Select all

postfix   2332  0.0  0.1  91836  4560 ?        S    Feb09   0:21 qmgr -l -t unix -u

Using check_procs, I can monitor all instances of qmgr just fine:

Code: Select all

[root@localhost ~]# /usr/local/nagios/libexec/check_procs -C qmgr
PROCS OK: 12 process with command name 'qmgr' | procs=12;;;0;

But maybe I only care specifically about qmgr processrs with the arguments -l -t unix -u. To look only at those processes, I need to leverage the -a argument like so:

Code: Select all

[root@localhost ~]# /usr/local/nagios/libexec/check_procs -C qmgr -a '-l -t unix -u'
PROCS OK: 1 process with command name 'qmgr', args '-l -t unix -u' | procs=1;;;0;

So, what I would try locally on the machine running OpenSim.exe, is to run the following and seeing if that produces the desired behavior:

Code: Select all

/usr/lib/nagios/plugins/check_procs -C mono -a 'OpenSim.exe -inidirectory=/home/simulators/tampashop/'

[Tampa]

Thank you for your reply.
The network stuff is a bit strange to me, there is only iptables running on both machines, but neither has any block rule assigned to that port and iptables should by default allow anything above 1024 to go through. My guess is my command syntax is not correct, which is why I posted all my changes. I will try the normal check_procs instead of making my own command, hopefully that works better.

[Tampa]

Sorry for the double post, but I seem to have figured it out:

The monitor.cfg:

Code: Select all

check_command check_procs!mono!"OpenSim.exe -inidirectory=/home/simulators/tampashop/"

monitors nrpe.cfg:

Code: Select all

command[check_procs]=/usr/lib/nagios/plugins/check_procs -C $ARG1$ -a $ARG2$

and the commands.cfg:

Code: Select all

# 'check_procs' command definition
define command{
       command_name    check_procs
       command_line    $USER1$/check_procs -C $ARG1$ -a $ARG2$
       }

Which means I don't need check_nrpe at all on the monitor.cfg, which was my initial impression. I should just be able to create commands using the commands.cfg and nrpe.cfg no need to then call check_nrpe.
I should now be able to create own check functions I think.

[Tampa]

Some more hours of tinkering attempting to get anything out of it. I can easily make something on the monitors nrpe to give me back the info if stuff is running or now, but ideally I would like to only have to edit nagios host server.cfgs.

On the monitor:

Code: Select all

/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1
NRPE v3.0.1

/usr/lib/nagios/plugins/check_procs -c 1:1 -C mono -a inidirectory=/home/simulators/tampaindustries/
PROCS OK: 1 process with command name 'mono', args 'inidirectory=/home/simulators/tampaindustries/' | procs=1;;1:1;0;

on the host:

Code: Select all

/usr/lib/nagios/plugins/check_nrpe -H xx.xx.xx.xx -p 5666 -c check_opensim -a "mono" "inidirectory=/home/simulators/tampaindustries"
NRPE: Unable to read output

any other check like check_load results in the same NRPE error.

Code: Select all

command[check_opensim]=/usr/local/nagios/libexec/check_procs -c 1:1 -C $ARG1$ -a $ARG2$

# 'check_nrpe' command definition
define command{
        command_name    check_nrpe
        command_line    /usr/lib/nagios/plugins/check_nrpe -H $HOSTADDRESS$ -t 30 -c $ARG1 -a
       }

# 'check_opensim' command definition
define command{
        command_name    check_opemsim
        command_line    $USER1$/check_procs -C $ARG1 -a $ARG2$
       }

define service {
      host_name tampaserver
      service_description OpenSim tampaindustries
      check_command check_nrpe!check_opensim!mono!-inidirectory=/home/simulators/tampaindustries/
      max_check_attempts 2
      check_interval 2
      retry_interval 2
      check_period 24x7
      check_freshness 1
      contact_groups admins
      notification_interval 2
      notification_period 24x7
      notifications_enabled 1
      register 1
}

I have about 100 tabs open now about this, tried it all, compiled nrpe from scratch to make sure commands are enabled. Apart from server address and blame_nrpe and allowed hosts I have not changed the default nrpe.cfg. All other entries above. Could you provide me with the proper syntax for what I am trying to do, I can't seem to wrap my head around it?

[mcapra]

It looks like here you're calling check_opensim instead of check_procs. Though check_opensim exists in your Nagios Core configuration set, that set is not transferred to the remote NRPE machine.

How check_nrpe works is by feeding the remote machine a command to run with the -c argument. However, this argument does not represent a Nagios Core command; It represents an NRPE command. If the command is not defined in your NRPE configuration, you'll run into issues. In this case, it looks like you are attempting to pass a Nagios Core command:

Code: Select all

/usr/lib/nagios/plugins/check_nrpe -H xx.xx.xx.xx -p 5666 -c check_opensim -a "mono" "inidirectory=/home/simulators/tampaindustries"
NRPE: Unable to read output

...

# 'check_opensim' command definition
define command{
        command_name    check_opemsim
        command_line    $USER1$/check_procs -C $ARG1 -a $ARG2$
       }

The problem with this is that NRPE (on it's own) has zero concept of what is defined in your Nagios Core environment. This is by design.

NRDS/NRDP was designed for what it looks like you're trying to achieve; Have all the things you'd ever need to configure set up on your Nagios Core machine and never ever touch the NRPE configuration on any given remote machine beyond initial setup. Though setting up any sort of push/pull between Nagios Core and it's various monitoring agents is non-trivial. More info on NRDS/NRDP:
https://assets.nagios.com/downloads/nrd ... erview.pdf

[Tampa]

EDIT: Last question, best way to install the plugins on the nrpe machine? I only have check_nrpe in that libexec, can I just copy the rest from nagios itself?

Figured this out, hence edit, moved the check_procs to the nrpe machine and edited the definitions slightly now it reports back properly.

Code: Select all

command[check_opensim]=/usr/local/nagios/libexec/check_procs -c 1:1 -C '$ARG1$' -a '$ARG2$'

define service {
      host_name tampaserver
      service_description OpenSim tampashop
      check_command check_opensim!mono!inidirectory=/home/simulators/tampashop/
      max_check_attempts 2
      check_interval 2
      retry_interval 2
      check_period 24x7
      check_freshness 1
      contact_groups admins
      notification_interval 2
      notification_period 24x7
      notifications_enabled 1
      register 1
}

# 'check_opensim' command definition
define command{
        command_name    check_opensim
        command_line    $USER1$/check_nrpe -H $HOSTADDRESS$ -t 30 -c check_opensim -a '$ARG1$' '$ARG2$'
       }

Hopefully the next person having this trouble finds this thread and doesn't have to tinker for two days Thanks for the help.

[mcapra]

Thanks for sharing your solution!

EDIT: Last question, best way to install the plugins on the nrpe machine? I only have check_nrpe in that libexec, can I just copy the rest from nagios itself?

If we're talking about the nagios-plugins package, the best way is to build them on the remote machine. Copy+paste will only sometimes work if it's the same architecture+os as the Nagios XI machine. This document covers building NRPE/nagios-plugins from scratch:
https://support.nagios.com/kb/article.php?id=515

Since you've already got NRPE figured out, refer to the "Installing The Nagios Plugins" section.