Event handler on hostgroups

[sinaowolabi]

Hi

I am wondering what the proper configuration syntax is, in order to have event handler on a host group service.
By this I mean, what is the correct way to inform the event handler service of the correct hostname/address to pass on to the event handlers script?

Currently I have these defined:

Code: Select all

define command  {
                        command_name     restart-x
                        command_line     $USER1$/eventhandlers/x_reset  $SERVICESTATE$ $SERVICESTATETYPE$ $SERVICEATTEMPT$ $HOSTNAME$ $ARG1$
}

define service {
        check_command               check_nrpe!check_x
        contacts                            contactsadmin
        event_handler                  restart-x
        event_handler_enabled          1
        hostgroup_name                 x_servers
        max_check_attempts             3
        notification_interval          60
        notification_period            24x7
        service_description            Check X Service
}

I am expecting the service check to send the hostname or the IP address of the host that triggers a problem, I see this in the logs, but the event handler script isn't triggered:

Code: Select all

[Tue Apr 20 14:19:41 2021] SERVICE EVENT HANDLER: server1;Check X Service;CRITICAL;SOFT;1;restart-x
[Tue Apr 20 14:20:51 2021] SERVICE ALERT: server1;Check X Service;CRITICAL;SOFT;2;CRITICAL - Plugin timed out while executing system call
[Tue Apr 20 14:20:51 2021] SERVICE EVENT HANDLER: server1;Check X Service;CRITICAL;SOFT;2;restart-x

Please how do I make the event handler script receive the hostname or IP address of the host on which to run the script?

[benjaminsmith]

Hi,

Please how do I make the event handler script receive the hostname or IP address of the host on which to run the script?

You'll want to pass the host address ($HOSTADDRESS$) macro as an argument to the restart-x event handler using the standard Nagios Core Marcos that are available.

https://assets.nagios.com/downloads/nag ... olist.html

This is a handy guide for learning how to make event handlers. Take a look at page 10 for an example case of adding macros to an event handler.

### EVENT HANDLERS
Introduction to Event Handlers

Let me know if that helps answer your question or not.

Cheers,
Benjamin

[sinaowolabi]

Thanks for the reply!
Im a little confused, I thought I was passing macros already by doing this:

Code: Select all

define command  {
                        command_name     restart-x
                        command_line     $USER1$/eventhandlers/x_reset  $SERVICESTATE$ $SERVICESTATETYPE$ $SERVICEATTEMPT$ $HOSTNAME$ $ARG1$
}

Or do you mean I should do this?

Code: Select all

define service {
        check_command               check_nrpe!check_x
        contacts                            contactsadmin
        event_handler                  restart-x!$HOSTADDRESS$
        event_handler_enabled          1
        hostgroup_name                 x_servers
        max_check_attempts             3
        notification_interval          60
        notification_period            24x7
        service_description            Check X Service
}

[benjaminsmith]

Hi,

I would recommend using the host address field instead here I would also recommend using the IP address instead of the FQDN in case the DNS fails.

According to the log, the event handler is called. However, Check X Service is timing out. That's usually a network connectivity issue. Run the plugin directly from the command line and let me know if it's working or not.

[Tue Apr 20 14:19:41 2021] SERVICE EVENT HANDLER: server1;Check X Service;CRITICAL;SOFT;1;restart-x
[Tue Apr 20 14:20:51 2021] SERVICE ALERT: server1;Check X Service;CRITICAL;SOFT;2;CRITICAL - Plugin timed out while executing system call

[sinaowolabi]

Yes its working, its a DNS check.
I just generalised it for posting here, it what happens when the dns service that its contacting, isn't running or failed.
The event handler is meant to try to restart the dns service.

[benjaminsmith]

Hi @sinaowolabi,

Okay, that makes sense. Well, the event handler is getting called. I think this it's an issue with the passing of arguments (correct order?) or the script itself. You might want to put in some debugging code in the script to figure what's working and what's not. For example, write data out to a temporary log file as shown in the example guide.

[sinaowolabi]

Hi Benjamin

Yes its a problem with what the script is doing.
I got the script to log what its receiving and doing, and I see that the correct arguments are sent and with the correct sequence.
However the script is meant to use expect to open an ssh session with the target server and run another script, but the ssh sessions keep failing ('Connection closed by remote host' errors).
Running the script by hand (su - nagios -s /bin/bash) and the script works correctly.
So Ive taken to launching the event handling script by passing it to nrpe (check_nrpe!script.sh).
This seems to work but it presents a different problem, the eventhandling script seems to run even when the event is OK

Code: Select all

[1619348814] SERVICE EVENT HANDLER: serverX;Check X Service;OK;HARD;3;check_nrpe!restart_x

which I can see from the script's logging on the target.