Event Handler Not Working

[swalker76]

Hi -
I'm trying to create an event handler that runs a restart script when Nagios XI finds the web server in a CRITICAL state. I'm new to this so I found documentation and set up the handler based on what I read. When the HTTP service goes down the handler does not run. I can't seem to figure out what I have missed.

Here's a little more information:

Running Nagios XI 5.4.7.

I created a service monitor to watch the HTTP service attached to a domain. Has been working perfectly. I also created an event handler that should fire off a script to restart the ColdFusion process that causes the HTTP service to go CRITICAL. This handler has been added to the HTTP Service Monitor.

If I take the script the event handler is calling and run it on the command line, with CRITICAL included, it connects to the remote server and restarts the ColdFusion process just fine. If I shut the Cold Fusion service down on the server(which causes the HTTP service to go CRITICAL) and force Nagios XI to do immediate checks the event handler does not run.

Can anyone help me down the path to get this to work or is this something I should contact Support on?

Thanks in advance -
Steve

[cdienger]

In addition to the drop down to select the event handler under the service's Check Settings tab, there is an Event handler enabled" option. Make sure that that is also set to On.

Other than that, it sounds like a possible permissions issue. When you run the script from the command line you're probably logged in as root, but when the script is triggered by XI it would be running with the apache/nagios account. This account may not have permissions to execute the scrip or commands in the script. check and try changing the permissions to execute for everybody as a test:

Code: Select all

ll scriptname

and

Code: Select all

chmod a+x scriptname
chown apache:nagios scriptname

[swalker76]

Thanks for the information.

It may be permissions related. I just tried to run the script as the Nagios user and it failed. I'll make some changes and update this once I can retry.

Thanks!!

[swalker76]

I changed the permissions for the Nagios user so they can run the script on the remote machine. I also changed the permissions so the user can connect to that machine via ssh. Retested the script for the Nagios XI server as the Nagios user and the script worked as expected.

I then stopped the Cold Fusion service to put HTTP into a CRITICAL state. I forced immediate checks through Nagios XI until the threshold was met (5) and also waited 5 minutes. Event handler did not execute.

Steve

[tgriep]

In the service check, do you have the setting to enable the Event Handler?
To check this, go to the Core Config Manager and open the service. Click on the Check Settings tab and verify that the service has the event handler configures and enabled.
If you still have the issue, can you post the configuration for the service and the event handler so we can check the settings?

[swalker76]

In the service check, do you have the setting to enable the Event Handler?
To check this, go to the Core Config Manager and open the service. Click on the Check Settings tab and verify that the service has the event handler configures and enabled.
If you still have the issue, can you post the configuration for the service and the event handler so we can check the settings?

Sorry forgot to mention in the last response that the event handler is enabled.

The HTTP service looks like this:
define service {
host_name goldrush.coalliance.org
service_description HTTP
use xiwizard_website_http_service
check_command check_xi_service_http! -f ok -I 97.107.76.30 -u "/" -p 80!!!!!!!
initial_state o
max_check_attempts 5
check_interval 5
retry_interval 1
active_checks_enabled 1
check_period xi_timeperiod_24x7
event_handler restart_GRWEB_coldfusion_handler
event_handler_enabled 1
notification_interval 60
notification_period xi_timeperiod_24x7
contacts chet,rose,steve
_xiwizard website
register 1
}

The Event Handler command looks like this:

define command {
command_name restart_GRWEB_coldfusion_handler
command_line $USER1$/restart-grweb-coldfusion.sh $SERVICESTATE$ $SERVICESTATETYPE$ $SERVICEATTEMPT$
}

Steve

[ssax]

It's usually the event handler script that has a logic issue in it, please attach it so that we can review it.

Please add this to the very top (below the #!/bin/bash line):

Code: Select all

echo "`date +%s%3N` - Made it" >> /tmp/EVENT_HANDLER_OUTPUT.txt

Then force a check or force it to execute and you should see it put new entries in /tmp/EVENT_HANDLER_OUTPUT.txt, you can watch it with (force a check first so the file gets created):

Code: Select all

tail -f /tmp/EVENT_HANDLER_OUTPUT.txt

Thank you

[swalker76]

It's usually the event handler script that has a logic issue in it, please attach it so that we can review it.

Please add this to the very top (below the #!/bin/bash line):

Code: Select all

echo "`date +%s%3N` - Made it" >> /tmp/EVENT_HANDLER_OUTPUT.txt

Then force a check or force it to execute and you should see it put new entries in /tmp/EVENT_HANDLER_OUTPUT.txt, you can watch it with (force a check first so the file gets created):

Code: Select all

tail -f /tmp/EVENT_HANDLER_OUTPUT.txt

Thank you

I don't know what happened but it appears to be working now. I added the code line to create the output file and then forced the checks and the event handler worked.

Thanks for your help!!

[bolson]

May we go ahead and close this topic?