Page 1 of 2
Event Handler Command not executing
Posted: Thu May 18, 2017 9:35 am
by socgenindia
Hi,
We have event handler enabled for one service , when service reaches the hard state nagios.log shows event handler has been evoked , but it doesnt execute the script inside the eventhandler.
/var/log/message
May 18 16:29:37 <nagios server> nagios: SERVICE EVENT HANDLER: <server name>;Lin_Process_cron;CRITICAL;HARD;1;SG_event_handler_cron
nagios.log
[1495117777] SERVICE EVENT HANDLER:<server name> ;Lin_Process_cron;CRITICAL;HARD;1;SG_event_handler_cron
command.cfg
SG_event_handler_cron /produits/nagiosXI/nagios/etc/script/cron.sh
sample script.
cat /produits/nagiosXI/nagios/etc/script/cron.sh
#!/bin/bash
echo "executing the script"
echo "Hi" > /produits/nagiosXI/nagios/etc/script/cron.txt
could you please help in this.
Re: Event Handler Command not executing
Posted: Thu May 18, 2017 1:17 pm
by tgriep
When a script it run by Nagios, it is run as the nagios user account so the script that you have created has to be able to be executed by the nagios user as well as the commands in the script and any folders the script needs to access.
In your example, verify that the /produits/nagiosXI/nagios/etc/script/cron.sh script can be run by the nagios user and that the folder - file /produits/nagiosXI/nagios/etc/script/cron.txt can be written by the nagios user.
To test this, login as root to the Nagios server and change to the nagios user account by running
Then try and run the script by running the following
Code: Select all
/produits/nagiosXI/nagios/etc/script/cron.sh
If it fails, verify that the file and folders are accessible be the nagios user and that should fix the issue for you.
Re: Event Handler Command not executing
Posted: Thu May 18, 2017 1:35 pm
by socgenindia
With nagios user i am able to run that script.
Re: Event Handler Command not executing
Posted: Thu May 18, 2017 3:41 pm
by dwhitfield
What is the output of ll /produits/nagiosXI/nagios/etc/script/cron.sh?
Also, can you PM me your Profile? You can download it by going to Admin > System Config > System Profile and click the ***Download Profile*** button towards the top. If for whatever reason you *cannot* download the profile, please put the output of View System Info (5.3.4+, Show Profile if older) in the thread (that will at least get us some info). This will give us access to many of the logs we would otherwise ask for individually. If security is a concern, you can unzip the profile take out what you like, and then zip it up again. We may end up needing something you remove, but we can ask for that specifically.
After you PM the profile, please update this thread. Updating this thread is the only way for it to show back up on our dashboard.
Re: Event Handler Command not executing
Posted: Fri May 19, 2017 9:17 am
by socgenindia
Hello,
I have PM the nagios profile to you.
when i run the script using the nagios user it just create the file cron.txt
echo "Hi" > /produits/nagiosXI/nagios/etc/script/cron.txt.
However we run the nagios with different user and with that user also i am able to run the script.
ocsp , notification are working fine , but not event handler.
where do we find the log what actually event handler is doing , i mean how it is executing the script.
I have enabled the debug option , but still unable to find the issue.
Re: Event Handler Command not executing
Posted: Fri May 19, 2017 11:20 am
by dwhitfield
In order to get good logs, you'll need to upgrade to 5.4.4...
OR you can modify your
/etc/cron.d/nagiosxi. Before I give you instructions for this, please note that you will either need to set up a log rotation policy, or you will need to change this back.
Replace your current
/etc/cron.d/nagiosxi with the following:
Code: Select all
# /etc/cron.d/nagiosxi: crontab fragment for nagiosxi
# Backup MySQL & PostgreSQL Databases
0 7 * * * root /root/scripts/automysqlbackup
0 7 * * * root /root/scripts/autopostgresqlbackup > /dev/null 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1
*/5 * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php >> /usr/local/nagiosxi/var/dbmaint.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1
01 * * * * nagios /usr/local/nagiosxi/cron/recurringdowntime.pl >> /usr/local/nagiosxi/var/recurringdowntime.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1
Of particular interest is
/usr/local/nagiosxi/var/event_handler.log but I am giving you all of them because often these logs go together when trying to piece together what is actually happening.
Does it work if you run nagios with the appropriate user?
What's the output of the following?
I mentioned this in the PM, but will mention it again. Can you PM me your Profile? You can download it by going to Admin > System Config > System Profile and click the ***Download Profile*** button towards the top. Since you are on an older version, I can tell you that it is NOT "Show Profile".
If you cannot download the profile, that is probably due to a modified sudoers. Instructions for resolving that are at
https://support.nagios.com/kb/article.p ... ategory=44
Re: Event Handler Command not executing
Posted: Mon May 22, 2017 8:36 am
by socgenindia
Hi,
Please find the command output
su nagios
umask
0066.
when i checked in eventman.log only global event handlers are processed , not local event handlers.
.....PROCESS EVENT: ID=8489033, SOURCE=2, TYPE=2, TIME=2017-05-22 15:23:15
*** GLOBAL HANDLER...
Array
(
[event_id] => 8489033
[event_source] => 2
[event_type] => 2
[event_time] => 2017-05-22 15:23:15
[event_meta] => Array
(
[notification-type] => service
[contact] => nagiosadmin
[contactemail] => nagios@localhost
[type] => PROBLEM
[escalated] => 0
[author] =>
[comments] =>
[host] => <server name>
[hostaddress] => <server IP>
[hostalias] => <server name>
[hostdisplayname] => <server name>
[service] => Lin_Process_cron
[hoststate] => UP
[hoststateid] => 0
[servicestate] => CRITICAL
[servicestateid] => 2
[lastservicestate] => CRITICAL
[lastservicestateid] => 2
[servicestatetype] => HARD
[currentattempt] => 1
[maxattempts] => 1
[serviceeventid] => 1730653
[serviceproblemid] => 794910
[serviceoutput] => PROCS CRITICAL: 1 process with command name crond
[longserviceoutput] =>
[datetime] => Mon May 22 15:23:15 CEST 2017
)
[logging_enabled] => 1
)
*** GLOBAL HANDLER (snmptrapsender)...
Re: Event Handler Command not executing
Posted: Mon May 22, 2017 1:35 pm
by tgriep
Can you post the full command definition for the Event Handler command as well as the service check that is using the Event Handler so we can check them for accuracy?
Then, login to the Nagios server in a shell, run the following and post the output.
Code: Select all
su nagios
ls -l /produits/nagiosXI/nagios/etc/script/cron.sh
bash -x /produits/nagiosXI/nagios/etc/script/cron.sh
Re: Event Handler Command not executing
Posted: Tue May 23, 2017 1:50 am
by socgenindia
Hi,
Please find the output
ls -l /produits/nagiosXI/nagios/etc/script/cron.sh
-rwxrwxr-x 1 apache monlc 94 May 22 12:44 /produits/nagiosXI/nagios/etc/script/cron.sh
bash -x /produits/nagiosXI/nagios/etc/script/cron.sh
+ echo 'executing the script'
executing the script
+ echo Hi
cat /produits/nagiosXI/nagios/etc/script/cron.sh
#!/bin/bash
echo "executing the script"
echo "Hi" > /usr/local/nagios/etc/script/cron.txt
In our setup nagios runs with mlcadm user ...
with mlcadm user when i run this below command .
bash-4.2$ ls -l /produits/nagiosXI/nagios/etc/script/cron.sh
-rwxrwxr-x 1 apache monlc 94 May 22 12:44 /produits/nagiosXI/nagios/etc/script/cron.sh
bash-4.2$ bash -x /produits/nagiosXI/nagios/etc/script/cron.sh
+ echo 'executing the script'
executing the script
+ echo Hi
bash-4.2$ id
uid=54966(mlcadm) gid=501(monlc) groups=501(monlc),48(apache),320(outils),504(nagcmd),69172(mysql),69332(postgresql)
bash-4.2$
bash-4.2$ id nagios
uid=18961(nagios) gid=501(monlc) groups=501(monlc),504(nagcmd),53702(nagios)
Note : - Event handlers are not working since 2 month , Earlier it worked.
Re: Event Handler Command not executing
Posted: Tue May 23, 2017 8:51 am
by tgriep
The testing that you did looks like the script ran without any errors as the mlcadm user.
It could be a configuration setting in Nagios XI and you can either open up a ticket by emailing to
[email protected] with a link to this post so we can continue on there.
If you go that route, email in your System Profile.
If you do not want to open up a ticket, can you PM me the System Profile?
To send us your system profile. Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and email it in.