System Monitoring OTRS/NAGIOS

[nathanplatt]

Bugger, i forgot to add that! Will go do it now

[rkennedy]

Sounds good. Glad to get it done now until no results are coming next week!

[nathanplatt]

Okay, so I got fed up waiting to see if this works;

So I've created a Linux box at home, set it to monitor my CPU usage. Purposely set the CPU to trigger at 40% and then at 50%, since I game a lot this should happen frequently. I'll let you know tonight if this works.

[hsmith]

Nice idea. Let us know what you come up with!

[nathanplatt]

Right, so i've triggered an alert. Nagios is showing as Critial, but no case in OTRS.

Doesn't look like our first attempt has worked. Can we look at any logs?

[nathanplatt]

This is what i found in the nagios.log

Code: Select all

nathan@ubuntu:/usr/local/nagios/var$ tail nagios.log
[1453322874] wproc:   early_timeout=0; exited_ok=1; wait_status=512; error_code=                            0;
[1453322874] wproc:   stderr line 01: execvp(/usr/local/nagios/libexec/eventhand                            lers/notify_otrs.sh, ...) failed. errno is 2: No such file or directory
[1453322874] wproc: SERVICE EVENTHANDLER job 8 from worker Core Worker 2889 is a                             non-check helper but exited with return code 2
[1453322874] wproc:   early_timeout=0; exited_ok=1; wait_status=512; error_code=                            0;
[1453322874] wproc:   stderr line 01: execvp(/usr/local/nagios/libexec/eventhand                            lers/host_notify_otrs.sh, ...) failed. errno is 2: No such file or directory
[1453322874] wproc: NOTIFY job 8 from worker Core Worker 2890 is a non-check hel                            per but exited with return code 127
[1453322874] wproc:   host=marvin; service=CPU Load; contact=nagiosadmin
[1453322874] wproc:   early_timeout=0; exited_ok=1; wait_status=32512; error_cod                            e=0;
[1453322874] wproc:   stderr line 01: /bin/sh: 1: /bin/mail: not found
[1453322874] wproc:   stderr line 02: /usr/bin/printf: write error: Broken pipe
nathan@ubuntu:/usr/local/nagios/var$

but i can see now that the eventhandler should be in the libexec folder, rather than the /usr/local/nagios/ dir

[rkennedy]

Try moving the files to the /usr/local/nagios/libexec/eventhandlers/ directory, and make sure to make they are executable as well.

As these bash scripts play a critical role, this could be the cause of it not working.

[nathanplatt]

Code: Select all

[1453325108] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;marvin;CPU Load;1453325107
[1453325108] SERVICE ALERT: marvin;CPU Load;CRITICAL;SOFT;1;CPU Load 97% (5 min average)
[1453325108] SERVICE EVENT HANDLER: marvin;CPU Load;CRITICAL;SOFT;1;host-notify-otrs
[1453325108] wproc: SERVICE EVENTHANDLER job 2 from worker Core Worker 3460 is a non-check helper but exited with return code 2
[1453325108] wproc:   early_timeout=0; exited_ok=1; wait_status=512; error_code=0;
[1453325108] wproc:   stderr line 01: execvp(/usr/local/nagios/libexec/eventhandlers/host_notify_otrs.sh, ...) failed. errno is 2: No such file or directory
nathan@ubuntu:/usr/local/nagios/var$ cd ..
nathan@ubuntu:/usr/local/nagios$ cd libexec/eventhandler/
nathan@ubuntu:/usr/local/nagios/libexec/eventhandler$ ls
host_notify_otrs.sh  host_otrs_event_ok.sh  notify_otrs.sh  otrs_event_ok.sh
nathan@ubuntu:/usr/local/nagios/libexec/eventhandler$

its in the right place and they are set as chmod -x but still it fails

[rkennedy]

What are the rest of the permissions on the folder?

Code: Select all

ls -l /usr/local/nagios/libexec/eventhandlers/

[nathanplatt]

Its okay, i just realised I made a rookie mistake, if you look at the last note, check the folder name... i missed the s off eventhandlers. Here's the new log;

Code: Select all

1453325867] wproc: NOTIFY job 2 from worker Core Worker 3632 is a non-check helper but exited with return $
[1453325867] wproc:   host=marvin; service=CPU Load; contact=nagiosadmin
[1453325867] wproc:   early_timeout=0; exited_ok=1; wait_status=32512; error_code=0;
[1453325867] wproc:   stderr line 01: /bin/sh: 1: /bin/mail: not found
[1453325867] wproc:   stderr line 02: /usr/bin/printf: write error: Broken pipe
[1453326194] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;marvin;CPU Load;1453326193
[1453326194] SERVICE ALERT: marvin;CPU Load;CRITICAL;SOFT;1;CPU Load 97% (5 min average)
[1453326194] SERVICE EVENT HANDLER: marvin;CPU Load;CRITICAL;SOFT;1;host-notify-otrs

So this looks like its worked, but OTRS isn't creating the case!