Page 1 of 2
Error with custom service macros and event handling.
Posted: Mon Mar 10, 2014 4:01 pm
by j1johnson
Hello!
I'm working on implementing an event handler that would create a trouble ticket with an external ticketing system when a service was in a CRITICAL state. Because of how the ticket system works, it needs a lot of variables and I've found that the custom _SERVICE macros can fill that role nicely. My problem is that I'm trying to pass a specific number of arguments (16) to an event handler and only a certain amount of data is actually getting through.
Here is the service:
Code: Select all
define service{
use prod-java ; Name of service template to use
host_name WorkQueueDelta-PFIX
service_description DeltaSessionManager: SW114
check_command check_nrpe_long!check_sw114
check_interval 3
_SERVICE restart_sw114
_CLASSIFICATIONID SC_WORKFLOW_PORTAL
_HIERARCHYPATH '"SC_APP_SYS \\\ SC_WORKFLOW_PORTAL"'
_CLASSSTRUCTUREID 5642
_DESCRIPTION '"sw114 DeltaSession Manager has stopped running."'
_FR1CODE '"Unknown - code issue."'
_FR2CODE '"Restart standalone java process."'
_PROBLEMCODE '"Standalone sw114 process terminated on it's own."'
_IMPACT 3
_INTERNALPRIORITY 4
_REPORTEDPRIORITY 3
_URGENCY 3
_OWNER AR333
_OWNERGROUP WASTEAM
retry_interval .5
max_check_attempts 2
event_handler restart-service
notifications_enabled 1
}
Here is the command.cfg entry:
Code: Select all
define command {
command_name restart-service
command_line $USER1$/restartservice.sh $SERVICESTATE$ $HOSTADDRESS$ $_SERVICECLASSIFICATIONID$ $_SERVICEHIERARCHYPATH$ $_SERVICECLASSSTRUCTUREID$ $_SERVICEDESCRIPTION$ $_SERVICEFR1CODE$ $_SERVICEFR2CODE$ $_SERVICEPROBLEMCODE$ $_SERVICEIMPACT$ $_SERVICEINTERNALPRIORITY$ $_SERVICEREPORTEDPRIORITY$ $_SERVICEURGENCY$ $_SERVICEOWNER$ $_SERVICEOWNERGROUP$ $SERVICEPROBLEMID$
}
Now here is the restartservice.sh script and a quick explanation. All I want do to at this moment is get a listing of the variables sent to me so that I can ensure that they are formatted correctly and look right before I actually try submitting the data to the ticketing system script:
Code: Select all
#!/bin/bash
#
case "$1" in
OK)
;;
WARNING)
;;
UNKNOWN)
;;
CRITICAL)
echo "$1" "$2" "$3" "$4" "$5" "$6" "$7" "$8" "$9" "$10" "$11" "$12" "$13" "$14" "$15" "$16"| mail -s "Variables going to perl" [email protected]
;;
esac
exit 0
Here is the email I get:
CRITICAL 10.14.172.238 SC_WORKFLOW_PORTAL SC_APP_SYS \ SC_WORKFLOW_PORTAL 5642 sw114 DeltaSession Manager
Here is the email I should get:
CRITICAL 10.14.172.238 SC_WORKFLOW_PORTAL SC_APP_SYS \ SC_WORKFLOW_PORTAL 5642 sw114 DeltaSession Manager has stopped running. Unknown - code issue. Restart standalone java process. Standalone sw114 process terminated on it's own. 3 4 3 3 AR333 WASTEAM *SERVICEPROBLEMID*
What am I doing wrong? Is there a limit that I'm hitting? Am I doing something super stupid that is obvious and I can't see? I've seen no documentation on the number of custom macros that I can use or anything that would explain what I'm doing wrong. I've looked through macros.h, limits.h, nagios.h, etc and I got nothin. I have tried a few different ways of getting the ARGV variables and processing / displaying them and I've hit a wall.
Any insights would be hugely appreciated - thanks!
- Jim
Re: Error with custom service macros and event handling.
Posted: Mon Mar 10, 2014 4:43 pm
by sreinhardt
So far I don't see anything actually wrong, have you tried outputting to a flat file and seeing if that gets all your stuff?
Re: Error with custom service macros and event handling.
Posted: Mon Mar 10, 2014 7:31 pm
by j1johnson
First attempt of this command:
Code: Select all
echo "$1" "$2" "$3" "$4" "$5" "$6" "$7" "$8" "$9" "$10" "$11" "$12" "$13" "$14" "$15" "$16" > /tmp/var.txt
Returned this:
CRITICAL 10.14.172.238 SC_WORKFLOW_PORTAL SC_APP_SYS \ SC_WORKFLOW_PORTAL 5642 sw114 DeltaSession Manager CRITICAL0 CRITICAL1 CRITICAL2 CRITICAL3 CRITICAL4 CRITICAL5 CRITICAL6
Then I tried this:
Code: Select all
echo "$1" "$2" "$3" "$4" "$5" "$6" "$7" "$8" "$9" "${10}" "${11}" "${12}" "${13}" "${14}" "${15}" "${16}" > /tmp/vars.txt
Which returned this:
CRITICAL 10.14.172.238 SC_WORKFLOW_PORTAL SC_APP_SYS \ SC_WORKFLOW_PORTAL 5642 sw114 DeltaSession Manager has stopped running. Unknown - code issue.
So that got a little farther but still well short of what I'm looking for ... thoughts on next steps?
Thanks!
- Jim
Re: Error with custom service macros and event handling.
Posted: Tue Mar 11, 2014 11:17 am
by sreinhardt
How many total arguments are you getting if you echo $# at the beginning? This should provide the total number of arguments passed to bash.
Re: Error with custom service macros and event handling.
Posted: Tue Mar 11, 2014 2:09 pm
by j1johnson
Here's the command
Here's the response:
34
Re: Error with custom service macros and event handling.
Posted: Tue Mar 11, 2014 2:36 pm
by j1johnson
When I make changes to how the custom macros are escaped, I can get a few more pieces to show up:
New Custom services:
Code: Select all
_CLASSIFICATIONID SC_WORKFLOW_PORTAL
_HIERARCHYPATH \""SC_APP_SYS \\\ SC_WORKFLOW_PORTAL\""
_CLASSSTRUCTUREID 5642
_DESCRIPTION \""sw114 DeltaSession Manager has stopped running.\""
_FR1CODE \""Unknown - code issue.\""
_FR2CODE \""Restart standalone java process.\""
_PROBLEMCODE \""Standalone sw114 process terminated on it's own.\""
_IMPACT 3
_INTERNALPRIORITY 4
_REPORTEDPRIORITY 3
_URGENCY 3
_OWNER AR333
_OWNERGROUP WASTEAM
Now I get this returned:
29 (Number of variables being passed to bash)
CRITICAL 10.14.172.238 SC_WORKFLOW_PORTAL SC_APP_SYS \ SC_WORKFLOW_PORTAL 5642 sw114 DeltaSession Manager has stopped running. Unknown - code issue. Restart standalone java (7 variables for sure passed through - the 8th is partially shown)
I was hoping to get the quote marks to pass through but I haven't been able to figure that out. Maybe this is all about how I'm escaping characters?
Re: Error with custom service macros and event handling.
Posted: Tue Mar 11, 2014 2:39 pm
by tmcdonald
I think escaping is an issue here. Where you have "\\\" that is going to create one literal backslash, then try to escape whatever comes after.
Also you might want to try escaping the inner quotes, or maybe turning the outer into single quotes.
Re: Error with custom service macros and event handling.
Posted: Tue Mar 11, 2014 4:08 pm
by j1johnson
So I tried a lot of different ways to escape it and failed courageously over and over.
However, these settings worked:
Code: Select all
_CLASSIFICATIONID SC_WORKFLOW_PORTAL
_HIERARCHYPATH SC_APP_SYS-SC_WORKFLOW_PORTAL
_CLASSSTRUCTUREID 5642
_DESCRIPTION Description_Specifics
_FR1CODE FR1CODE_Specifics
_FR2CODE FR2CODE_Specifics
_PROBLEMCODE PROBLEMCODE_Specifics
_IMPACT 3
_INTERNALPRIORITY 4
_REPORTEDPRIORITY 3
_URGENCY 3
_OWNER AR333
_OWNERGROUP WASTEAM
And resulted in this output:
16 (Number of variables being passed to bash)
CRITICAL 10.14.172.238 SC_WORKFLOW_PORTAL SC_APP_SYS-SC_WORKFLOW_PORTAL 5642 Description_Specifics FR1CODE_Specifics FR2CODE_Specifics PROBLEMCODE_Specifics 3 4 3 3 AR333 WASTEAM 176274
So it appears to be an issue with my custom macro definitions. Putting space separated values causes me hate and discontent and all the documentation I've found on escaping characters with Nagios centers around \ and $. There isn't a whole lot out there that talks about what I'm trying to do.
Is this a bug or is this expected behavior? If it's Working As Designed™, I'm open to suggestions ... I really want this to work the way I need it to ....
Thanks!
- Jim
Re: Error with custom service macros and event handling.
Posted: Wed Mar 12, 2014 12:01 pm
by slansing
What ticketing system are you trying to integrate these alerts with? I was going to do some hunting around for a guide, which I'm sure you already did. Have you tried working in actual newlines, or are you sticking to hoping for a way to get commas in? We could probably figure out a notification handler based on one of the email notification handlers that would do this properly, since that is what most users who have integrated with ticketing systems have done in the past.
Re: Error with custom service macros and event handling.
Posted: Wed Mar 12, 2014 12:21 pm
by j1johnson
I'm trying to integrate with
Maximo. There are built-in SOAP and REST services and I'm using their SOAP WSDL to create, update and then close Incident tickets that Nagios handles automatically through event handlers. If I were just opening tickets, it wouldn't be problem. But because I'm also closing tickets, I need to provide extra data to Maximo which is where all the custom service macros come from. I hadn't thought of using newlines and I'm not sure how they would work as part of the variable. The piece of the script that I haven't been showing is the perl script that sucks in all the variables I'm sending and then creates the properly formatted XML payload and submits it to the webservice. Ideally I would like all of my arguments passed to the perl script in quotes so that there's no confusion in the parsing. I like the idea of basing this off of the email notification handlers - I hadn't even thought about that. I'll have to mess around and see if I can make that work. Suggestions are always welcomed
