Capturing Multiline Output?

[mmartin]

Hello All,

Nagios Core 3.2.3

I'm trying to use the $LONGSERVICEOUTPUT$ Macro, which as I understand it, it holds everything after the first line by escaping
all "\n" newline chars and putting all the subsequent lines (*after the 1st line) onto a single line and stores it in $LONGSERVICEOUTPUT$ Macro.

But when I receive the Service Notification that contains the LONGSERVICEOUTPUT Macro it is blank....
I'm sending the command to Nagios using NSCA.

So if I run this command from the remote server:

Code: Select all

echo "Host_name1,Check_Service myName,2,CRITICAL: This is a Test Message for Nagios\nThis is Line 2\nThis is Line 3\nLines 1-2-3 should be in the LONGSERVICEOUTPUT Macro" | send_nsca -H 192.168.111.1 -p 5667 -d , -c /usr/local/nagios/etc/send_nsca.cfg

Nagios receives the command and shows this in /var/log/messages:

Code: Select all

Jan 11 17:31:14 Localhost nagios: EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;Host_name1;Check_Service myName;2;CRITICAL: This is a Test Message for Nagios
Jan 11 17:31:14 Localhost nagios: EXTERNAL COMMAND: PROCESS_HOST_CHECK_RESULT;Lines 1;2;3 should be in the LONGSERVICEOUTPUT Macro
Jan 11 17:31:14 Localhost nagios: Warning:  Passive check result was received for host 'Lines 1', but the host could not be found!
Jan 11 17:31:20 Localhost nagios: PASSIVE SERVICE CHECK: Host_name1;Check_Service myName;2;CRITICAL: This is a Test Message for Nagios
Jan 11 17:31:20 Localhost nagios: SERVICE ALERT: Host_name1;Check_Service myName;CRITICAL;HARD;1;CRITICAL: This is a Test Message for Nagios

Not sure why the LONGSERVICEOUTPUT is being read by Nagios as a new Host/Service check...??

I created 2 scripts that simply take in a few variables (which will be the Macros I want to use) and write their values to a file.
Here are the Commands Config:
These get called because they are defined in my Contact's Config which is apart of the Main Service Definition for this Service in question.
So the service gets run, it then checks my contact which contains the 2 commands below. Then after they run it DOES write to the file but
some of the MACROS are NOT getting any data... Mostly just want LONGSERVICEOUTPUT

Code: Select all

define command {
	command_name	get-host-macros
	command_line	$USER1$/prelude_logSize_check/TEST_MACROS/write_hostMacros "$HOSTNAME$" "$HOSTDISPLAYNAME$" "$HOSTALIAS$" "$HOSTADDRESS$" "$HOSTOUTPUT$" "$HOSTNOTES$"
}

define command {
	command_name	get-service-macros
	command_line	$USER1$/prelude_logSize_check/TEST_MACROS/write_serviceMacros "$SERVICEDESC$" "$SERVICEDISPLAYNAME$" "$SERVICESTATE$" "$SERVICESTATEID$" "$SERVICEDURATION$" "$LONGSERVICEOUTPUT$" "$SERVICEPERFDATA$" "$SERVICENOTES$"
}

If I'm not making sense let me know and I'll try and explain a little better...
Any thoughts would be MUCH appreciated!!

Thanks in Advance,
Matt

[slansing]

First of all how many characters are being returned in your service output as a whole? If it does not reach the maximum number of characters and you use the longserviceoutput method your notifications will return empty as there is simply no information following the first line. Can you check the output and let us know how many characters are being sent? One way to do this would be to drop the output in a editor such as Notepad++ or Geany and check from there.

[mmartin]

Hey slansing, thanks for your reply!

Sorry, I haven't gotten back to this sooner, but I didn't know that I did not have a subscription setup for this post so I didn't know anyone responded...

Anyway... So I was sending it a test Message just to see what I'd get with multiple lines.

This is one of the test Strings I was sending to Nagios using NSCA:

Code: Select all

"CRITICAL: TEST... This Service is Critical\n This is Line 2\nThis is line 3"

So the above Output is a TOTAL of 75 Characters.
The "Notification" command sends some information which includes a few Macros, including $SERVICEOUTPUT$, $LASTSERVICECHECK$,
and &LONGSERVICEOUTPUT$ Macros just to name a few.

Here is the notification received by me:

Code: Select all

	 		 ****** Nagios Core 3.2.3 ******

Notification Type: PROBLEM

		 Service: Test_Service Description
		 Host: localhost_name1
		 Address: 10.x.x.x
		 State: CRITICAL

Date/Time: Fri Jan 18 11:56:37 EST 2013

Service Message:
		 CRITICAL: TEST... This Service is Critical

Long Service Output:
		 ""

As you can see above, the Last section of the email body contains the $LONGSERVICEOUTPUT$ Macro, but the spot is empty...
The data should be within those empty double-quotes...

Sorry, I'm not sure what you meant exactly with your first sentence... So are you saying if my output doesn't hit the maximum number of characters then there
wouldn't be any information in the $LONGSERVICEOUTPUT$ Macro...?

Thanks Again for your reply!

Thanks,
Matt

[slansing]

Well, you can place the regular service output and long output together and this will show you the first line and the remaining ones you created.

As you can see above, the Last section of the email body contains the $LONGSERVICEOUTPUT$ Macro, but the spot is empty...
The data should be within those empty double-quotes...

However, I can not see the notification definition so I do not know what it looks like unfortunately, can you please drop that definition in here?

[mmartin]

Hey slansing, thanks for the reply...

Well, you can place the regular service output and long output together and this will show you the first line and the remaining ones you created.

Do you mean combining them into, say one line of output with "\n" escaped..?

However, I can not see the notification definition so I do not know what it looks like unfortunately, can you please drop that definition in here?

Sure... I created a new "notify-by-email" command, it's basically the same thing except that I added the $LONSERVICEOUTPUT$ Macro to it.
And I also created a little Bash script that just takes a bunch of arguments and prints them to a file. This way I can see exactly what was being saved inside
the Macros.

commands.cfg

Code: Select all

# Notification Command Definition:
define command{
	command_name	notify-by-email-2
	command_line	/usr/bin/printf "%b" "\n\t\t****** Nagios Core 3.2.3 ******\n\nNotification Type: $NOTIFICATIONTYPE$\n\n\tService: $SERVICEDESC$\n\tHost: $HOSTDISPLAYNAME$\n\tAddress: $HOSTADDRESS$\n\tState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nService Message:\n\t$SERVICEOUTPUT$\n\nLong Service Output:\n\t\"$LONGSERVICEOUTPUT$\"\n\nPerformance Data:\n\t$SERVICEPERFDATA$\n\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Alert - $HOSTDISPLAYNAME$ --- $SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$
}

# Script which also gets called at the same time the Notification Command gets executed...
define command {
	command_name	get-service-macros
	command_line	$USER1$/write_serviceMacros "$SERVICEDESC$" "$SERVICEDISPLAYNAME$" "$SERVICESTATE$" "$SERVICESTATEID$" "$SERVICEDURATION$" "$LONGSERVICEOUTPUT$" "$SERVICEPERFDATA$" "$SERVICENOTES$"
}

# Contact Definition --- Which calls the two commands above:
define contact{
	contact_name			Contact_TEST2
	alias				Test_email
	host_notification_period	24x7
	service_notification_period	24x7
	service_notification_options	w,u,c,r
	host_notification_options	d,u,r
	service_notification_commands	notify-by-email-2, get-service-macros
	host_notification_commands	host-notify-by-email
	email				[email protected]
}

Bash Script --> write_serviceMacros

Code: Select all

#!/bin/bash

#########################################
#     --- NAGIOS SERVICE MACROS ---	
#					
#	1 - $SERVICEDESC$		
#	2 - $SERVICEDISPLAYNAME$	
#	3 - $SERVICESTATE$		
#	4 - $SERVICESTATEID$		
#	5 - $SERVICEDURATION$		
#	6 - $LONGSERVICEOUTPUT$		
#	7 - $SERVICEPERFDATA$		
#	8 - $SERVICENOTES$		
#					
#########################################

SERVICE_DESC="$1"
SERVICE_DISPLAY_NAME="$2"
SERVICE_STATE="$3"
SERVICE_STATE_ID="$4"
SERVICE_DURATION="$5"
LONG_SERVICE_OUTPUT="$6"
SERVICE_PERF_DATA="$7"
SERVICE_NOTES="$8"

LOG="/usr/local/nagios/libexec/TEST_MACROS/service_macros.txt"
DATE=$(date "+%b %d, %Y at %r")

echo -ne "LISTING SERVICE MACROS: (*$DATE)\n" >> $LOG
echo -ne "\tSERVICE_DESC = $SERVICE_DESC\n" >> $LOG
echo -ne "\tSERVICE_DISPLAY_NAME = $SERVICE_DISPLAY_NAME\n" >> $LOG
echo -ne "\tSERVICE_STATE = $SERVICE_STATE\n" >> $LOG
echo -ne "\tSERVICE_STATE_ID = $SERVICE_STATE_ID\n" >> $LOG
echo -ne "\tSERVICE_DURATION = $SERVICE_DURATION\n" >> $LOG
echo -ne "\tLONG_SERVICE_OUTPUT = $LONG_SERVICE_OUTPUT\n" >> $LOG
echo -ne "\tSERVICE_PERF_DATA = $SERVICE_PERF_DATA\n" >> $LOG
echo -ne "\tSERVICE_NOTES = $SERVICE_NOTES\n" >> $LOG
echo -e "\n\n\n" >> $LOG

exit 0

Thanks Again for the reply!!


Thanks,
Matt

[slansing]

Can you show the output of the NSCA command you are using to send the test to your nagios system? If you are not sending it as a normal string but instead say..echoing it with a echo -e then the newline characters will be literally breaking it into new lines which will not create new lines within the status text.

[mmartin]

Hey slansing, thanks again for your quick reply!

Humm.... You may be on to something there.
I was literally about to hit shutdown when I saw your response, so first thing tomorrow I'll grab that "echo" command I'm using.

Though now that I think about it I also tried creating a txt file with 3 lines of text... Literally 3 lines of the files have actual characters
on them, and it still did the same.... I think...

But I'll post back first thing tomorrow with that. Thanks again, very much appreciated!


Thanks AGAIN,
Matt

[slansing]

No problem, have a great night and let me know whats up tomorrow.

[mmartin]

Hey slansing, sorry yesterday was pretty hectic and never got back around to this then...
So, below is a couple different ways I've tried. All using the send_nsca from a remote server...

B.T.W. This is being run from an AIX server, which for their 'echo' command, it does not allow ANY options with it at all.
It's just echo [string]...

With an "echo" Command:
I'm assuming, like you had said, which makes perfect sense, that this won't work because it takes the "\n" characters literally. So
it's breaking into newlines before it even gets to Nagios... Does that sound right??

Code: Select all

## 1ST ECHO COMMAND TRY:

> echo "hostname,Test_Service Description,2,CRITICAL: This is a test\nThis is line 2\nThis is line 3\n" | send_nsca -H 192.168.x.xxx -d 5667 -to 10 -d , -c /usr/local/nagios/etc/send_nsca.cfg

## 2ND ECHO COMMAND TRY: i.e. the "\n" is now "\\n"
#  After just testing this one without piping to send_nsca I noticed it still breaks into newlines even though I'm trying
#  to escape it. Thanks to AIX's generic echo command...

> echo "hostname,Test_Service Description,2,CRITICAL: This is a test\\nThis is line 2\\nThis is line 3\\n" | send_nsca -H 192.168.x.xxx -d 5667 -to 10 -d , -c /usr/local/nagios/etc/send_nsca.cfg

So both of the echo commands above split the string into newlines before it pipes to send_nsca I think..?

I also tried entering the "output" in a file and then using "cat" and then piping, but still same result. Which I assume is because it is
basically doing the same thing as the above echo commands.

Code: Select all

# The File's contents:
> cat test_outputFile.txt
hostname,Test_Service Description,2,CRITICAL: This is a test
This is line 2
This is line 3

> cat test_outputFile.txt | send_nsca -H 192.168.x.xxx -d 5667 -to 10 -d , -c /usr/local/nagios/etc/send_nsca.cfg

And I get the same result with that 'cat' command as well...

After you were telling me about the echo command, it got me thinking, maybe I'll try with the "printf" command, and if that doesn't work
I don't think I've tried putting the "string" into a file and instead of literally putting each 'line' on it's own line I'll try just inserting a "\n"
instead... And maybe that will work.


Anyway, I'll post back after trying those I just mentioned, and thanks again for your replies...

Thanks Again,
Matt

[mmartin]

Hey...

So I just tried putting into a file and using "\n" instead of literally putting each thing on a new line and it worked...!!!
I'm going to experiment a little more and give the printf command a try and see if that will work too...

Here is the File Contents of the one that worked:

Code: Select all

hostname,Test_Service: Description,2,CRITICAL: THIS IS A NEW TEST (Line 1)\nThis should be on line 2\nThis should be line 3\nHere's line 4

And here is the Notification Email I got:

Code: Select all

		 		 ****** Nagios Core 3.2.3 ******

Notification Type: PROBLEM

		 Service: Test_Service Description
		 Host: hostname
		 Address: 10.xx.xx.xxx
		 State: CRITICAL

Date/Time: Thu Jan 24 11:01:01 EST 2013

Service Message:
		 CRITICAL: THIS IS A NEW TEST (Line 1)

Long Service Output:
"This should be on line 2
This should be line 3
Heres line 4"

Performance Data:
		 

So it looks like I got it working..!!!
I'm still going to try the printf way because I'd rather do that then output into a file.

Thanks again slansing for you guidance on this... I had been trying to get this working for weeks... Awesome!

Thanks AGAIN,
Matt