Page 1 of 3

Alert info being truncated by Hipchat notfications.

Posted: Thu Aug 20, 2015 11:39 am
by mlbsysops
Running into the following issue.

On the Nagios Core page the alert is showing as
Status Information: Critical Cert "cert_1" will expire in 0 days
Critical Cert "cert_2" will expire in 0 days
However when the notification comes through via hipchat it truncates the second value:
Thu Aug 20 16:30:57 GMT 2015 - check_certs_ns_PROD on SOME_HOST (nagios@nagios01)
Type: PROBLEM
Host: SOME_HOST (SOME_IP)
State: CRITICAL
Info:
Critical Cert "cert_1" will expire in 0 days
Info:
Critical Cert "cert_1" will expire in 0 days
Here is the template for the notification:

Code: Select all

"""
Templates used to build hipchat api payloads
"""

host_template = """
<strong>{timestamp} - {hostname}  (nagios@{nagios_host})</strong><br/>
<strong>Type:</strong> {ntype}<br/>
<strong>Host:</strong> {hostname} (<a href="{hostaddress}">{hostaddress}</a>)<br/>
<strong>State:</strong> {state}<br>
<strong>Info:</strong>
<pre>{hostoutput}</pre>
"""

host_medium_template = "<strong>{timestamp} - {hostname} ({hostaddress}) - {ntype}/{state}</strong><br/><pre>{hostoutput}</pre>"
host_short_template = """[{ntype}] {hostname}: {hostoutput}"""

service_template = """
<strong>{timestamp} - {servicedesc} on {hostalias} (nagios@{nagios_host})</strong><br/>
<strong>Type:</strong> {ntype}<br/>
<strong>Host:</strong> {hostalias} (<a href="{hostaddress}">{hostaddress}</a>)<br/>
<strong>State:</strong> {state}<br/>
<strong>Info:</strong>
<pre>{serviceoutput}</pre>
<strong>Info:</strong>
<pre>{serviceoutput}</pre>
"""

service_medium_template = "<strong>{timestamp} - {servicedesc} on {hostalias} ({hostaddress}) - {ntype}/{state}</strong><br/><pre>{serviceoutput}</pre>"
service_short_template = "[{ntype}] {hostalias} {servicedesc}: {serviceoutput}"


templates = {'host': host_template, 'medium-host': host_medium_template, 'short-host': host_short_template,
             'service': service_template, 'medium-service': service_medium_template, 'short-service': service_short_template}
My question is, is <pre>{serviceoutput}</pre> supposed to be populated with all the info or are there other variables that need to be added like <pre>{serviceoutput_2}</pre> for example.

Re: Alert info being truncated by Hipchat notfications.

Posted: Thu Aug 20, 2015 1:27 pm
by jdalrymple
Hopefully I'm not supposed to know about any magic hipchat integration, because I don't. I definitely see a number of online methods, but none I've worked with.

That said - I'm not sure how the data is getting passed into your template, but you should note the differences between the macros $SERVICEOUTPUT$ and $LONGSERVICEOUTPUT$:
Standard Macros in Nagios wrote:$SERVICEOUTPUT$ The first line of text output from the last service check (i.e. "Ping OK").
Standard Macros in Nagios wrote:$LONGSERVICEOUTPUT$ The full text output (aside from the first line) from the last service check.

Re: Alert info being truncated by Hipchat notfications.

Posted: Fri Aug 21, 2015 9:17 am
by mlbsysops
Cool thanks I'll try that out, I'm thinking it's going to be the solution.

Re: Alert info being truncated by Hipchat notfications.

Posted: Fri Aug 21, 2015 9:27 am
by tmcdonald
Let us know how that works out for you!

Please also keep in mind that if this turns out to be something that must be solved on the HipChat side, we will have to ask that their support handle it from that point on. We can certainly cover the Nagios side of things, but once it moves out of our realm we can't guarantee we will have a fix or the knowledge of the other product required to find one.

Re: Alert info being truncated by Hipchat notfications.

Posted: Fri Aug 21, 2015 12:47 pm
by mlbsysops
Thanks but after further digging I've found that it seems to be Nagio XI who is responsible for truncating (I've added email notifications with the same result). I'm going to open a ticket with the XI guys and have them take a look, but I'll post the question and answer here as well for clarity's sake.

This is what the alert looks like on the Nagios Core HTTP page:

Code: Select all

Current Status:	
  CRITICAL  
 (for 1d 1h 42m 30s)
Status Information:	Critical Cert “Cert_01.crt" will expire in 0 days
                      Critical Cert “Cert_02.crt" will expire in 0 days
Performance Data:	
Current Attempt:	5/5  (HARD state)
Last Check Time:	08-21-2015 17:16:16
Check Type:	ACTIVE
Check Latency / Duration:	0.000 / 0.231 seconds
Next Scheduled Check:  	08-21-2015 17:21:16
Last State Change:	08-20-2015 15:37:19
Last Notification:	08-21-2015 17:16:17 (notification 21)
Is This Service Flapping?	
  NO  
 (0.00% state change)
In Scheduled Downtime?	
  NO  
Last Update:	08-21-2015 17:19:43  ( 0d 0h 0m 6s ago)
However when you look at this alert in Nagios XI’s Service Detail page you see the following:

Code: Select all


Critical Critical Cert “Cert_01.crt" will expire in 0 days

Critical Cert "Cert_02.crt" will expire in 0 days <-– this seems to be displayed as a comment and so is getting stripped out when sent as a notification. 


Status Details
Service State:	Critical
Duration:	21h 46m 49s
Service Stability:	Unchanging (stable)
Last Check:	2015-08-21 17:21:16
Next Check:	2015-08-21 17:26:16
Quick Actions
Acknowledge this problem
Acknowledge this problem
Disable notifications
Disable notifications
Schedule a forced immediate check
Schedule a forced immediate check
 

Re: Alert info being truncated by Hipchat notfications.

Posted: Fri Aug 21, 2015 12:57 pm
by tmcdonald
mlbsysops wrote:I'm going to open a ticket with the XI guys and have them take a look
We are the XI guys :)

By a ticket do you mean you are going to use email support? We can always move this thread into the XI forum and continue from there.

You can always check out this thread for information on increasing the DB table size to combat the truncation:

https://support.nagios.com/forum/viewto ... 16&t=27918

Re: Alert info being truncated by Hipchat notfications.

Posted: Fri Aug 21, 2015 1:11 pm
by mlbsysops
Wasn't sure of the team makeup, I'll check out the link. You can close the ticket I opened and we can continue in here.

Thanks!

Re: Alert info being truncated by Hipchat notfications.

Posted: Fri Aug 21, 2015 1:22 pm
by tmcdonald
mlbsysops wrote:Wasn't sure of the team makeup
Anyone with a green name is a Nagios employee, and we all cover all the boards. For reference, this page shows all of our staff.

Let us know how that DB increase turns out!

Re: Alert info being truncated by Hipchat notfications.

Posted: Fri Aug 21, 2015 2:03 pm
by mlbsysops
No dice,

When looking at the service detail page I see the following for that service.

Code: Select all

<HOST FQDN>            check_certs_ns_PROD	Critical 	23h 20m 3s 	5/5 	2015-08-21 18:53:05 	Critical Cert "Cert_01.crt" will expire in 0 days
It seems that the character limit increase didn't help in this instance as Cert_02.crt is still getting truncated.

Re: Alert info being truncated by Hipchat notfications.

Posted: Fri Aug 21, 2015 2:11 pm
by tmcdonald
What XI version are you on?