Page 1 of 1
Nagios Acknowledge cmd.cgi takes more time
Posted: Tue Jul 25, 2017 8:20 am
by Pradeeps
Hello Folks,
I have OTRS installed which is able to send the acknowledgement to Nagios alarm through the plugin. Whenever I try bulk close ticket in OTRS it will send an acknowledgment to all the selected tickets which are normal. However, in Nagios, I am seeing multiple cmd.cgi processes running during bulk close and takes more than normal time to acknowledge those alarms.Is there any way to optimize the acknowledge latency?
Re: Nagios Acknowledge cmd.cgi takes more time
Posted: Tue Jul 25, 2017 12:46 pm
by mcapra
cmd.cgi isn't doing anything more sophisticated than accepting a request and writing that request out to the external commands file:
https://github.com/NagiosEnterprises/na ... 2138-L2144
That external commands file is then churned through by some sub-process within Nagios Core. The external commands file is basically just a spooler.
But if you're submitting hundreds/thousands of requests via cmd.cgi at once, I can see where the processes might get a bit hung since you're opening/closing the file as many times as you have requests (I think, haven't looked at
cmd_submitf). The solution to this in my mind is to write all the commands to the external commands file in one shot; One single open/close cycle of the file.
I don't think there's anything native to cmd.cgi that would allow for bulk/batched requests though. Might be worth raising an issue on GitHub if it's functionality you'd like to see.
You could probably build a PHP/Perl page to accept bulk commands and write them all out in a single open/close cycle of the external commands file. Then you'd have a single process competing for the resource rather than infinitely many. I'm thinking either JSON or appending numbers to POST fields. In the latter case, something like:
Code: Select all
command_0=[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;service0;2;1;1;Some One;Some Acknowledgement Comment
command_1=[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;service1;2;1;1;Some One;Some Acknowledgement Comment
command_2=[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;service2;2;1;1;Some One;Some Acknowledgement Comment
...
command_n=[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;servicen;2;1;1;Some One;Some Acknowledgement Comment
Or as JSON:
Code: Select all
{
"commands":[
"[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;service1;2;1;1;Some One;Some Acknowledgement Comment",
"[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;service2;2;1;1;Some One;Some Acknowledgement Comment",
"[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;service3;2;1;1;Some One;Some Acknowledgement Comment",
....
"[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;servicen;2;1;1;Some One;Some Acknowledgement Comment"
]
}
Then let the PHP/Perl deconstruct the data and push it directly into the external commands file.
Re: Nagios Acknowledge cmd.cgi takes more time
Posted: Tue Jul 25, 2017 2:36 pm
by dwhitfield
mcapra wrote:
I don't think there's anything native to cmd.cgi that would allow for bulk/batched requests though. Might be worth raising an issue on GitHub if it's functionality you'd like to see.
Here:
https://github.com/NagiosEnterprises/na ... issues/new
I did check to see if there was anything similar and there is not. I also checked the roadmap to see if there was anything planned. There's "Use multi-threading in critical sections" but not sure this is really a critical section. I can try to get some clarity on what is planned, if you like.
Re: Nagios Acknowledge cmd.cgi takes more time
Posted: Thu Jul 27, 2017 7:43 am
by Pradeeps
Thanks for responding. Getting this feature will be really helpful for managing our environment. I will approach github regarding this.
Re: Nagios Acknowledge cmd.cgi takes more time
Posted: Thu Jul 27, 2017 9:16 am
by dwhitfield
This is XI, not Core, but it may also be useful for you thinking about this issue:
https://support.nagios.com/forum/viewto ... 16&t=44824
Re: Nagios Acknowledge cmd.cgi takes more time
Posted: Thu Jul 27, 2017 9:36 am
by Pradeeps
I am unable to view this link as it say am not authorized.
Re: Nagios Acknowledge cmd.cgi takes more time
Posted: Thu Jul 27, 2017 2:01 pm
by dwhitfield
You can run the following script via SSH to acknowledge:
Code: Select all
#!/bin/sh
now=`date +%s`
commandfile='/usr/local/nagios/var/rw/nagios.cmd'
/bin/printf "[%lu] ACKNOWLEDGE_SVC_PROBLEM;localhost;Total Processes;2;1;1;test;Oh boy\n" $now > $commandfile
may be helpful.