Nagios Acknowledge cmd.cgi takes more time

[Pradeeps]

Hello Folks,

I have OTRS installed which is able to send the acknowledgement to Nagios alarm through the plugin. Whenever I try bulk close ticket in OTRS it will send an acknowledgment to all the selected tickets which are normal. However, in Nagios, I am seeing multiple cmd.cgi processes running during bulk close and takes more than normal time to acknowledge those alarms.Is there any way to optimize the acknowledge latency?

[mcapra]

cmd.cgi isn't doing anything more sophisticated than accepting a request and writing that request out to the external commands file:
https://github.com/NagiosEnterprises/na ... 2138-L2144

That external commands file is then churned through by some sub-process within Nagios Core. The external commands file is basically just a spooler.

But if you're submitting hundreds/thousands of requests via cmd.cgi at once, I can see where the processes might get a bit hung since you're opening/closing the file as many times as you have requests (I think, haven't looked at cmd_submitf). The solution to this in my mind is to write all the commands to the external commands file in one shot; One single open/close cycle of the file.

I don't think there's anything native to cmd.cgi that would allow for bulk/batched requests though. Might be worth raising an issue on GitHub if it's functionality you'd like to see.

You could probably build a PHP/Perl page to accept bulk commands and write them all out in a single open/close cycle of the external commands file. Then you'd have a single process competing for the resource rather than infinitely many. I'm thinking either JSON or appending numbers to POST fields. In the latter case, something like:

Code: Select all

command_0=[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;service0;2;1;1;Some One;Some Acknowledgement Comment
command_1=[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;service1;2;1;1;Some One;Some Acknowledgement Comment
command_2=[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;service2;2;1;1;Some One;Some Acknowledgement Comment
...
command_n=[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;servicen;2;1;1;Some One;Some Acknowledgement Comment

Or as JSON:

Code: Select all

{
   "commands":[
        "[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;service1;2;1;1;Some One;Some Acknowledgement Comment",
        "[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;service2;2;1;1;Some One;Some Acknowledgement Comment",
        "[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;service3;2;1;1;Some One;Some Acknowledgement Comment",
        ....
        "[%lu] ACKNOWLEDGE_SVC_PROBLEM;host1;servicen;2;1;1;Some One;Some Acknowledgement Comment"
    ]
}

Then let the PHP/Perl deconstruct the data and push it directly into the external commands file.

[dwhitfield]

I don't think there's anything native to cmd.cgi that would allow for bulk/batched requests though. Might be worth raising an issue on GitHub if it's functionality you'd like to see.

Here: https://github.com/NagiosEnterprises/na ... issues/new

I did check to see if there was anything similar and there is not. I also checked the roadmap to see if there was anything planned. There's "Use multi-threading in critical sections" but not sure this is really a critical section. I can try to get some clarity on what is planned, if you like.

[Pradeeps]

Thanks for responding. Getting this feature will be really helpful for managing our environment. I will approach github regarding this.

[dwhitfield]

This is XI, not Core, but it may also be useful for you thinking about this issue: https://support.nagios.com/forum/viewto ... 16&t=44824

[Pradeeps]

I am unable to view this link as it say am not authorized.

[dwhitfield]

You can run the following script via SSH to acknowledge:

Code: Select all

#!/bin/sh

now=`date +%s`
commandfile='/usr/local/nagios/var/rw/nagios.cmd'

/bin/printf "[%lu] ACKNOWLEDGE_SVC_PROBLEM;localhost;Total Processes;2;1;1;test;Oh boy\n" $now > $commandfile

https://www.cyberciti.biz/faq/unix-linu ... using-ssh/ may be helpful.