Nagios services stopped updating from plugins

[hank.wojteczko]

Nagios services stopped updating. Testing the plugin from the CLI as the nagios user proved the plugins are correctly reporting the state of services to be monitored. Deleting the services from Nagios core and then re-importing using the REST API service, followed by an apply and restart of the nagios service, appears to have resolved the issue for now. sudo /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg was run prior to applying the import to ensure there were no detected configuration errors. No changes to plugins were made in any way during this test. My concern is that this issue will happen again. Nagios version 5.7.2, OS is OLE 4.14.35-1902.304.6.3.el7uek.x86_64, running on a VM in the Oracle cloud.

[scottwilkerson]

If this happens again it would be useful for us to get a system profile.zip before taking corrective action from
Admin -> System Profile

This will give us some information on what is possibly happening

[hank.wojteczko]

Scott:

So far it has not happened again. I do note that 5.7.3 is out, but do not want to upgrade for the sake of upgrading. Do you recommend we upgrade? Or should we hold at the current version?

How sensitive is Nagios Core to thread contention?

I noted this morning there has been thread contention on the host OS. My plugin code runs as an instance on the Nagios VM and continuously requires 2 threads, with bursts to 4. I will be increasing the VM core count to 8 and will constrain the number of active threads for my plugin to 4. This will ensure there will always be at least 4 available threads for Nagios Core and XI.

[scottwilkerson]

Scott:

So far it has not happened again. I do note that 5.7.3 is out, but do not want to upgrade for the sake of upgrading. Do you recommend we upgrade? Or should we hold at the current version?

How sensitive is Nagios Core to thread contention?

I noted this morning there has been thread contention on the host OS. My plugin code runs as an instance on the Nagios VM and continuously requires 2 threads, with bursts to 4. I will be increasing the VM core count to 8 and will constrain the number of active threads for my plugin to 4. This will ensure there will always be at least 4 available threads for Nagios Core and XI.

I would highly recommend updating to 5.7.3

There are some NDO fixes in there that could fix what could be perceived as the checks not happening because of a delay in the UI updating

[hank.wojteczko]

ok, I'll get it scheduled. Thanks for the advise. I'll consider this issue resolved unless it comes up again after the upgrade to 5.7.3.

[scottwilkerson]

ok, I'll get it scheduled. Thanks for the advise. I'll consider this issue resolved unless it comes up again after the upgrade to 5.7.3.

Sounds good

Going to close this thread, please feel free to open another if the issue comes up again