Page 1 of 1

Nagios is lying to us

Posted: Wed Apr 26, 2017 6:19 am
by ibnetworking
Hello,

it appears nagios is lying to us. We have 10k checks, and it's automated via multiple python scripts. The issue happens is that script detects changes in services and does the correct API calls, and then nagios goes ahead and lies to us.
Example:

{
"success": "Removed SERVER-A :: Service - someservice from the system. Config imported but not yet applied."
}


Once we reconfigure nagios, the service is still there.
When we try adding apply in the api call, it says the restart command was sent, but nothing happens. The service is still there.

reconfigure_nagios.sh finishes correctly, without any issues, but the service is magically still there.

Do you have any idea why this is happening?
Also, why is nagios reporting that service was removed when in fact it wasn't?

Re: Nagios is lying to us

Posted: Wed Apr 26, 2017 9:18 am
by dwhitfield
Please log in to mysql with mysql -uroot -pnagiosxi

Then provide the output of the following select statements (but pay attention to the "use" commands):

Code: Select all

use nagios;
select * from nagios_services where display_name="SERVER-A";
use nagiosql;
select * from tbl_services where service_description="SERVER-A";
Also, you can try going to "Config File Management" in the CCM. Then Delete Files, Write Configs, Verify Files in that order. If you can't verify, then solve that issue. Once you can verify, apply the configuration and see if the issue persists.

If it does, can you PM me your Profile? You can download it by going to Admin > System Config > System Profile and click the Download Profile button towards the top. If for whatever reason you *cannot* download the profile, please put the output of View System Info (5.3.4+, Show Profile if older) in the thread (that will at least get us some info). This will give us access to many of the logs we would otherwise ask for individually. If security is a concern, you can unzip the profile take out what you like, and then zip it up again. We may end up needing something you remove, but we can ask for that specifically.

After you PM the profile, please update this thread. Updating this thread is the only way for it to show back up on our dashboard.

Also, if you PM, can you give me the actual name of the server? If it's SERVER-A, then great, but I'm suspicious. :)

Re: Nagios is lying to us

Posted: Wed Apr 26, 2017 10:28 am
by eloyd
Trust the computer. The computer is your friend.

Re: Nagios is lying to us

Posted: Wed Apr 26, 2017 10:41 am
by dwhitfield
@ibnetworking please let us know if you have any problems running the commands I gave. Thanks!