Unable to remove hosts and services from NagiosXI

[ibnetworking]

Hi all,

we have some hosts and services that we cannot remove from NagiosXI.

We're seeing hosts/services in status page while they are not present in Core Config Manager. Also, there are no config files for those hosts/services in /usr/local/nagios/etc/hosts|services.

Only place we're seeing them is in "nagios" database, they are not present "nagiosql" database.

NagiosXI version:
Installed Version: 5.5.4

Please advise on how to remove them. Thx.

[benjaminsmith]

It's possible there are multiple instances of Nagios running. Let's try stopping and restarting Nagios and see if that takes care of the problem.

Code: Select all

service nagios stop
killall -9 nagios
service nagios start

More information on Ghost Hosts is available on our knowledgebase.

https://support.nagios.com/kb/article.php?id=27

[ibnetworking]

That helped but it seems that we have to do it on every delete, if we do Apply Config or start reconfigure_nagios.sh manually hosts/services are not removed. They are gone only after the procedure you wrote above.

Any advices? Thx.

[benjaminsmith]

Another option to resolve "ghost" host/services issues is to delete all host/service configuration files and then apply configuration. Please try the following:

Go to the Core Config Manager
Tools > Config File Management (in older versions of XI it may be under the heading "Write Config FIles")
Click on Write and Verify to check for config errors
If you don't have any errors, click on Delete to remove all host and service configuration files.
Next, apply configuration. The configs will be recreated with whatever is in the database.

Before making any other changes, it's a good idea to make a sysetem backup. This can easily be done manually or through the web GUI, Go to Admin > System Backups > Local Backup Archives... and select Create Backup.

Backup Instructions:
https://assets.nagios.com/downloads/nag ... ios-XI.pdf

[ibnetworking]

Ok, we have situation update.

Deleting configuration files and writing them again didn't really help.

What we do is, we delete service using Core Config Manager and hit Apply configuration.
Files are deleted from disk, they are no longer visible in CCM but we can still see them in search at home page.

Services are visible until we do systemctl stop nagios && killall nagios && systemctl start nagios. After we restart nagios they are no longer visible.

Do you have some solutions or ideas why nagios is not properly restarted after apply configuration?

[lmiltchev]

Is this a new install or an upgrade from an older version of Nagios XI?

Run the following command to begin capturing log output:

Code: Select all

tail -f /usr/local/nagiosxi/var/cmdsubsys.log &> /tmp/cmd.txt

Apply Configuration from the web interface. After the browser has returned some output to the screen, press Ctrl+c to stop the log tail, and post the cmd.txt file that was generated in the /tmp directory by the above instructions.

Also, post the following files (whichever exists on your system) - /usr/lib/systemd/system/nagios.service or /etc/init.d/nagios.

[ibnetworking]

It is an upgrade of older version of Nagios XI to 5.5.4.
Nagios Core is at 4.4.2 version.

You can see cmdsubsys.log attached in cmd.txt file.
And nagios init file can be found in attachment init_nagios.txt

Do you need anything else?

[lmiltchev]

I didn't see any issues with the nagios init file - it hasn't been modified. Also, there are no issues in the cmdsubsys.log. Can you PM me your profile? You can obtain it by going to Admin > System Config > System Profile > Download Profile.

[tgriep]

Thanks for the profile.
I found a couple of things that need to be addressed.

One thing I found is that there are some Max Connection errors when the system is writing to the MYSQL database.
When this happens, the data cannot be updated and could cause the issue you are having.
Follow this KB article to increase the Max Connection settings for the MYSQL database.
https://support.nagios.com/kb/article/n ... s-513.html

Second, it looks like SELinux is enabled on the Nagios server and that causes issues as well. It needs to be disabled so the Nagios processes can fully function so disable that.

After the changes, let up know if it helps out in the issue.

[lmiltchev]

Nagios cannot be stopped properly (or killed) because the nagios.lock cannot be found. We believe the issue is caused by different paths to the lock file in the nagios init script and the main config file. In the /etc/init.d/nagios, you have:

Code: Select all

NagiosRunFile=/var/run/nagios.lock

but in the /usr/local/nagios/etc/nagios.cfg file, you have:

Code: Select all

lock_file=/usr/local/nagios/var/nagios.lock

So, let's try the following.

1. Stop nagios.

Code: Select all

service nagios stop
killall nagios

2. Change the line below in the /etc/init.d/nagios from this:

Code: Select all

NagiosRunFile=/var/run/nagios.lock

to this:

Code: Select all

NagiosRunFile=/usr/local/nagios/var/nagios.lock

Save and exit.

3. Remove nagios.lock from both places (if it still exist).

Code: Select all

rm -f /usr/local/nagios/var/nagios.lock
rm -f /var/run/nagios.lock

4. Start nagios

Code: Select all

service nagios start

Let us know if this helped.