Unable to Apply Configuration; Orphaned Objects

[dealertrackut]

Nagios XI Version : 2014R1.5
2.6.32-431.29.2.el6.x86_64 x86_64

Receiving the following errors upon trying to Apply Configuration:

Error: Invalid max_attempts, check_interval, retry_interval, or notification_interval value for service 'SafeNet Port Check' on host 'NODE01'
Error: Could not register service (config file '/usr/local/nagios/etc/services/SafeNet-TCP-9443.cfg', starting on line 16)

A couple of notes about the errors above:
*I'm unable to locate the object "NODE01" in the UI
*"SafeNet-TCP-9443.cfg" Does not exist on the filesystem and doesn't show up in the UI

The only files that exist which are related to the "SafeNet" objects, are as follows:
/usr/local/nagios/share/perfdata/NODE01
/usr/local/nagios/etc/hosts/NODE01.cfg

[ssax]

Please go to Configure > Core Config Manager > Tools > Write Config Files
- Click Delete (don't worry, it's safe, they get re-written)
- Click Write
- Click Verify

Are you still getting the error when you click the Verify button?
- If you are, post the full output of the verify and try to look at the file now without applying configuration (a failed apply will revert the files to a last known good state).
- If you are not, try to apply configuration.

Let us know the results.

[dealertrackut]

That worked.... temporarily. I was able to make changes to some existing hosts and services and apply those okay, but the Apply Configuration option now spins and spins on "Waiting for verification". If I click "Verify", under the Write Config tool, I see the following errors:

Error: Invalid max_check_attempts value for host 'NODE01'
Error: Could not register host (config file '/usr/local/nagios/etc/hosts/NODE01.cfg', starting on line 16)

I have defined a "Max Check Attempts" in the UI as "1" for this particular host, but it doesn't help. Several other hosts do not even have this parameter defined.
If I delete the host, verification is successful, but upon trying to Apply Configuration, I still get the same error.

Attached is the config file for this host.

[ssax]

The parameter needs to be defined on either the host or a template that is applied to the host, the other hosts probably have it assigned to a template.

Let's do this:

Please go to Configure > Core Config Manager > Tools > Write Config Files
- Click Delete (don't worry, it's safe, they get re-written)
- Click Write

Now open up an SSH session to the XI server and run this command:

Code: Select all

tail -f /usr/local/nagiosxi/var/cmdsubsys.log

Then go and try to apply configuration again and post the full output (sanitized).

[dealertrackut]

Deleting and Writing gets things working for about 2 or 3 'Apply Configuration' sequences, but eventually fails with the same message. Here is what the log file shows:

--2015-08-20 16:38:47-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... failed: Name or service not known.
wget: unable to resolve host address “localhost”
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/ - -no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=n@gweb' -O na giosql.loginNAGIOSQL LOGIN FAILED!
OUTPUT=/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components /ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=n@gweb ' -O nagiosql.loginNAGIOSQL LOGIN FAILED!
RETURNCODE=2
..............
PROCESSED 1 COMMANDS
tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated

[Box293]

OK so we are seeing some issues here.

Can you please run these commands

Code: Select all

tail /var/log/messages -n 100 > /tmp/messages_log.txt
tail /var/log/httpd/error_log -n 100 > /tmp/httpd_log.txt
tail /var/log/mysqld.log -n 100 > /tmp/mysqld_log.txt

Send us these files:
/tmp/messages_log.txt
/tmp/httpd_log.txt
/tmp/mysqld_log.txt

Also, can you please post the file /etc/sudoers and any files in /etc/sudoers.d/

[dealertrackut]

I ended up just doing a restore of the Virtual Machine to get things back to where they were. This did require that we repaired some MYSQL tables, so if you have any recommendations for backing up Nagios with Veeam that would be awesome. Otherwise, consider this issue closed.

Thanks!

[lmiltchev]

Unfortunately, we haven't tested/documented Veeam backups. Here's our official documentation on backing up and restoring XI:

https://assets.nagios.com/downloads/nag ... ios-XI.pdf

You can easily snapshot VMs in vmware workstation/ESX/Vsphere and revert to a snapshot if needed.

This did require that we repaired some MYSQL tables...

The database can get corrupted if the VM is not shut down properly. The correct way to shut the VM down would be to run in a terminal:

Code: Select all

shutdown -h now

Do not right-click on the VM->Power->Power Off. This may lead to crashed db tables.

I am locking this post and marking it as "resolved". If you have any more issues/questions, please start a new thread.

[Box293]

I created a plugin that will check for any crashed tables called box293_check_mysql_table_status:

https://exchange.nagios.org/directory/P ... us/details

This way you'll find out about it as soon as it occurs.