Can't apply config since upgrade to 2014.2.3
Posted: Thu Jan 15, 2015 2:07 pm
After upgrading Nagios XI to latest version (last weekend) I am unable to apply configuration. After some investigation, I found that if I re-run the upgrade script it does something that causes Nagios XI to be able to apply configuration again without error.
For example, on my smallest nagios server I receive the following error:
Error: Contact 'etsi-all' is not defined anywhere!
Error: Could not add contact 'etsi-all' to host (config file '/usr/local/nagios/etc/hosts/<SERVERNAMEOMMITTED>.<DOMAINNAME>.com.cfg', starting on line 14)
When I look at the /usr/local/nagios/etc/contacts.cfg I see the following:
define contact {
contact_name etsi-all
alias etsi-all
host_notification_period etsi-all_notification_times
service_notification_period etsi-all_notification_times
host_notification_options d,u,r,f,s
service_notification_options w,u,c,r,f,s
email <EMAILADDRESS>@<DOMAINNAME>.com
use xi_contact_generic
}
So it's clearly defined but I'll play along. I then deleted all of the references to the contact on hosts and services using the bulk modification tool. Unfortunately, still unable to apply config. So I re-run the upgrade. Now I'm able to apply config (for a short time.) To hopefully settle things down I removed the Nagios XI user and the CCM contact for etsi-all (and verified that the user was removed from contacts.cfg.) Wait 20 minutes. Try applying configuration, same exact error:
Error: Contact 'etsi-all' is not defined anywhere!
Error: Could not add contact 'etsi-all' to host (config file '/usr/local/nagios/etc/hosts/<SERVERNAMEOMMITTED>.<DOMAINNAME>.com.cfg', starting on line 14)
So then I attempted to run a database repair (/usr/local/nagiosxi/scripts/repair_databases.sh) Again, the error persists. Then it dawned on me, I went to look in the database and when running a simple query like 'SELECT * FROM nagios.nagios_contacts;' MySQL Workbench completely crashes or (on subsequent attempts) hangs trying to run the query. I'm seriously starting to think its directly database related based on these results.
Any ideas how to fix this beyond starting over from scratch?
For example, on my smallest nagios server I receive the following error:
Error: Contact 'etsi-all' is not defined anywhere!
Error: Could not add contact 'etsi-all' to host (config file '/usr/local/nagios/etc/hosts/<SERVERNAMEOMMITTED>.<DOMAINNAME>.com.cfg', starting on line 14)
When I look at the /usr/local/nagios/etc/contacts.cfg I see the following:
define contact {
contact_name etsi-all
alias etsi-all
host_notification_period etsi-all_notification_times
service_notification_period etsi-all_notification_times
host_notification_options d,u,r,f,s
service_notification_options w,u,c,r,f,s
email <EMAILADDRESS>@<DOMAINNAME>.com
use xi_contact_generic
}
So it's clearly defined but I'll play along. I then deleted all of the references to the contact on hosts and services using the bulk modification tool. Unfortunately, still unable to apply config. So I re-run the upgrade. Now I'm able to apply config (for a short time.) To hopefully settle things down I removed the Nagios XI user and the CCM contact for etsi-all (and verified that the user was removed from contacts.cfg.) Wait 20 minutes. Try applying configuration, same exact error:
Error: Contact 'etsi-all' is not defined anywhere!
Error: Could not add contact 'etsi-all' to host (config file '/usr/local/nagios/etc/hosts/<SERVERNAMEOMMITTED>.<DOMAINNAME>.com.cfg', starting on line 14)
So then I attempted to run a database repair (/usr/local/nagiosxi/scripts/repair_databases.sh) Again, the error persists. Then it dawned on me, I went to look in the database and when running a simple query like 'SELECT * FROM nagios.nagios_contacts;' MySQL Workbench completely crashes or (on subsequent attempts) hangs trying to run the query. I'm seriously starting to think its directly database related based on these results.
Any ideas how to fix this beyond starting over from scratch?