Page 1 of 1
configuration update failure
Posted: Mon Nov 12, 2012 10:27 pm
by kotterbein
When attempting to apply a configuration, the configuration fails, but does not write any configuration snapshot, therefore I cannot troubleshoot what the issue is that is keeping it from applying properly.
running Nagios 2012- anyone ever experience this? is there another spot I can look to try to cobble together the issue? I have attempted backing out the changes, and it no longer shows that the configuration is out of date, but continue to get a failure notification with no snapshot created.
Thanks-
Re: configuration update failure
Posted: Tue Nov 13, 2012 11:25 am
by scottwilkerson
This could be a php max_execution_time issue
http://support.nagios.com/wiki/index.ph ... _Completes
Additionally, you can always go to Configure -> CCM -> Tools -> Write Config Files
Then work from the top down looking for errors.
Only do the restart if you have no errors.
Re: configuration update failure
Posted: Tue Nov 13, 2012 4:37 pm
by kotterbein
I had previously applied this due to the large number of service checks we are doing, and just added further to extend:
Code: Select all
308 ;;;;;;;;;;;;;;;;;;;
309 ; Resource Limits ;
310 ;;;;;;;;;;;;;;;;;;;
311
312 max_execution_time = 180 ; Maximum execution time of each script, in seconds
313 max_input_time = 180 ; Maximum amount of time each script may spend parsing request data
314 memory_limit = 512M ; Maximum amount of memory a script may consume
315
however it is not as though it just stalls and does not come back- it says it fails to write the configuration, and does not provide any snapshot.
as far as pre-flight check, here is the verify:
Code: Select all
Nagios Core 3.4.1
Copyright (c) 2009-2011 Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 05-11-2012
License: GPL
Website: http://www.nagios.org
Reading configuration data...
Read main config file okay...
Processing object config directory '/usr/local/nagios/etc/static'...
Processing object config file '/usr/local/nagios/etc/static/xiobjects.cfg'...
Processing object config file '/usr/local/nagios/etc/static/xitemplates.cfg'...
Processing object config file '/usr/local/nagios/etc/static/xitest.cfg'...
Processing object config file '/usr/local/nagios/etc/contacttemplates.cfg'...
Processing object config file '/usr/local/nagios/etc/contactgroups.cfg'...
Processing object config file '/usr/local/nagios/etc/contacts.cfg'...
Processing object config file '/usr/local/nagios/etc/timeperiods.cfg'...
Processing object config file '/usr/local/nagios/etc/commands.cfg'...
Processing object config file '/usr/local/nagios/etc/hostgroups.cfg'...
Processing object config file '/usr/local/nagios/etc/servicegroups.cfg'...
Processing object config file '/usr/local/nagios/etc/hosttemplates.cfg'...
Processing object config file '/usr/local/nagios/etc/servicetemplates.cfg'...
Processing object config file '/usr/local/nagios/etc/servicedependencies.cfg'...
Processing object config file '/usr/local/nagios/etc/serviceescalations.cfg'...
Processing object config file '/usr/local/nagios/etc/hostdependencies.cfg'...
Processing object config file '/usr/local/nagios/etc/hostescalations.cfg'...
Processing object config file '/usr/local/nagios/etc/hostextinfo.cfg'...
Processing object config file '/usr/local/nagios/etc/serviceextinfo.cfg'...
Processing object config directory '/usr/local/nagios/etc/hosts'...
Processing object config file '/usr/local/nagios/etc/hosts/pa-pe01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/iec-fw02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pd-di01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pd-sh01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-lb02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ic-sw01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pd-sw02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-di02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ic-sw05.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ied-fw04.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ic-fw01-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/td-di01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ic-sw06.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ied-fw02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-lb01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-wa03.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ed-fw02-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/td-di02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-wa01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ied-fw03.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pa-pe02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ic-sw02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pd-pe02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/id-sw01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pd-lb01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pd-wa01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/tc-lb02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/id-sw04.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-sw01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-wa05.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/id-fw01-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-wa04.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pd-md01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ied-fw01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-sh01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/id-sw03.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pd-pe01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/localhost.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/id-fw02-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/iec-fw01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/tc-di01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/id-sw06.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-wa02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/id-di01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/iec-fw03.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ec-fw02-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/id-di02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ed-fw01-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-wa06.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pd-md02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-di01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pd-di02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/id-sw05.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/tc-lb01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pd-sw01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pd-wa02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/iec-fw04.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-md02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-pe02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-md01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ic-sw03.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ic-sw04.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pd-lb02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ic-fw02-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-pe01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/id-sw02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/pc-sw02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/hosts/ec-fw01-bb1.inf.ise.com.cfg'...
Processing object config directory '/usr/local/nagios/etc/services'...
Processing object config file '/usr/local/nagios/etc/services/pa-pe01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/iec-fw02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pd-di01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pd-sh01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-lb02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ic-sw01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pd-sw02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-di02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ic-sw05.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ied-fw04.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ic-fw01-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/td-di01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ic-sw06.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ied-fw02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-lb01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-wa03.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ed-fw02-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/td-di02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-wa01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ied-fw03.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pa-pe02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ic-sw02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pd-pe02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/id-sw01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pd-lb01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pd-wa01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/tc-lb02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/id-sw04.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-sw01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-wa05.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/id-fw01-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-wa04.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pd-md01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ied-fw01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-sh01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/id-sw03.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pd-pe01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/localhost.cfg'...
Processing object config file '/usr/local/nagios/etc/services/id-fw02-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/iec-fw01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/id-sw06.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-wa02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/id-di01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/iec-fw03.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ec-fw02-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/id-di02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ed-fw01-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-wa06.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pd-md02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-di01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pd-di02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/id-sw05.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/tc-lb01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pd-sw01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pd-wa02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/iec-fw04.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-md02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-pe02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-md01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ic-sw03.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ic-sw04.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pd-lb02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ic-fw02-bb1.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-pe01.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/id-sw02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/pc-sw02.inf.ise.com.cfg'...
Processing object config file '/usr/local/nagios/etc/services/ec-fw01-bb1.inf.ise.com.cfg'...
Read object config files okay...
Running pre-flight check on configuration data...
Checking services...
Checked 5916 services.
Checking hosts...
Warning: Host 'ic-sw03.inf.ise.com' has no default contacts or contactgroups defined!
Warning: Host 'tc-di01.inf.ise.com' has no default contacts or contactgroups defined!
Checked 68 hosts.
Checking host groups...
Checked 1 host groups.
Checking service groups...
Checked 2 service groups.
Checking contacts...
Checked 7 contacts.
Checking contact groups...
Checked 2 contact groups.
Checking service escalations...
Checked 0 service escalations.
Checking service dependencies...
Checked 0 service dependencies.
Checking host escalations...
Checked 0 host escalations.
Checking host dependencies...
Checked 0 host dependencies.
Checking commands...
Checked 109 commands.
Checking time periods...
Checked 12 time periods.
Checking for circular paths between hosts...
Checking for circular host and service dependencies...
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...
Total Warnings: 2
Total Errors: 0
Things look okay - No serious problems were detected during the pre-flight check
it does seem to be timing out, but I cannot verify why. should I increase the time to 600 seconds?
Re: configuration update failure
Posted: Tue Nov 13, 2012 4:48 pm
by kotterbein
applied the following:
Code: Select all
max_execution_time = 600 ; Maximum execution time of each script, in seconds
max_input_time = 600 ; Maximum amount of time each script may spend parsing request data
seems to have been timing out. we are now over 7K checks, so I'm concerned about the saturation point- we have gearman deployed, but still am concerned.
Thanks for the replies.
Re: configuration update failure
Posted: Tue Nov 13, 2012 5:07 pm
by slansing
Looks like you are getting into that grey area as far as checks goes per installation, if you are running into performance issues have you considered offloading the database and implementing Ram Disk usage?
Re: configuration update failure
Posted: Wed Nov 14, 2012 8:55 am
by kotterbein
I haven't looked at this yet, but seems it may become something to do- do you have a resource doc on this proceedure should I decide we need to move in that direction?
Re: configuration update failure
Posted: Wed Nov 14, 2012 9:04 am
by scottwilkerson