configuration update failure

[kotterbein]

When attempting to apply a configuration, the configuration fails, but does not write any configuration snapshot, therefore I cannot troubleshoot what the issue is that is keeping it from applying properly.

running Nagios 2012- anyone ever experience this? is there another spot I can look to try to cobble together the issue? I have attempted backing out the changes, and it no longer shows that the configuration is out of date, but continue to get a failure notification with no snapshot created.

Thanks-

[scottwilkerson]

This could be a php max_execution_time issue
http://support.nagios.com/wiki/index.ph ... _Completes

Additionally, you can always go to Configure -> CCM -> Tools -> Write Config Files

Then work from the top down looking for errors.

Only do the restart if you have no errors.

[kotterbein]

I had previously applied this due to the large number of service checks we are doing, and just added further to extend:

Code: Select all

308 ;;;;;;;;;;;;;;;;;;;
 309 ; Resource Limits ;
 310 ;;;;;;;;;;;;;;;;;;;
 311 
 312 max_execution_time = 180     ; Maximum execution time of each script, in seconds
 313 max_input_time = 180    ; Maximum amount of time each script may spend parsing request data
 314 memory_limit = 512M    ; Maximum amount of memory a script may consume
 315

however it is not as though it just stalls and does not come back- it says it fails to write the configuration, and does not provide any snapshot.

as far as pre-flight check, here is the verify:

Code: Select all

Nagios Core 3.4.1 
Copyright (c) 2009-2011 Nagios Core Development Team and Community Contributors 
Copyright (c) 1999-2009 Ethan Galstad 
Last Modified: 05-11-2012 
License: GPL 

Website: http://www.nagios.org 
Reading configuration data... 
Read main config file okay... 
Processing object config directory '/usr/local/nagios/etc/static'... 
Processing object config file '/usr/local/nagios/etc/static/xiobjects.cfg'... 
Processing object config file '/usr/local/nagios/etc/static/xitemplates.cfg'... 
Processing object config file '/usr/local/nagios/etc/static/xitest.cfg'... 
Processing object config file '/usr/local/nagios/etc/contacttemplates.cfg'... 
Processing object config file '/usr/local/nagios/etc/contactgroups.cfg'... 
Processing object config file '/usr/local/nagios/etc/contacts.cfg'... 
Processing object config file '/usr/local/nagios/etc/timeperiods.cfg'... 
Processing object config file '/usr/local/nagios/etc/commands.cfg'... 
Processing object config file '/usr/local/nagios/etc/hostgroups.cfg'... 
Processing object config file '/usr/local/nagios/etc/servicegroups.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosttemplates.cfg'... 
Processing object config file '/usr/local/nagios/etc/servicetemplates.cfg'... 
Processing object config file '/usr/local/nagios/etc/servicedependencies.cfg'... 
Processing object config file '/usr/local/nagios/etc/serviceescalations.cfg'... 
Processing object config file '/usr/local/nagios/etc/hostdependencies.cfg'... 
Processing object config file '/usr/local/nagios/etc/hostescalations.cfg'... 
Processing object config file '/usr/local/nagios/etc/hostextinfo.cfg'... 
Processing object config file '/usr/local/nagios/etc/serviceextinfo.cfg'... 
Processing object config directory '/usr/local/nagios/etc/hosts'... 
Processing object config file '/usr/local/nagios/etc/hosts/pa-pe01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/iec-fw02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pd-di01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pd-sh01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-lb02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ic-sw01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pd-sw02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-di02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ic-sw05.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ied-fw04.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ic-fw01-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/td-di01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ic-sw06.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ied-fw02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-lb01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-wa03.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ed-fw02-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/td-di02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-wa01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ied-fw03.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pa-pe02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ic-sw02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pd-pe02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/id-sw01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pd-lb01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pd-wa01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/tc-lb02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/id-sw04.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-sw01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-wa05.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/id-fw01-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-wa04.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pd-md01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ied-fw01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-sh01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/id-sw03.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pd-pe01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/localhost.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/id-fw02-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/iec-fw01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/tc-di01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/id-sw06.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-wa02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/id-di01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/iec-fw03.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ec-fw02-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/id-di02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ed-fw01-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-wa06.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pd-md02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-di01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pd-di02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/id-sw05.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/tc-lb01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pd-sw01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pd-wa02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/iec-fw04.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-md02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-pe02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-md01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ic-sw03.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ic-sw04.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pd-lb02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ic-fw02-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-pe01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/id-sw02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/pc-sw02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/hosts/ec-fw01-bb1.inf.ise.com.cfg'... 
Processing object config directory '/usr/local/nagios/etc/services'... 
Processing object config file '/usr/local/nagios/etc/services/pa-pe01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/iec-fw02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pd-di01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pd-sh01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-lb02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ic-sw01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pd-sw02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-di02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ic-sw05.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ied-fw04.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ic-fw01-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/td-di01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ic-sw06.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ied-fw02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-lb01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-wa03.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ed-fw02-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/td-di02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-wa01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ied-fw03.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pa-pe02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ic-sw02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pd-pe02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/id-sw01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pd-lb01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pd-wa01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/tc-lb02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/id-sw04.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-sw01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-wa05.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/id-fw01-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-wa04.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pd-md01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ied-fw01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-sh01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/id-sw03.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pd-pe01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/localhost.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/id-fw02-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/iec-fw01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/id-sw06.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-wa02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/id-di01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/iec-fw03.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ec-fw02-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/id-di02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ed-fw01-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-wa06.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pd-md02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-di01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pd-di02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/id-sw05.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/tc-lb01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pd-sw01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pd-wa02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/iec-fw04.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-md02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-pe02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-md01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ic-sw03.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ic-sw04.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pd-lb02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ic-fw02-bb1.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-pe01.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/id-sw02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/pc-sw02.inf.ise.com.cfg'... 
Processing object config file '/usr/local/nagios/etc/services/ec-fw01-bb1.inf.ise.com.cfg'... 
Read object config files okay... 

Running pre-flight check on configuration data... 

Checking services... 
Checked 5916 services. 
Checking hosts... 
Warning: Host 'ic-sw03.inf.ise.com' has no default contacts or contactgroups defined! 
Warning: Host 'tc-di01.inf.ise.com' has no default contacts or contactgroups defined! 
Checked 68 hosts. 
Checking host groups... 
Checked 1 host groups. 
Checking service groups... 
Checked 2 service groups. 
Checking contacts... 
Checked 7 contacts. 
Checking contact groups... 
Checked 2 contact groups. 
Checking service escalations... 
Checked 0 service escalations. 
Checking service dependencies... 
Checked 0 service dependencies. 
Checking host escalations... 
Checked 0 host escalations. 
Checking host dependencies... 
Checked 0 host dependencies. 
Checking commands... 
Checked 109 commands. 
Checking time periods... 
Checked 12 time periods. 
Checking for circular paths between hosts... 
Checking for circular host and service dependencies... 
Checking global event handlers... 
Checking obsessive compulsive processor commands... 
Checking misc settings... 

Total Warnings: 2 
Total Errors: 0 

Things look okay - No serious problems were detected during the pre-flight check 

it does seem to be timing out, but I cannot verify why. should I increase the time to 600 seconds?

[kotterbein]

applied the following:

Code: Select all

max_execution_time = 600     ; Maximum execution time of each script, in seconds
max_input_time = 600    ; Maximum amount of time each script may spend parsing request data

seems to have been timing out. we are now over 7K checks, so I'm concerned about the saturation point- we have gearman deployed, but still am concerned.

Thanks for the replies.

[slansing]

Looks like you are getting into that grey area as far as checks goes per installation, if you are running into performance issues have you considered offloading the database and implementing Ram Disk usage?

[kotterbein]

I haven't looked at this yet, but seems it may become something to do- do you have a resource doc on this proceedure should I decide we need to move in that direction?

[scottwilkerson]

http://assets.nagios.com/downloads/nagi ... p#boosting