all host checks pending after migration

[pnewlon]

After migrating from 32 bit platform to 64 bit all host checks are pending. I installed xi-2012r2.9.tar.gz when prompted that my installation was out of date. I did find that the new server was set to UTC instead of America/New_York so I changed that and restarted all the Nagios services. Checks are still pending.

Code: Select all

==================
UPGRADE COMPLETED!
==================

You can access the Nagios XI web interface by visiting:
    http://10.255.215.202/nagiosxi/

I selected one host and ran check_icmp which returned 'OK'. Below is that output and the output of the host configuration. I'm not sure what to check next, thoughts?

Code: Select all

[etc]# /usr/local/nagios/libexec/check_icmp -H 10.8.51.167 -w 3000.0,80% -c 5000.0,100% -p 5
OK - 10.8.51.167: rta 231.113ms, lost 60%|rta=231.113ms;3000.000;5000.000;0; pl=60%;80;100;;

Code: Select all

###############################################################################
#
# Host configuration file
#
# Created by: Nagios QL Version 3.0.3
# Date:	      2014-04-01 16:08:58
# Version:    Nagios 3.x config file
#
# --- DO NOT EDIT THIS FILE BY HAND --- 
# Nagios QL will overwite all manual settings during the next update
#
###############################################################################

define host {
	host_name			00851_IB01-MP
	use				DMB_host_delay_1
	alias				00851_IB01-MP
	address				10.8.51.167
	hostgroups			1-MP
	max_check_attempts		5
	check_interval			5
	retry_interval			1
	contacts			nagiosadmin
	contact_groups			DMB Group
	notification_options		d
	icon_image			network_node.png
	statusmap_image			network_node.png
	_xiwizard			genericnetdevice
	register			1
	}	

###############################################################################
#
# Host configuration file
#
# END OF FILE
#

[abrist]

The scheduling information was most likely retained between time changes. You could wait an additional 5 hours for the server to catch up, or force all the checks to reschedule:
1) Go to the Mass Acknowledge component.
2) Change command type to "Schedule an immediate check".
3) Check all items.
4) Click Submit. All checks should now be rescheduled.
Did you also change /etc/php.ini and /etc/localtime with the proper timezone settings?

[pnewlon]

> You could wait an additional 5 hours for the server to catch up
I left the system running all night, all checks were still pending when I came in this morning

> or force all the checks to reschedule:
I installed the MA component and ran it, waiting to see if anything good happens

> Did you also change /etc/php.ini and /etc/localtime with the proper timezone settings?
Sure did!

[pnewlon]

Same status - all still pending. Interesting factoids - checking the running processes I see there are checks happening (very few though, at most 70 of 2070 defined hosts) and the status of said hosts are 'pending'.

NEW INFO: when I click the 'view text config' icon in the host configuration panel, the first line of the output says 'error connecting to database'. However, data is still displayed. Is that a red herring?



nagios 27058 27050 0 08:34 ? 00:00:00 /usr/local/nagios/libexec/check_ping -H 10.18.96.205 -w 3000,80% -c 5000,100%
nagios 27079 27058 0 08:34 ? 00:00:00 /bin/ping -n -U -w 30 -c 5 10.18.96.205

01896_OB04-AMT
Host check is pending...

<p class="error">Error connecting to database.</p>###############################################################################
#
# Host configuration file
#
# Created by: Nagios QL Version 3.0.3
# Date: 2014-04-02 08:38:47
# Version: Nagios 3.x config file
#
# --- DO NOT EDIT THIS FILE BY HAND ---
# Nagios QL will overwite all manual settings during the next update
#
###############################################################################

define host {
host_name 01896_OB04-AMT
use DMB_host_delay_1
alias 01896_OB04-AMT
address 10.18.96.205
parents 01896_RTR
hostgroups 6-AMT
contact_groups DMB Group
notification_options d
icon_image network_node.png
statusmap_image network_node.png
_xiwizard genericnetdevice
register 1
}

###############################################################################
#
# Host configuration file
#
# END OF FILE
#
###############################################################################

[pnewlon]

Note to self - remember that you have configured NagiosXI to utilize a RAM disk to enhance performance. When migrating to new server/platform you have to redo all the RAM disk configuration work again. Host checks no longer pending.

[slansing]

Ahh, that would do it, are they all good now?

[pnewlon]

Host checks are running now! I am not getting any performance graphs though.

[tmcdonald]

The performance graphs can take some time to catch up, especially if you have a high load. You can stop nagios for a minute and let npcd (the service responsible for the perfdata processing) catch up.

service nagios stop
<wait 10 min>
service nagios start

[pnewlon]

It turns out that the version of RRD on the 32 bit Nagios installation creates .rrd files that the 64 bit can't read. I just whacked all the .rrd and .xml files in perfdata and all is well. That is something to think about when following the document http://assets.nagios.com/downloads/nagi ... ing_XI.pdf to migrate from one platform to another.

Fix for lack of performance graphs that worked for me:
http://support.nagios.com/forum/viewtop ... 16&t=26421

[tmcdonald]

Thanks for the update, we'll take a look at that doc. I will be closing this thread now, but feel free to start another if you need anything else.