Problems with upgrade to 2012R1.5b

[hhlodge]

Hi,

I upgraded from 2011r3.3 this morning. This is CentOS 5.6 64 bit physical machine. My customized check_xi_service_wmiplus command didn't come across and my host and service and notification handlers didn't bring over the " --notificationrecipients="$NOTIFICATIONRECIPIENTS$"" option I had set. Worse thing though is I had to empty out my contact groups so scores of people wouldn't get flooded with the hundreds of errors from the messed up wmi_plus commands but when I go to add contacts back in, I add them, save and apply the configuration and when I go back in, it's empty again! No contacts line is written in contactgroups.cfg for that object. If I go into a contact and add a contact group to that, it doesn't stick either. Also, I see this in the nagios.log right after a configuration change restart.

Code: Select all

Sun Feb  3 11:14:52 2013  ndomod: NDOMOD 1.5.1 (05-15-2012) Copyright (c) 2009 Nagios Core Development Team and Community Contributors
Sun Feb  3 11:14:52 2013  ndomod: Could not open data sink!  I'll keep trying, but some output may get lost...

[hhlodge]

Update - I logged out as myself and back in as nagiosadmin and the contacts stick with the contact groups now, but the configuration won't apply because of this error. Oddly, even as nagiosadmin, the host and service notification commands I add for myself do not stick.

Code: Select all

Checking contacts...
Error: Contact 'ktucker' has no service notification commands defined!
Error: Contact 'ktucker' has no host notification commands defined!

I also see I lost all my wizards! Argggg!

[slansing]

Are your wizard's completely missing from the list in XI? Can you take a screen shot of the contact details page of ktucker where the notification commands are defined?

Run the following and report the output back:

Code: Select all

service ndo2db stop

service nagios stop

service ndo2db start

service nagios start

tail -f /usr/local/nagios/var/nagios.log

Let it tail for about a minute then post the information back, unless you are getting flooded by the same message repetitively.

[hhlodge]

No wizards show. I just uploaded two of them and it said they uploaded successfully and still don't show. Attached is screen shot of my contact pages. And here is tail output after 5 minutes.

Code: Select all

Starting nagios: done.
[1359992767] ndomod: Shutdown complete.
[1359992767] Event broker module '/usr/local/nagios/bin/ndomod.o' deinitialized successfully.
[1359992798] Nagios 3.4.1 starting... (PID=28806)
[1359992798] Local time is Mon Feb 04 10:46:38 EST 2013
[1359992798] LOG VERSION: 2.0
[1359992798] ndomod: NDOMOD 1.5.1 (05-15-2012) Copyright (c) 2009 Nagios Core Development Team and Community Contributors
[1359992798] ndomod: Successfully connected to data sink.  52 queued items to flush.
[1359992798] ndomod: Successfully flushed 52 queued items to data sink.
[1359992798] Event broker module '/usr/local/nagios/bin/ndomod.o' initialized successfully.
[1359992799] Finished daemonizing... (New PID=28811)
[1359992819] SERVICE NOTIFICATION: syslog-local7;PSMSC02SQLFIN;Eventlog - System;WARNING;notify-service-by-syslog;WARNING - [Triggered by _ItemCount0] - 12 event(s) of at least Severity Level Warning, were recorded in th)
[1359992859] SERVICE ALERT: noatl-corehp01;SNMP Traps;OK;HARD;1;A linkUp trap signifies that the SNMP entity, acting in an 66 up up / ifIndex.66 (INTEGER32):66 ifAdminStatus.66 (INTEGER):up ifOperStatus.66 (INTEGER):up
[1359992889] SERVICE ALERT: noatl-corehp01;SNMP Traps;OK;HARD;1;A linkDown trap signifies that the SNMP entity, acting in 66 down down / ifIndex.66 (INTEGER32):66 ifAdminStatus.66 (INTEGER):down ifOperStatus.66 (INTEGER)n
[1359992899] SERVICE ALERT: noatl-corehp01;SNMP Traps;OK;HARD;1;A linkUp trap signifies that the SNMP entity, acting in an 66 up up / ifIndex.66 (INTEGER32):66 ifAdminStatus.66 (INTEGER):up ifOperStatus.66 (INTEGER):up
[1359992939] SERVICE ALERT: localhost;Current Load;WARNING;SOFT;1;WARNING - load average: 3.48, 4.67, 4.51
[1359992999] SERVICE ALERT: localhost;Current Load;WARNING;SOFT;2;WARNING - load average: 2.56, 4.18, 4.35

contact1.png

contact2.png

[slansing]

I will do some digging into the issues you are seeing with this broken upgrade, meanwhile can you click on the buttons that say Manage Host Notification Commands, and Service Commands and show what commands you have defined for each? Is the contact in sync when you back out to the list of contacts in the CCM?

Also please run the following and show the output:

Code: Select all

ll /usr/local/nagiosxi/html/includes/configwizards

[hhlodge]

I am not sure what you're asking regarding the backing out the list of contacts?
Here is the Host command. It's empty after each configuration save.

contact3.png

And here is the ll output.
[code# ll /usr/local/nagiosxi/html/includes/configwizards
total 208
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 autodiscovery
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 bpiwizard
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 bulkhostimport
-rwxr-x--- 1 nagios nagios 178 Feb 3 09:40 configwizardhelper.inc.php
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 dhcp
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 dnsquery
drwxr-x--- 7 nagios nagios 4096 Feb 3 09:24 email-delivery
drwxr-x--- 6 nagios nagios 4096 Feb 3 09:24 esensors-websensor
drwxr-x--- 6 nagios nagios 4096 Feb 3 09:24 exchange
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 ftpserver
drwxr-x--- 2 nagios nagios 4096 Feb 3 09:30 genericnetdevice
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 ldapserver
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 linux-server
drwxr-x--- 6 nagios nagios 4096 Feb 3 09:24 linux_snmp
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 macosx
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 mailserver
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 mssql_database
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 mssql_query
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 mssql_server
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 mysqlquery
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 mysqlserver
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 nagiostats
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 nagiosxiserver
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 nrpe
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 oracle_query
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 oracle_serverspace
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 oracle_tablespace
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 passivecheck
drwxr-x--- 6 nagios nagios 4096 Feb 3 09:24 passiveobject
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 postgresdb
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 postgresquery
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 postgresserver
drwxr-x--- 2 nagios nagios 4096 Apr 11 2011 printer
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 radiusserver
drwxr-x--- 2 nagios nagios 4096 Feb 3 09:24 snmp
drwxr-x--- 5 nagios nagios 4096 Dec 31 2010 snmp-trap
drwxr-x--- 2 nagios nagios 4096 Feb 3 09:24 snmpwalk
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 solaris
drwxr-x--- 7 nagios nagios 4096 Feb 3 09:24 ssh-expect
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 sshproxy
drwxr-x--- 3 nagios nagios 4096 Feb 4 10:37 switch
drwxr-x--- 2 nagios nagios 4096 Feb 3 09:24 tcpudpport
drwxr-x--- 6 nagios nagios 4096 Feb 3 09:24 vmware
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 watchguard
drwxr-x--- 2 nagios nagios 4096 Feb 3 09:30 website
drwxr-x--- 3 nagios nagios 4096 Feb 3 09:30 websiteurl
drwxr-x--- 2 nagios nagios 4096 Apr 11 2011 webtransaction
drwxr-x--- 2 nagios nagios 4096 Feb 3 09:24 windowsdesktop
drwxr-x--- 5 nagios nagios 4096 Feb 3 09:24 windowseventlog
drwxr-x--- 2 nagios nagios 4096 Feb 3 09:24 windowsserver
drwxr-x--- 5 nagios nagios 4096 Apr 18 2012 windowssnmp
drwxr-x--- 6 nagios nagios 4096 Feb 3 09:24 windowswmi
][/code]

[abrist]

To troubleshoot the wizards, could you do a running tail on the http error log while trying to open the wizards interface and post the output here in a code wrap:

Code: Select all

tail -f /var/log/httpd/error_log

Open the wizards link and post the output. I am expecting to see php errors.

[hhlodge]

Sure enough.

Code: Select all

[Mon Feb 04 11:30:34 2013] [error] [client 10.10.2.161] PHP Fatal error:  Cannot redeclare snmpwalk_configwizard_init() (previously declared in /usr/local/nagiosxi/html/includes/components/snmpwalk/snmpwalk.inc.php:15) in /usr/local/nagiosxi/html/includes/configwizards/snmpwalk/snmpwalk.inc.php on line 28, referer: http://psm-itmon/nagiosxi/config/

[abrist]

XI didn't have sanity checking in 2011 for importing a wizard through the component import and vice versa. You will need to delete the the snmpwalk component folder to get the wizards working once again:

Code: Select all

rm -rf /usr/local/nagiosxi/html/includes/components/snmpwalk

The sanity checking has been added in 2012 and this was most likely an old problem that is just rearing its head now.

[hhlodge]

Yup, all showing now. Thanks for that.