NagiosXI partially working after Red Hat 5 updates and reboo

[drakeu]

After applying patches to our Red Hat server NagiosXI is on, it is not working correctly. I first noticed this when trying to "Apply Configuration" and it just hung. I tried to repair the database and also restore the mysql database to eliminate any possibilities there. In the Web Interface I see the following with Red Exclamation Marks.

XI System Component Status:

Database Backend
Database Maintenance
Command Subsystem
Event Manager
Feed Processor
Report Engine
Cleaner
Nonstop Operations Manager
System Statistics

Also when I login I notice that the Nagios Server Statistics are not updating. For instance IO on the server has been stuck at 16.41% and never updates. I did see an error in the logs around the time of the patch installation and reboot time:


[1344546717] Caught SIGTERM, shutting down...
[1344546717] ndomod: Error writing to data sink! Some output may get lost...
[1344546717] ndomod: Please check remote ndo2db log, database connection or SSL Parameters
[1344546717] Successfully shutdown... (PID=18708)
[1344546717] ndomod: Shutdown complete.
[1344546717] Event broker module '/usr/local/nagios/bin/ndomod.o' deinitialized successfully.
[1344546955] Nagios 3.4.1 starting... (PID=3838)
[1344546955] Local time is Thu Aug 09 16:15:55 CDT 2012
[1344546955] LOG VERSION: 2.0
[1344546955] ndomod: NDOMOD 1.5.1 (05-15-2012) Copyright (c) 2009 Nagios Core Development Team and Community Contributors
[1344546955] ndomod: Successfully connected to data sink. 4 queued items to flush.
[1344546955] ndomod: Successfully flushed 4 queued items to data sink.
[1344546955] Event broker module '/usr/local/nagios/bin/ndomod.o' initialized successfully.

However it seems to be running now. The monitoring engine is running and appears OK. Please advise ASAP. Thanks.

[drakeu]

I think I have found the issue. Just need to know how to fix it. In the system profile of NagiosXI I see the following issues:

Ping Test localhost

Running:
/bin/ping -c 3 localhost 2>&1
ping: unknown host localhost
Test wget To locahost

WGET From URL: http://localhost/nagiosql/index.php
Running:
/usr/bin/wget http://localhost/nagiosql/index.php
--2012-08-09 22:25:40-- http://localhost/nagiosql/index.php
Resolving localhost... failed: Name or service not known.
wget: unable to resolve host address `localhost'

I then followed the support wiki and ran output while trying to run an Apply Configuration and got this:

PHP Warning: pg_pconnect(): Unable to connect to PostgreSQL server: could not translate host name "localhost" to address: Name or service not known in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-postgres64.inc.php on line 682
DB Connect Error [nagiosxi]: Database connection failed
ERROR CONNECTING TO DATABASES!
tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
PHP Warning: pg_pconnect(): Unable to connect to PostgreSQL server: could not translate host name "localhost" to address: Name or service not known in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-postgres64.inc.php on line 682
DB Connect Error [nagiosxi]: Database connection failed
ERROR CONNECTING TO DATABASES!


This is likely related to the patches that were applied. There were several php and postgresql patches applied today (partial list):

php.i386 0:5.1.6-39.el5_8
postgresql.i386 0:8.1.23-5.el5_8

I suspect one of those or their related patches. Please advise on what to do. Thanks!

[scottwilkerson]

Can you run the following commands and return the output

Code: Select all

ip link show lo
ll /etc/nsswitch.conf
cat /etc/hosts

Also, can we open /etc/nsswitch.conf and you have a line like this

Code: Select all

hosts:      files dns

[scottwilkerson]

I just found this
https://bugzilla.redhat.com/show_bug.cgi?id=846631

Looks like it can be fixed by running

Code: Select all

https://bugzilla.redhat.com/show_bug.cgi?id=846631

[drakeu]

Thank you very much! It was that bug. The nsswitch file permissions were changed by the Red Hat sudo patch. We changed the permissions on the file and we are back in business!