Post Upgrade Process State won't turn green

[At0mrail]

Ran the upgrade successfully. Now we cannot change the process state from red to green. HELP!!!!

[lmiltchev]

Run the following commands and show us the output:

Code: Select all

service nagios status
service ndo2db status
/usr/local/nagios/bin/nagios | head -2
/usr/local/nagios/bin/ndo2db | head -2

Are you using Mod Gearman, Livestatus, etc.?

[At0mrail]

Last login: Fri May 23 14:06:20 2014 from 172.16.1.77
[root@clcsrv117 ~]# service nagios status
nagios is not running
[root@clcsrv117 ~]# service ndo2db status
ndo2db (pid 1462) is running...
[root@clcsrv117 ~]# /usr/local/nagios/bin/nagios | head -2

Nagios Core 4.0.5
[root@clcsrv117 ~]# /usr/local/nagios/bin/ndo2db | head -2

NDO2DB 2.0.0
[root@clcsrv117 ~]#

Ok, found this in the nagios log. This shows up when restarting the monitoring engine. something in there it doesn't like.

[1400872029] ndomod registered for retention data'
[1400872029] ndomod registered for contact data'
[1400872029] ndomod registered for contact notification data'
[1400872029] ndomod registered for acknowledgement data'
[1400872029] ndomod registered for state change data'
[1400872029] ndomod registered for contact status data'
[1400872029] ndomod registered for adaptive contact data'
[1400872029] Event broker module '/usr/local/nagios/bin/ndomod.o' initialized successfully.
[1400872029] Error: Module loading failed. Aborting.

[lmiltchev]

You didn't answer my question:

Are you using Mod Gearman, Livestatus, etc.?

Can you post the nagios.cfg file?

[abrist]

Run a verification:

Code: Select all

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

Post the output.

[At0mrail]

login as: root
root@clcsrv117's password:
Last login: Fri May 23 14:06:20 2014 from 172.16.1.77
[root@clcsrv117 ~]# service nagios status
nagios is not running
[root@clcsrv117 ~]# service ndo2db status
ndo2db (pid 1462) is running...
[root@clcsrv117 ~]# /usr/local/nagios/bin/nagios | head -2

Nagios Core 4.0.5
[root@clcsrv117 ~]# /usr/local/nagios/bin/ndo2db | head -2

NDO2DB 2.0.0
[root@clcsrv117 ~]# ^C
[root@clcsrv117 ~]# /usr/local/nagios/bin/nagios -v /usr/local/etc/nagios/etc/nagios.cfg

Nagios Core 4.0.5
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 04-11-2014
License: GPL

Website: http://www.nagios.org
Reading configuration data...
Error: Cannot open main configuration file '/usr/local/etc/nagios/etc/nagios.cfg' for reading!
Error processing main config file!

[root@clcsrv117 ~]# /usr/local/nagios/bin/nagios -v /usr/local/etc/nagios/etc/nagios.cfg

Nagios Core 4.0.5
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 04-11-2014
License: GPL

Website: http://www.nagios.org
Reading configuration data...
Error: Cannot open main configuration file '/usr/local/etc/nagios/etc/nagios.cfg' for reading!
Error processing main config file!

[root@clcsrv117 ~]#

We are thinking that we have to upgrade mod gearmon. Where and how are the next questions...

[scottwilkerson]

Can you confirm the permissions of the following file

Code: Select all

ls -l /usr/local/etc/nagios/etc/nagios.cfg

Also can you verify disk space

Code: Select all

df -h
df -i

[At0mrail]

We ended up updating ModGearman. That seems to have solved that issue.


Of course now some of my service checks (server uptime) aren't working.

Good times, good times.

[sreinhardt]

Are those checks that are normally processed by gearman or something else like passive checks? Also just to confirm, verify now completes without any errors?