Page 1 of 2

Active Hosts, Service, Notifications disabled

Posted: Mon Jun 23, 2014 4:24 pm
by akepley
The "System OK" at the top right hand corner of the server web page has 3 green checks and 3 blue i's. When I click on the blue information buttons, I get the "monitoring Engine Status" screen. It shows the process state as 'Not Running'. I click the "Start" Action button and it says "Your request was not processed in a timely manner. It may still execute, as the server may be temporarily busy." The nagios.log has the following:

Code: Select all

[1403558167] Warning: use_embedded_perl_implicitly is deprecated and will be removed.
[1403558167] Warning: sleep_time is deprecated and will be removed.
[1403558167] Warning: p1_file is deprecated and will be removed.
[1403558167] Warning: external_command_buffer_slots is deprecated and will be removed. All commands are always processed upon arrival
[1403558167] Warning: enable_embedded_perl is deprecated and will be removed.
[1403558167] Warning: command_check_interval is deprecated and will be removed. Commands are always handled on arrival
[1403558167] Lockfile '/usr/local/nagios/var/nagios.lock' looks like its already held by another instance of Nagios (PID 1584).  Bailing out...
I have tried stopping nagios, mysqld, npcd, postgresql, crond, ndo2db and then starting them all one at a time to no avail. Then I restarted the entire server and it still didn't help.
Here is the var directory:

Code: Select all

root@nagiosxi akepley# ls -la /usr/local/nagios/var/
total 25596
drwxrwxr-x 6 nagios nagios     4096 Jun 23 16:23 .
drwxr-xr-x 9 root   root       4096 May 16 09:19 ..
drwxrwxr-x 2 nagios nagios    36864 Jun 22 23:59 archives
-rw-r--r-- 1 nagios nagios        0 Jun 23 16:23 host-perfdata
-rw-r--r-- 1 nagios nagios        5 Jun 23 16:20 nagios.lock
-rw-r--r-- 1 nagios nagios   345288 Jun 23 16:20 nagios.log
-rw-r--r-- 1 nagios nagios        5 Jun 23 16:14 ndo2db.lock
srwxr-xr-x 1 nagios nagios        0 Jun 23 16:14 ndo.sock
-rw-r--r-- 1 nagios nagios  3513642 Jun 23 16:23 npcd.log
-rw-r--r-- 1 nagios nagios 10485813 Jun 23 07:00 npcd.log.old
-rw-r--r-- 1 nagios nagios  1642570 Jun 23 16:14 objects.cache
-rw-rw-rw- 1 nagios nagios  4857277 Jun 23 16:22 perfdata.log
-rw------- 1 nagios nagios  2639134 Jun 23 16:20 retention.dat
drwxrwsr-x 2 nagios nagcmd     4096 Jun 23 16:16 rw
-rw-r--r-- 1 nagios nagios     1121 Jun 23 16:23 service-perfdata
drwxr-xr-x 5 root   root       4096 Sep 12  2013 spool
drwxr-xr-x 2 nagios nagios     4096 Jun 23 16:22 stats
-rw-rw-r-- 1 nagios nagios  2624303 Jun 23 16:23 status.dat
And here is a grep for lock_file in nagios.cfg:

Code: Select all

root@nagiosxi akepley# grep lock_file /usr/local/nagios/etc/nagios.cfg
lock_file=/usr/local/nagios/var/nagios.lock

Re: Active Hosts, Service, Notifications disabled

Posted: Mon Jun 23, 2014 8:22 pm
by Box293
Have you tried deleting the lock file?

Code: Select all

rm -f /usr/local/nagios/var/nagios.lock
Does that let Nagios start?

Re: Active Hosts, Service, Notifications disabled

Posted: Mon Jun 23, 2014 8:51 pm
by akepley
Nagios doesn't have a problem stopping or starting. That happens just fine. Regardless, I removed the nagios.lock file and restarted. It recreated and nothing has changed. The System Ok across the top is still green, green, green, blue, blue, blue with the readouts: Monitoring Engine is running (Green), Performance Grapher is Running (Green), Database Backend is Running (Green), Active Host Checks Are Disabled (Blue), Active Service Checks are Disabled (Blue), Notifications are Disabled (Blue). Log file still shows "nagios.lock held by another instance of Nagios"

Also, Apply Configuration seems to just stick now forever "waiting for configuration verification". The only option I have left is to revert back to a prior good snapshot. Is there anything you can recommend?

Re: Active Hosts, Service, Notifications disabled

Posted: Mon Jun 23, 2014 10:20 pm
by Box293
Try running this script which will reset the permissions:

Code: Select all

/usr/local/nagiosxi/scripts/reset_config_perms.sh
Does this help?

Re: Active Hosts, Service, Notifications disabled

Posted: Tue Jun 24, 2014 7:27 am
by akepley

Code: Select all

root@nagiosxi akepley# /usr/local/nagiosxi/scripts/reset_config_perms.sh
RESETTING PERMS
Still having the same issues.

Re: Active Hosts, Service, Notifications disabled

Posted: Tue Jun 24, 2014 1:12 pm
by tmcdonald
What version of XI is this on? And what OS and version?

Re: Active Hosts, Service, Notifications disabled

Posted: Tue Jun 24, 2014 1:15 pm
by akepley
For security reasons, I was asked to setup Nagios to use SSL for HTTPS traffic yesterday. This appears to have caused the issue, although I need to know why and how to setup SSL without running into this issue.

I used this document to set it up:

http://assets.nagios.com/downloads/nagi ... ios_XI.pdf

Re: Active Hosts, Service, Notifications disabled

Posted: Tue Jun 24, 2014 1:16 pm
by akepley
The version is 2014R1.0. OS is CentOS 6.5

Re: Active Hosts, Service, Notifications disabled

Posted: Tue Jun 24, 2014 4:43 pm
by sreinhardt
There was a bug with 2014r1.0 with forced ssl. You should update to 2014r1.1 or newer, r1.2 was released today. This should resolve your issues with ssl, please let us know if they persist.

Re: Active Hosts, Service, Notifications disabled

Posted: Tue Jun 24, 2014 6:57 pm
by akepley
Thanks as always! You guys are great!