cant get the nagios service to start

[benhank]

I just installed nagios on a freshly wiped server.
the os is centos 6.8
first I installed version xi-2014r2.7 inorder to get postgresql installed, then I upgraded to version 5.2.0 and then I upgraded to the latest version and did a restore from a backup of a working version of nagios 5.6.12.
however the nagios service wont start:

Code: Select all

[Stopping nagios:kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
done.
Starting nagios: done.
[root@lkennagiosp03 ~]# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

Nagios Core 4.4.5
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 2019-08-20
License: GPL

Website: https://www.nagios.org
Reading configuration data...
Read main config file okay...
Read object config files okay...

Running pre-flight check on configuration data...

Checking objects...
Checked 14 services.
Checked 1 hosts.
Checked 1 host groups.
Checked 0 service groups.
Checked 2 contacts.
Checked 2 contact groups.
Checked 135 commands.
Checked 8 time periods.
Checked 0 host escalations.
Checked 0 service escalations.
Checking for circular paths...
Checked 1 hosts
Checked 0 service dependencies
Checked 0 host dependencies
Checked 8 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors: 0

Things look okay - No serious problems were detected during the pre-flight check

[root@lkennagiosp03 ~]# service nagios status
nagios is not running
[root@lkennagiosp03 ~]# killall -9 nagios
[root@lkennagiosp03 ~]# service nagios start
Starting nagios: done.
[root@lkennagiosp03 ~]# service nagios status
nagios is not running
[root@lkennagiosp03 ~]#

on a side note. the lock file is located in
/var/run/nagios.lock

[scottwilkerson]

What do you see if you run the following

Code: Select all

tail -20 /usr/local/nagios/var/nagios.log

[benhank]

here you go

Code: Select all

tail -20 /usr/local/nagios/var/nagios.log
[1585855447] ndomod registered for program status data'
[1585855447] ndomod registered for host status data'
[1585855447] ndomod registered for service status data'
[1585855447] ndomod registered for adaptive program data'
[1585855447] ndomod registered for adaptive host data'
[1585855447] ndomod registered for adaptive service data'
[1585855447] ndomod registered for external command data'
[1585855447] ndomod registered for aggregated status data'
[1585855447] ndomod registered for retention data'
[1585855447] ndomod registered for contact data'
[1585855447] ndomod registered for contact notification data'
[1585855447] ndomod registered for acknowledgement data'
[1585855447] ndomod registered for state change data'
[1585855447] ndomod registered for contact status data'
[1585855447] ndomod registered for adaptive contact data'
[1585855447] Event broker module '/usr/local/nagios/bin/ndomod.o' initialized successfully.
[1585855447] Warning: Host performance file processing command 'process-host-perfdata-file' was not found - host performance data file will not be processed!
[1585855447] Warning: Service performance file processing command 'process-service-perfdata-file' was not found - service performance data file will not be processed!
[1585855447] Successfully launched command file worker with pid 16355
[1585859047] Auto-save of retention data completed successfully.

[scottwilkerson]

Did you have a ramdisk setup on the old system?

What is the output of the following

Code: Select all

ps -ef|grep nagios
grep nagios /var/log/messages|tail -20

[benhank]

no on the ramdisk.

Code: Select all

ps -ef|grep nagios
nagios    1050  1035  0 17:30 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1
nagios    1051  1037  0 17:30 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1
nagios    1053  1040  0 17:30 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1
nagios    1054  1038  0 17:30 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1
nagios    1055  1039  0 17:30 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
nagios    1060  1050  1 17:30 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php
nagios    1061  1051  1 17:30 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php
nagios    1065  1053  2 17:30 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php
nagios    1066  1054  3 17:30 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php
nagios    1067  1055  3 17:30 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php
postgres  1072  2437  0 17:30 ?        00:00:00 postgres: nagiosxi nagiosxi ::1(56558) idle
postgres  1073  2437  0 17:30 ?        00:00:00 postgres: nagiosxi nagiosxi ::1(56560) idle
postgres  1090  2437  0 17:30 ?        00:00:00 postgres: nagiosxi nagiosxi ::1(56566) idle
postgres  1159  2437  0 17:30 ?        00:00:00 postgres: nagiosxi nagiosxi ::1(56570) idle
postgres  1166  2437  0 17:30 ?        00:00:00 postgres: nagiosxi nagiosxi ::1(56572) idle
nagios    1314  1065  0 17:30 ?        00:00:00 sh -c /usr/bin/iostat -c 5 2 | tail --lines=2 | head --lines=1 | awk '{ print $1,$2,$3,$4,$5,$6 }'
nagios    1315  1314  0 17:30 ?        00:00:00 /usr/bin/iostat -c 5 2
nagios    1316  1314  0 17:30 ?        00:00:00 tail --lines=2
nagios    1317  1314  0 17:30 ?        00:00:00 head --lines=1
nagios    1318  1314  0 17:30 ?        00:00:00 awk { print $1,$2,$3,$4,$5,$6 }
root      1320  1213  0 17:30 pts/1    00:00:00 grep nagios
nagios    2545     1  0 14:50 ?        00:00:01 /usr/local/pnp4nagios/bin/npcd -d -f /usr/local/pnp4nagios/etc/npcd.cfg
nagios    2583     1  0 14:50 ?        00:00:00 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
postgres  3898  2437  0 14:53 ?        00:00:05 postgres: nagiosxi nagiosxi ::1(53214) idle
postgres  3901  2437  0 14:53 ?        00:00:04 postgres: nagiosxi nagiosxi ::1(53216) idle
postgres  3904  2437  0 14:53 ?        00:00:05 postgres: nagiosxi nagiosxi ::1(53218) idle
postgres  3905  2437  0 14:53 ?        00:00:05 postgres: nagiosxi nagiosxi ::1(53220) idle
postgres  3910  2437  0 14:53 ?        00:00:05 postgres: nagiosxi nagiosxi ::1(53222) idle
postgres  3913  2437  0 14:53 ?        00:00:05 postgres: nagiosxi nagiosxi ::1(53224) idle
postgres  3914  2437  0 14:53 ?        00:00:06 postgres: nagiosxi nagiosxi ::1(53226) idle
postgres  3944  2437  0 14:53 ?        00:00:04 postgres: nagiosxi nagiosxi ::1(53228) idle
postgres  7874  2437  0 15:03 ?        00:00:05 postgres: nagiosxi nagiosxi ::1(53444) idle
nagios   16325     1  0 15:24 ?        00:00:01 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios   16327 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16328 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16329 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16330 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16331 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16332 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16333 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16334 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16335 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16336 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16337 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16338 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16339 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16340 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16341 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16342 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16343 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16344 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16345 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16346 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16347 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16348 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16349 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16350 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   16351  2583  0 15:24 ?        00:00:00 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios   16352 16351  0 15:24 ?        00:00:04 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios   16355 16325  0 15:24 ?        00:00:00 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
postgres 28498  2437  0 17:17 ?        00:00:01 postgres: nagiosxi nagiosxi ::1(56296) idle
postgres 29043  2437  0 17:18 ?        00:00:00 postgres: nagiosxi nagiosxi ::1(56318) idle
[root@lkennagiosp03 ~]# grep nagios /var/log/messages|tail -20
Apr  2 17:26:27 lkennagiosp03 ndo2db: Trimming timedevents.
Apr  2 17:26:27 lkennagiosp03 ndo2db: Trimming systemcommands.
Apr  2 17:26:27 lkennagiosp03 ndo2db: Trimming servicechecks.
Apr  2 17:26:27 lkennagiosp03 ndo2db: Trimming hostchecks.
Apr  2 17:26:27 lkennagiosp03 ndo2db: Trimming eventhandlers.
Apr  2 17:27:28 lkennagiosp03 ndo2db: Trimming timedevents.
Apr  2 17:27:28 lkennagiosp03 ndo2db: Trimming systemcommands.
Apr  2 17:27:28 lkennagiosp03 ndo2db: Trimming servicechecks.
Apr  2 17:27:28 lkennagiosp03 ndo2db: Trimming hostchecks.
Apr  2 17:27:28 lkennagiosp03 ndo2db: Trimming eventhandlers.
Apr  2 17:28:33 lkennagiosp03 ndo2db: Trimming timedevents.
Apr  2 17:28:33 lkennagiosp03 ndo2db: Trimming systemcommands.
Apr  2 17:28:33 lkennagiosp03 ndo2db: Trimming servicechecks.
Apr  2 17:28:33 lkennagiosp03 ndo2db: Trimming hostchecks.
Apr  2 17:28:33 lkennagiosp03 ndo2db: Trimming eventhandlers.
Apr  2 17:29:36 lkennagiosp03 ndo2db: Trimming timedevents.
Apr  2 17:29:36 lkennagiosp03 ndo2db: Trimming systemcommands.
Apr  2 17:29:36 lkennagiosp03 ndo2db: Trimming servicechecks.
Apr  2 17:29:36 lkennagiosp03 ndo2db: Trimming hostchecks.
Apr  2 17:29:36 lkennagiosp03

[scottwilkerson]

Hmm, this appears like it is running, how about this

Code: Select all

ls -l /var/run/nagios.lock
cat /var/run/nagios.lock

[benhank]

there yah go:

Code: Select all

[root@lkennagiosp03 ~]# ls -l /var/run/nagios.lock
-rw-r--r-- 1 root root 6 Apr  2 15:24 /var/run/nagios.lock
[root@lkennagiosp03 ~]# cat /var/run/nagios.lock
16325
[root@lkennagiosp03 ~]#

[scottwilkerson]

Well this all looks correct, and nagios seems running, I am confused why the status is showing what it is

Can you share your nagios.cfg?

[benhank]

so there are two lock files on the server....
/var/run/nagios.lock which contains a pid #
and
/usr/local/nagios/var/nagios.lock which is blank

and sadly no I cant send the profile because:

Code: Select all

PROFILE BUILD FAILED

Array
(
)

CODE: 1

I'm going to try another restore and see if that fixes this.

[lmiltchev]

Try fixing the issue with the profile by following the steps, outlined in the KB article below:

https://support.nagios.com/kb/article/n ... d-533.html