Page 1 of 1

nagios.cmd missing

Posted: Wed Aug 27, 2014 12:52 pm
by kbackman
Hi all,

Okay so I am running into an issue,

ls -al /usr/local/nagios/var/rw/nagios.cmd
ls: cannot access /usr/local/nagios/var/rw/nagios.cmd: No such file or directory

This is causing issues with the nagiosxi dash board and will prevent users from acknowledging alerts (to name but one issue).

Restarting the nagios service does not resolve this issue nor does force-reload. Permissions and users on the file system are correct:
$ ls -al /usr/local/nagios/var/rw/
total 8
drwxrwsr-x 2 nagios nagcmd 4096 Aug 27 17:27 .
drwxrwxr-x 7 nagios nagios 4096 Aug 27 17:27 ..
srw-rw---- 1 nagios nagcmd 0 Aug 27 17:27 nagios.qh

Current version of nagios is
Nagios XI 2014R1.3
and it's running on a centOS 2.6.32-431.3.1.el6.x86_64

Is there anything else I can check, the /etc/init.d/nagios looks okay and hasn't been changed.

Thanks.

Re: nagios.cmd missing

Posted: Wed Aug 27, 2014 1:04 pm
by tmcdonald
Are you sure the nagios service is running?

Code: Select all

service nagios status

Re: nagios.cmd missing

Posted: Wed Aug 27, 2014 1:43 pm
by kbackman
Apologies looks like I forgot a paragraph of my question. Reason I was looking at the /etc/init.d/nagios script because the service doesn't want to start and logs aren't showing any errors. This is why I was asking if there is anything I could check.

I also check to see what processes were running in case something was preventing nagios from starting.

$ ps -ef | grep nagios | grep -v grep
nagios 1765 1 0 15:21 ? 00:00:00 /usr/local/nagios/bin/npcd -d -f /usr/local/nagios/etc/pnp/npcd.cfg
nagios 1950 1 0 15:22 ? 00:00:00 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
postgres 2866 1631 0 15:23 ? 00:00:02 postgres: nagiosxi nagiosxi [local] idle
postgres 2868 1631 0 15:23 ? 00:00:02 postgres: nagiosxi nagiosxi [local] idle
postgres 2894 1631 0 15:24 ? 00:00:02 postgres: nagiosxi nagiosxi [local] idle
postgres 2895 1631 0 15:24 ? 00:00:02 postgres: nagiosxi nagiosxi [local] idle
postgres 2896 1631 0 15:24 ? 00:00:02 postgres: nagiosxi nagiosxi [local] idle
postgres 2930 1631 0 15:24 ? 00:00:02 postgres: nagiosxi nagiosxi [local] idle
postgres 3015 1631 0 15:24 ? 00:00:02 postgres: nagiosxi nagiosxi [local] idle
postgres 3016 1631 0 15:24 ? 00:00:02 postgres: nagiosxi nagiosxi [local] idle
postgres 3893 1631 0 15:28 ? 00:00:02 postgres: nagiosxi nagiosxi [local] idle
postgres 12517 1631 0 16:12 ? 00:00:01 postgres: nagiosxi nagiosxi [local] idle
postgres 14715 1631 0 16:24 ? 00:00:01 postgres: nagiosxi nagiosxi [local] idle
postgres 23307 1631 0 17:09 ? 00:00:00 postgres: nagiosxi nagiosxi [local] idle
nagios 42526 42519 0 18:50 ? 00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1
nagios 42528 42526 0 18:50 ? 00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php
nagios 42529 42520 0 18:50 ? 00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1
nagios 42530 42529 0 18:50 ? 00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php
nagios 42531 42521 0 18:50 ? 00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1
nagios 42532 42531 1 18:50 ? 00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php
nagios 42534 42523 0 18:50 ? 00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1
nagios 42536 42534 1 18:50 ? 00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php
nagios 42540 42522 0 18:50 ? 00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
nagios 42543 42540 1 18:50 ? 00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php
postgres 42547 1631 0 18:50 ? 00:00:00 postgres: nagiosxi nagiosxi [local] idle
postgres 42550 1631 0 18:50 ? 00:00:00 postgres: nagiosxi nagiosxi [local] idle
postgres 42551 1631 0 18:50 ? 00:00:00 postgres: nagiosxi nagiosxi [local] idle
postgres 42553 1631 0 18:50 ? 00:00:00 postgres: nagiosxi nagiosxi [local] idle
postgres 42554 1631 0 18:50 ? 00:00:00 postgres: nagiosxi nagiosxi [local] idle
nagios 42692 42536 0 18:50 ? 00:00:00 sh -c /usr/bin/iostat -c 5 2 | tail --lines=2 | head --lines=1 | awk '{ print $1,$2,$3,$4,$5,$6 }'
nagios 42693 42692 0 18:50 ? 00:00:00 /usr/bin/iostat -c 5 2
nagios 42694 42692 0 18:50 ? 00:00:00 tail --lines=2
nagios 42695 42692 0 18:50 ? 00:00:00 head --lines=1
nagios 42696 42692 0 18:50 ? 00:00:00 awk { print $1,$2,$3,$4,$5,$6 }

Re: nagios.cmd missing

Posted: Wed Aug 27, 2014 3:12 pm
by kbackman
Just to update my issue I checked the apache error logs and found the following error:

File does not exist: /var/www/html/nagiosxo

This makes me think it's a configuration error, however what config file would this be pulled from. A grep of the config files in /user/local doesn't turn anything up.

Re: nagios.cmd missing

Posted: Wed Aug 27, 2014 3:14 pm
by tmcdonald
That would almost certainly be your apache configs. Run the grep against those and you should find your culprit.

Re: nagios.cmd missing

Posted: Thu Aug 28, 2014 9:50 am
by nunes
Hello,

I started looking into this and as far as I can tell, this nagios.cmd is created when Nagios is started. I tried starting Nagios up and did not receive any errors but it does not appear to start. I get "nagios is not running" when I check service status. Looking in the logs I see the following:

Code: Select all

[1409230457] ndomod: NDOMOD 2.0.0 (02-28-2014) Copyright (c) 2009 Nagios Core Development Team and Community Contributors
[1409230457] ndomod: Successfully connected to data sink.  0 queued items to flush.
[1409230457] ndomod registered for process data
[1409230457] ndomod registered for log data'
[1409230457] ndomod registered for system command data'
[1409230457] ndomod registered for event handler data'
[1409230457] ndomod registered for notification data'
[1409230457] ndomod registered for comment data'
[1409230457] ndomod registered for downtime data'
[1409230457] ndomod registered for flapping data'
[1409230457] ndomod registered for program status data'
[1409230457] ndomod registered for host status data'
[1409230457] ndomod registered for service status data'
[1409230457] ndomod registered for adaptive program data'
[1409230457] ndomod registered for adaptive host data'
[1409230457] ndomod registered for adaptive service data'
[1409230457] ndomod registered for external command data'
[1409230457] ndomod registered for aggregated status data'
[1409230457] ndomod registered for retention data'
[1409230457] ndomod registered for contact data'
[1409230457] ndomod registered for contact notification data'
[1409230457] ndomod registered for acknowledgement data'
[1409230457] ndomod registered for state change data'
[1409230457] ndomod registered for contact status data'
[1409230457] ndomod registered for adaptive contact data'
[1409230457] Event broker module '/usr/local/nagios/bin/ndomod.o' initialized successfully.
[1409230457] Error: Module loading failed. Aborting.
I am under the impression that the start process holds up when hitting that module error. I have been unable to locate any error logs so far outside of that one error line that reference the module error.

I looked into the Apache logs, but I did not see any entries today when I started tested this out. I am not sure if Apache would have any hand in the Nagios service not able to run.

Thanks,
Brian

Re: nagios.cmd missing

Posted: Thu Aug 28, 2014 12:12 pm
by slansing
What neb modules do you have defined in your nagios.cfg? You should by default, only have this:

Code: Select all

broker_module=/usr/local/nagios/bin/ndomod.o config_file=/usr/local/nagios/etc/ndomod.cfg
Check through and see if you have any other modules.

Re: nagios.cmd missing

Posted: Thu Aug 28, 2014 12:42 pm
by nunes
Looking in the nagios.cfg I see the broker_module you have listed as well as the following:

Code: Select all

broker_module=/usr/local/nagios/lib/dnxPlugin.so /usr/local/nagios/etc/dnxServer.cfg

Re: nagios.cmd missing

Posted: Thu Aug 28, 2014 12:44 pm
by slansing
Do you know if your team actively relies on DNX? If not, throw a "#" in front of that line, save the file, and restart nagios to see if it starts properly:

Code: Select all

service nagios restart
You can check the same log to see if that resolves the module crash.