Nagios XI problems

[SDohmen]

I have been able to login just fine with that password. I changed the password to something without special chars and it looks like everything works just fine now. Kinda stupid of me not to check this before since on the test install i used a similar password without special chars aswell.

As requested i added the locale data aswell below.

Code: Select all

LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

I will start adding groups etc to see how it goes and respond later on this week.

[SDohmen]

Just a small update. So far all seems to go fine. I have added about 30 hosts/hostgroups/services/commands without a hitch. What i still miss however are the hosts itself in the overview.

I am wondering if this is because we use a distributed monitoring solution where the hosts only get shown after the first result is in or if the host should be visible after i added it? Can someone explain this perhaps.

Also the hostgroups etc arent showing up atm. I tried to use the write config files option in the tools menu but i get the error when i try to restart nagios:

Code: Select all

Nagios daemon is not running, cannot send restart command!

This is a bit strange since the component status says all is running just fine. Also when i do check the running procs on the server it seems to be running (3rd line):

Code: Select all

nagios    1238  0.0  0.0   3196  1868 ?        S    Sep09   0:09 /usr/local/nagios/bin/npcd -d -f /usr/local/nagios/etc/pnp/npcd.cfg
nagios    1281  0.0  0.0   8000   596 ?        Ss   Sep09   0:00 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios    7077  0.0  0.0  17852  1620 ?        Ssl  10:25   0:00 /usr/sbin/nagios -d /etc/nagios/nagios.cfg
nagios    7153  0.0  0.0   2940  1052 ?        Ss   10:26   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1
nagios    7155  3.0  0.3  34012 14660 ?        S    10:26   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php
nagios    7156  0.0  0.0   2940  1052 ?        Ss   10:26   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1
nagios    7157  3.0  0.3  34012 14672 ?        S    10:26   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php
postgres  7158  0.0  0.1  47968  4412 ?        Ss   10:26   0:00 postgres: nagiosxi nagiosxi ::1(39818) idle
postgres  7159  0.0  0.1  47968  4416 ?        Ss   10:26   0:00 postgres: nagiosxi nagiosxi ::1(39819) idle
nagios    7163  0.0  0.0   2940  1048 ?        Ss   10:26   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1
nagios    7166  3.1  0.3  34200 14724 ?        S    10:26   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php
nagios    7169  0.0  0.0   2940  1056 ?        Ss   10:26   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
nagios    7172  3.1  0.3  34372 14936 ?        S    10:26   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php
nagios    7176  0.0  0.0   2940  1052 ?        Ss   10:26   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1
nagios    7177  3.7  0.3  34316 15200 ?        S    10:26   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php
postgres  7193  0.0  0.1  48036  4712 ?        Ss   10:26   0:00 postgres: nagiosxi nagiosxi ::1(39821) idle
nagios    7195  0.0  0.0   2940  1052 ?        Ss   10:26   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1
nagios    7196  4.1  0.4  36536 17100 ?        S    10:26   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php
postgres  7199  0.0  0.1  47968  4476 ?        Ss   10:26   0:00 postgres: nagiosxi nagiosxi ::1(39823) idle
postgres  7204  0.1  0.1  47968  4488 ?        Ss   10:26   0:00 postgres: nagiosxi nagiosxi ::1(39824) idle
postgres  7220  0.0  0.1  48028  4660 ?        Ss   10:26   0:00 postgres: nagiosxi nagiosxi ::1(39827) idle
nagios    7244  0.0  0.0   2940  1052 ?        S    10:26   0:00 sh -c /usr/bin/iostat -c 5 2 | tail --lines=2 | head --lines=1 | awk '{ print $1,$2,$3,$4,$5,$6 }'
nagios    7245  0.0  0.0   2008   680 ?        S    10:26   0:00 /usr/bin/iostat -c 5 2
nagios    7246  0.0  0.0   2004   480 ?        S    10:26   0:00 tail --lines=2
nagios    7247  0.0  0.0   1984   440 ?        S    10:26   0:00 head --lines=1
nagios    7248  0.0  0.0   2500   732 ?        S    10:26   0:00 awk { print $1,$2,$3,$4,$5,$6 }
root      7251  0.0  0.0   4312   744 pts/0    R+   10:26   0:00 grep nagios
postgres 16573  0.0  0.1  47968  5012 ?        Ss   01:39   0:00 postgres: nagiosxi nagiosxi ::1(44154) idle
postgres 17107  0.0  0.1  47968  5064 ?        Ss   09:13   0:00 postgres: nagiosxi nagiosxi ::1(36119) idle
postgres 28412  0.0  0.1  47968  5016 ?        Ss   Sep11   0:02 postgres: nagiosxi nagiosxi ::1(34541) idle
postgres 28466  0.0  0.1  47968  4964 ?        Ss   Sep11   0:02 postgres: nagiosxi nagiosxi ::1(34545) idle
postgres 28515  0.0  0.1  47968  4764 ?        Ss   Sep11   0:02 postgres: nagiosxi nagiosxi ::1(34548) idle
postgres 28585  0.0  0.1  47968  4940 ?        Ss   Sep11   0:02 postgres: nagiosxi nagiosxi ::1(50294) idle
postgres 28702  0.0  0.1  47968  5024 ?        Ss   Sep11   0:02 postgres: nagiosxi nagiosxi ::1(50304) idle
postgres 28749  0.0  0.1  47968  5000 ?        Ss   Sep11   0:02 postgres: nagiosxi nagiosxi ::1(50307) idle
postgres 28802  0.0  0.1  47972  5108 ?        Ss   Sep11   0:02 postgres: nagiosxi nagiosxi ::1(50311) idle
postgres 28924  0.0  0.1  47972  4992 ?        Ss   Sep11   0:02 postgres: nagiosxi nagiosxi ::1(50323) idle

[mguthrie]

Just to clarify, when you add objects in the Core Config Manager, this information is stored in a database. When you either Apply Configuration or use the Write Config Tool this new information gets written to config files that Nagios uses to run it's checks and display in the interface. Changes made in the Core Config Manager won't be seen in the interface until the new configs are written and Nagios is restarted.

What are the results from the following commands?

Code: Select all

killall -9 nagios
service nagios start

[SDohmen]

After trying the command you asked nothing has changed. The host/hostgroups still wont show up in the general screens for it.

On my testing machine where we also had nagiosxi (before we bought it) everything worked fine after applying. The hosts etc also showed up without writing/restarting nagios from the tools menu.

[EDIT]
I just tried a reboot to see if that solves the problem but also here no solution. According to a post on the NagiosQL forum the standard lock file might be a different file. I checked that and it seemed that the standard file that was in the config (nagios.lock) wasn't on the server. When i changed it to the nagios.pid file it still didn't want to restart nagios because the daemon wasn't running.

[mguthrie]

Can you try adding something again, only this time before you Apply Configuration, can you run this from the command-line:

Code: Select all

tail -f /usr/local/nagiosxi/var/cmdsubsys.log | tee output.txt

Then Apply Config. When information has stopped writing out to
the log file (should only be about 5-10 seconds), go ahead and grab that
output.txt file and send it to us.

[SDohmen]

I have done the command as you asked. The first time i did it i didn't get a error. The second time i did it i got the error.

In short the error that i see in the file is:

Code: Select all

Running configuration check...done.
Stopping nagios: done.
rm: cannot remove `/var/run/nagios.pid': Permission denied
rm: cannot remove `/var/lock/subsys/nagios': Permission denied
touch: cannot touch `/var/lock/subsys/nagios': Permission denied
Starting nagios: done.
RESETTING PERMS
SETUID ROOT OK

I have the idea that something went wrong during the install. Even so far that i am wondering if i shouldn't call in support as incident to get it resolved faster (due too the time difference).

[mguthrie]

I'm seeing a couple of problems here, all of which appear to be related to the environment that XI was installed in.

Here's what a typical log should look like for reconfigure_nagios.sh

Code: Select all

Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:80... connected.

here's what I'm seeing on yours:

Code: Select all

Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.

Can you show us the contents of your /etc/hosts file?

The other is here:

Code: Select all

Running configuration check...done.
Stopping nagios: done.
rm: cannot remove `/var/run/nagios.pid': Permission denied
rm: cannot remove `/var/lock/subsys/nagios': Permission denied
touch: cannot touch `/var/lock/subsys/nagios': Permission denied

It's trying to manipulate files in the wrong location. Did you take any steps outside of our documented installation instructions when you installed Nagios XI?

[SDohmen]

As requested the content of the hosts file.

Code: Select all

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

172.16.21.7     ****  central

The 3 stand for out website dns which i prefer not to post for obvious reasons

I am guessing the ::1 line is a wrong one although it was like that after install. Only the last line got added.

For the install itself i havent changed a thing. A co-worker who has alot of linux knowledge did some minor changes like installing vim with some preset settings so it was easier working for me but for the rest nothing has changed. If needed i can setup a remote session if that will help solve the problems.

[mguthrie]

Ok, so first lets check the file locations. I'm wondering if an old nagios.cfg file was used to overwrite what XI uses? Look in /usr/local/nagios/etc/nagios.cfg and verify the following configs.

Code: Select all

command_file=/usr/local/nagios/var/rw/nagios.cmd
debug_file=/usr/local/nagios/var/nagios.debug
log_file=/usr/local/nagios/var/nagios.log
lock_file=/usr/local/nagios/var/nagios.lock

Was Nagios previously installed on this system from yum? The file locations on your install do not match our installer script, but they're consistent with a yum install of Nagios.

Not sure if it helps either, but here's my /etc/hosts file:

Code: Select all

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1               localhost.localdomain localhost
::1             localhost6.localdomain6 localhost6

I'm not a big DNS guy, but I would try resetting the first two lines to look like the above settings.

[SDohmen]

Code: Select all

command_file=/usr/local/nagios/var/rw/nagios.cmd -> /var/spool/nagios/cmd/nagios.cmd
debug_file=/usr/local/nagios/var/nagios.debug -> cant find it
log_file=/usr/local/nagios/var/nagios.log -> /usr/local/nagios/var/nagios.log and /var/log/nagios/nagios.log
lock_file=/usr/local/nagios/var/nagios.lock -> cant find it

Everything related to nagios has been installed by the install script. Right after the install i did wget installer, unpack, ./fullinstall. Only afterwards my co-worker changed some settings. I asked him about the dns settings and from what he said they are fine.

Would it help if you have direct contact with the machine in question so you can troubleshoot right on the machine itself?