Weird nagios issues

[t3dus]

I have some weird nagios issues going on.

First this nagios server is running on Ubuntu 16.04 on a VPS server in California.
Nagios version is 4.3.4.

The server was working great up until the past few days where now it shows two website keywords down as critical all the time, doesn't send out e-mail notifications and I'm pretty sure it stopped actually checking websites all together.

When I try to issue a command to have it recheck these two critical services or acknowledge it just sits there and won't let me submit the command.

I'm not sure what happened but it just fails to work anymore.

[kyang]

Could you give me the output of these commands?

Code: Select all

df -h
df -i

Also the permissions on these files in this dir.

Code: Select all

ls -la /usr/local/nagios/var/rw

Thanks!

[t3dus]

Here you go.

Code: Select all

# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/simfs       30G  1.3G   29G   5% /
devtmpfs        1.0G     0  1.0G   0% /dev
tmpfs           1.0G     0  1.0G   0% /dev/shm
tmpfs           1.0G   22M 1003M   3% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           1.0G     0  1.0G   0% /sys/fs/cgroup
none            1.0G     0  1.0G   0% /run/shm

Code: Select all

# df -i
Filesystem       Inodes IUsed    IFree IUse% Mounted on
/dev/simfs     15728640 47412 15681228    1% /
devtmpfs         262144    52   262092    1% /dev
tmpfs            262144     1   262143    1% /dev/shm
tmpfs            262144   170   261974    1% /run
tmpfs            262144     6   262138    1% /run/lock
tmpfs            262144    10   262134    1% /sys/fs/cgroup
none             262144     1   262143    1% /run/shm

Code: Select all

# ls -la /usr/local/nagios/var/rw
total 8
drwxrwsr-x 2 nagios nagcmd 4096 Feb  5 13:32 .
drwxrwxr-x 5 nagios nagios 4096 Feb  4 03:30 ..
prw-rw---- 1 nagios nagcmd    0 Feb  1 16:39 nagios.cmd
srw-rw---- 1 nagios nagcmd    0 Feb  5 13:32 nagios.qh

[t3dus]

I think I have resolved this.

Turns out there was a contact group named "mattsites" that had no contacts defined for it.

It created these errors below

Code: Select all

Feb 05 13:32:43 nagios2 nagios[7870]: Error: Could not find any contact matching 'matt' (config file '/usr/local/nagios/etc/objects/contacts.cfg', starting on line 88)
Feb 05 13:32:43 nagios2 nagios[7870]: Error: Failed to expand contacts for contactgroup 'mattsites' (config file '/usr/local/nagios/etc/objects/contacts.cfg', starting at line 88
Feb 05 13:32:43 nagios2 nagios[7870]: Bailing out due to one or more errors encountered in the configuration files. Run Nagios from the command line with the -v option to verify
Feb 05 13:32:43 nagios2 nagios[7870]: Event broker module 'NERD' deinitialized successfully.

Upon deleting the contact group it started working again.

[kyang]

Sounds good!

Glad you found the issue!

Did you have any more questions on this or are we okay to lock this up?