NagiosXI Sporadic Notifications

[blariv]

Hi,

we seem to be missing notifications, the event log says it is sending them but we never receive them, this is not happening on all systems though. I've had our mail team trace the emails and they are not being generated from Nagios. The way I've been able to fix this is by running a repair on the database. There are always a ton of errors.


System:
Nagios XI Version : 5.4.13
nagiosxi.na.hasbro.com 3.10.0-693.11.6.el7.x86_64 x86_64
Red Hat Enterprise Linux Server release 7.4 (Maipo)
Gnome is not installed

[scottwilkerson]

Can you show us a screenshot of the notifications log of ones that aren't being received?

[blariv]

attached

[scottwilkerson]

Do these users have notifications enabled if when they are logged in to XI they click their username (upper right) and go to Notification Preferences?
What is set in the Notification Methods sub-menu?

[blariv]

they all do, this includes myself as well.


Receive alerts via email. checked

notifications are enabled.

[scottwilkerson]

I just caught this I overlooked in your original post

The way I've been able to fix this is by running a repair on the database. There are always a ton of errors.

Is this server being shutdown abruptly (not using the CLI), or is it possible it is losing connectivity to the disk? These are the only 2 scenarios where you should be seeing database errors

[blariv]

this server is never shutdown, maybe rebooted once a quarter. it is a VM on redhat with all flash storage. no errors anywhere. Also I could run the DB repair script over and over and it will still find errors.

[scottwilkerson]

How are you seeing error when running the script? What errors are you seeing?

[blariv]

just constant index repairs.

- recovering (with sort) MyISAM-table 'xi_auditlog.MYI'
Data records: 4656
- Fixing index 1
- Fixing index 2
- Fixing index 3
- Fixing index 4
- Fixing index 5

---------

- recovering (with sort) MyISAM-table 'xi_commands.MYI'
Data records: 12
- Fixing index 1

[scottwilkerson]

Those are not errors, the script goes through every index and rebuild (fixing) it.

More notable would be crashed table errors found in the mysqld.log