Notification Mess!

[rkane]

Need help, for some reason in the last 48 hours my Nagios instance has gone nuts with notifications. I'm sure it's something I messed up. Would love a hand running it down. Some things I've observed:

- I'm getting "Flapping Stop" notifications with no other associated notifications (would assume I'd see some other notifications and then a flapping start)
- In Nagios I go to look at notification history (either through reports or through Incident management) and I don't see most of the notifications I've received. I checked the filters (24hrs, no host / group / etc selected)

OK - point me in a direction please.

[rkane]

Some logs attached

[rkane]

More logs

[rkane]

Checked some state histories for these host / services I'm getting ghost "flapping stop" notifications for. They have no reported state changes that correspond with the flapping notifications.

[rkane]

added

Code: Select all

enable_flap_detection=0

to my config file, still getting phantom flapping notifications

[benjaminsmith]

Hi Rkane,

If you made a change to a configuration file, you'll need to restart nagios to take effect. Try stopping all the processes and restarting:

Code: Select all

service nagios stop
killall -9 nagios
service nagios start

Have you tried to rollback to a previous configuration to see if that takes care of the issue?

[rkane]

I have restarted all the services. I think I've figured out what the issue is....how do I flush all messages (notifications) out of the Nagios queue? Traced a notification I got and it is over a day old....

[benjaminsmith]

Good to hear. You'll have to truncate the database to clear the messages ( see commands below).

Code: Select all

echo "truncate table xi_events; truncate table xi_meta; truncate table xi_eventqueue;" | mysql -uroot -pnagiosxi nagios

[rkane]

ERROR 1146 (42S02) at line 1: Table 'nagios.xi_events' doesn't exist

Good to hear. You'll have to truncate the database to clear the messages ( see commands below).

Code: Select all

echo "truncate table xi_events; truncate table xi_meta; truncate table xi_eventqueue;" | mysql -uroot -pnagiosxi nagios

[scottwilkerson]

Sorry about that they missed 2 characters

Code: Select all

echo "truncate table xi_events; truncate table xi_meta; truncate table xi_eventqueue;" | mysql -uroot -pnagiosxi nagiosxi