postponed alert notification

[slansing]

Great! Keep me/us up to date with your findings!

[MichielvM]

The hunch was that it had something to do with exclusions in time-periods.
So we made a new timeperiod without exclusions.

example:
Old:
monday - 00:00-24:00
exclusion monday - 08:00-17:30

New:
monday - 00:00-08:00,17:30-24:00
exclusion empty

I would say: different approach, with the same effect! Right?
Well, In fact it had some effect, it showed up in home - notifications, but still no alert e-mail was sent out.
While another host - with the same time-period - started sending alerts after 17:30.
Both critical states happened before 17:30!

I will send a pm with profile and the two hostnames.

[slansing]

What in the world... still seeing that odd behavior? And yes, prior to exclusions that is how you had to do it, same effect, but one less step. Did you get that PM out?

[MichielvM]

I bumped into another one on another Nagios Xi server.

A disk has low freespace and a few processes which cause it to flap around my given tresholds. First of: flap detection is not enabled nor detected.
Here's a short time line of what happens:

Notifications
20.29 W
23.28 W
23.48 O
23.57 W
00.32 C
Here is where notification log stops. No other alerts were sent. State history goes on though...
00.57 W
01.02 C
02.17 W
02.52 O

Check interval is 5 min
Retry interval is 1 min
Max attempts is 5
Notification options are C / U / R
Notification interval is set at 60 min.

I would have expected a Critical Service alert around 01.02 and an OK around 02.52

[slansing]

So I took a look at the host you mentioned in your PM and everything looks fine, the time period looks good and should start your notifications at 17:30, and end at 08:00 the following day. The only thing that jumped out was the notification interval being 3900 minutes. This would mean you only get one notification for that problem. Is it possible this host recovered before 17:30'ish? On your example above, is this using the same notification time periods?

Your nagiosadmin_notification_times, and Team 1 times look fine as well, they are all 24x7. Were you examining the XI server's notification log, or your mail server's log?

[MichielvM]

Hi all, sorry for the late reply, busy busy, holiday...

alas we fixed it!

In the old situation we had time periods spanning 24/7 and used exclusions for office hours. We don't want mail alerts when we're at the office.
We took the exclusions out and altered the time periods like so:
monday 00:00-08:00,17:30-24:00

after this change, the misbehaviour stopped. Something for the developers to find out???

[tmcdonald]

Glad to see it's working at any rate. What version of XI is this?

[MichielvM]

Is it possible this host recovered before 17:30'ish?

No, it hadn't.

On your example above, is this using the same notification time periods?

Yes, they are

Your nagiosadmin_notification_times, and Team 1 times look fine as well, they are all 24x7. Were you examining the XI server's notification log, or your mail server's log?

Both

[MichielvM]

Glad to see it's working at any rate. What version of XI is this?

At the time it was 2012R2,5 running on redhat enterprise 6.5. We've since upgraded XI to 2,9.
It still holds up. Meaning that We've not encountered this behavior after applying the modified time period.
Looks like a few servers had troubles with exclusions. Under certain circumstances, because the same setup worked flawlessly elsewhere.

[lmiltchev]

MichielvM, can we lock this topic as resolved or you still need help?