Nagios Support Forum

Hi,

We've been using Nagios for 4 years without any issues.

Yesterday morning (Sunday 07:30), we had a service outage on a web application. Nagios detected the outage and logged various SERVICE and HOST ALERTS, but for some reason, didn't send any SERVICE NOTIFICATIONS. The first SERVICE NOTIFICATION was at 16:10.

Initially, we thought this was some problem with our config, but when we tried to re-produce the error today, we couldn't. Each time we initiate a failure on the web application, Nagios sends out SERVICE NOTIFICATIONS as we would expect. Furthermore, during yesterdays outage, Nagios did send out SERVICE NOTIFICATIONS for other services that experienced brief outages.

We've been through the Nagios logfiles with a fine tooth comb. Nagios was not restarted or re-configured between 07:30 and 16:00 on the day in question.

Has anyone experienced anything like this before? Do you have any pointers?

Are your contacts configured to only be notified during a specific timeperiod (using the host_notification_period and service_notification_period directives)?

agriffin wrote:Are your contacts configured to only be notified during a specific timeperiod (using the host_notification_period and service_notification_period directives)?

No, we don't use these directives at all, as we have a 24/7 support operation.

Our nagios installation has been working seamlessly for years, so we're really stumped by this. We even re-enacted the failure (albeit later in the day) and the notifications issued.

From what I can see, if a HOST ALERT occurs, a HOST NOTIFICATION is sent, and all SERVICE NOTIFICATIONS are suppressed after this. We had a HOST ALERT at 07:36, which came back at 16:02, after which SERVICE NOTIFICATIONS started again.

It looks as if Nagios thought a HOST NOTIFICATION was sent, even though one wasn't, and then surpressed SERVICE NOTIFICATIONS until such time as the HOST ALERT cleared.

Are these services set up with any dependencies?