DB Alert Not Generated

[wiproltdwiv]

Dear Team,
We are monitoring Tablespace usage for few DBs in Nagios, we have set 30 mins check interval, retry interval of 1 mins and 5 check attempts. notification period & check period is 24/7.

the issue is like, 1 tablespace usage of the DB has crossed the defined treshold at a particular time but we have not received any alert for it in a whole day. we used to get alerts for the same table earlier. but now its not.
please confirm what may be the issue. configuration part screenshots are attached.

[tmcdonald]

As shown in your first screenshot you have notifications disabled. Is there a reason that is off?

[wiproltdwiv]

Dear Team,
We are using external handler for host and services, we are not using any mail or sms notification.
hence the notifications are disabled.
please confirm what may be the issue for alert not generated even though respective DB table space usage was high.

Regards,

[tmcdonald]

We are using external handler for host and services, we are not using any mail or sms notification.

That would be my first guess as to where to look. What changes did you make to Nagios? What external handler are you using? If not email or SMS, what is used to send notifications?

[wiproltdwiv]

Hi,
we did some customization as per need, for that
1) we created script for service and host check i.e. alertinserhost.sh.
2) created 2 commands i.e check_event_host & service with below syntax
$USER1$/alertinserthost.sh $HOSTSTATE$ $HOSTSTATETYPE$ $HOSTATTEMPT$ $HOSTNAME$ "$HOSTGROUPNAMES$" $HOSTSTATEID$ $HOSTOUTPUT$
$USER1$/alertinsertservice.sh $SERVICESTATE$ $SERVICESTATETYPE$ $SERVICEATTEMPT$ $HOSTNAME$ "$HOSTGROUPNAMES$" "$SERVICEDESC$" $SERVICESTATEID$ "$SERVICEOUTPUT$"
3) created database i.e alertintegration.

i have attached host script, please check based on it is working and we are getting notification.

[tmcdonald]

i have attached host script, please check based on it is working and we are getting notification.

So wait, are you getting alerts or not? Looking through your script there is nothing in there that directly sends an alert. From my understanding all it does is insert a new record into your alerttrack database. If the entries are properly being inserted into the database then it is not a problem with Nagios.

[wiproltdwiv]

this script inserting alert into alertrack db. but we did not get check result, so that my concern what can be issue for check not happened.

[tmcdonald]

Your checks are running check_oracle_health but you are saying you have issues with your custom alertinserthost.sh script. Where are you running alertinserthost.sh in this scenario?

[wiproltdwiv]

my concern is when any check happen that status should be inserted in nagios db. after that only our script will start work. so may be check not happened and cause of script could not found any status in nagios db. so is it any possibility to happen like this incident.

[sreinhardt]

What actually calls your alert script to run? It seems like you intend it to be a event handler, but this service has none defined, not that it couldn't be in a template. At this point it sounds like the flow of events would be:

service check returns NOT ok.
calls event handler assigned to service, to insert into alertdb
nagios returns to normal work and continues checking services.

Does that sound correct? We really need an outline of how you expect this to work, what is happening presently, and what is not happening.