Email Notifications Logs + Sendmail + not sending

[vnc786]

Hi All
Nagios XI 2011R3.3
NagiosQL 3.0.3
OS=Red Hat 6.3 64bit

Every thing is running fine. Emails notification are getting delivered to respective ID's through Exchange Server

The problem is there is 2 services out of 4568 services (till now) which are not sending email notification when they are going in critical state. This has happed first time since inception also i haven't done any changes before OR after

So how do i know it is in critical state ?
in the monitoring window we can see that as well as in state history it shows that it went in critical state

For last 2 days i am monitoring logs (/var/log/mail) but i am not able to find any entry for that particular time and in (/var/log/messages) it has entry CRITICAL STATE etc etc

nagios is using sendmail

how can i confirm that there is no issue with nagios ?

thanks

[abrist]

The first thing to do is to verify that the CRITICAL states were indeed HARD states.

[vnc786]

hi..
below is the excerpt from State History. in below state can you tell just by veiwing that whether Email Notification is sent. I have mention whether Email is sent or not

2014-01-02 23:32:54 p01 Physical Memory Usage CRITICAL HARD 3 of 3 Physical Memory: 96%used(15682MB/16374MB) (>95%) : CRITICAL Mail not Rec
2014-01-02 21:47:45 p01 Physical Memory Usage WARNING HARD 3 of 3 Physical Memory: 86%used(14161MB/16374MB) (>85%) : WARNING Mail Rec
2014-01-02 01:22:54 p01 Physical Memory Usage OK HARD 3 of 3 Physical Memory: 70%used(11427MB/16374MB) (<85%) : OK Mail Rec

Also I have verify that State is Hard..
Update: how do i verify OR is there any entry made in log files, till now i havent find anything in /var/log/maillog related sendmail (mail not sending)
thanks

[slansing]

Are you only getting single line returns in your maillog? A full message should be something like this:

Jan 13 08:45:02 localhost postfix/pickup[63301]: 4DCF7808EC: uid=0 from=<root>
Jan 13 08:45:02 localhost postfix/cleanup[18727]: 4DCF7808EC: message-id=<[email protected]>
Jan 13 08:45:02 localhost postfix/qmgr[1606]: 4DCF7808EC: from=<[email protected]>, size=879, nrcpt=1 (queue active)
Jan 13 08:45:02 localhost postfix/local[18730]: 4DCF7808EC: to=<[email protected]>, orig_to=<root>, relay=local, delay=0.09, delays=0.06/0.01/0/0.01, dsn=2.0.0, status=sent (delivered to mailbox)
Jan 13 08:45:02 localhost postfix/qmgr[1606]: 4DCF7808EC: removed

In the state history report you can see what state these hosts/services entered into in the past.

[vnc786]

Are you only getting single line returns in your maillog? A full message should be something like this:

Please see the attachment. I think sendmail is working properly because there are no error entry also when i am receiving Critical email for other services/host for which I can see entry(mail header time and to whom send) in /var/log/maillog clearly with no error.

In the state history report you can see what state these hosts/services entered into in the past.

please see attachment state history for this month. i have compare the mail recieved in outlook with state history

[abrist]

Can we see the configuration for this ram check? (service check, host check, and command definition)

[vnc786]

SERVICE MANAGEMENT
Comman Settings
plugin name: check_snmp_storage.pl (-C snmpname --v2c -m 'Physical Memory' -w 80 -c 90 -f)
snmp version : v2c
host: (itself)
hostgroups:no
Service group: 2 groups

Check Settings
max check attempts: 3
Retry interval: 5
Check interval: 15
rest of all : default no changes made

Alert Settings
contacts: nagiosadmin
contactsgroups: 4 groups (24x7 )
notification period: 24x7
notification interval: 60
notification enabled: on
rest of all : default no changes made

Misc Setting
nothing

HOST MANAGEMENT
Common Settings
parents: localhost
hostgroup: 1 group

Check Settings
max check attempts: 5
Retry interval: 1
Check interval: 5
rest of all : default no changes made

Alert Settings
contacts: nagiosadmin
contactsgroups: 4 groups (24x7 )
notification period: 24x7
notification interval: 60
notification enabled: on
rest of all : default no changes made

[sreinhardt]

I think abrist actually meant to ask to see the configuration for the service check, should be located in /usr/local/nagios/etc/services/[hostname].cfg, of course change out hostname for whatever host or IP you used.

[vnc786]

"sreinhardt" /usr/local/nagios/etc/services/[hostname].cfg, of course change out hostname for whatever host or IP you used.

PFA.
Just want to confirm that there is some issue with this services/host OR with nagios, since you can see that only after moving from warning to critical it is not sending email still.

[sreinhardt]

I believe abrist and I are both thinking that this is likely an issue with something restricting what notifications are being sent out for these services. Can you pm one of us your latest config snapshot please? That way we can dig through it without continually asking for new files.