Page 1 of 1

Nagios Notifications not being delivered off server.

Posted: Mon Jul 16, 2012 3:55 pm
by amc_oldsarge
I'm not receiving any notifications for services or hosts when an event occurs.

I am using sSMTP to relay messages off my Nagios server to my exchange server (local server). I have tested sSMTP using a test message via Terminal and it gets delivered immediately, so I know that relaying is good.

Nagios also thinks that everything is good. When I look at the Web Interface under notifications and the Event Log no errors are being produced.

But when I look at my mail.err file under /var/log I get the following (MyDomain.com has been substituted for my real domain name):

Jul 16 00:46:34 ENT-NAG-LNX-00 sSMTP[7941]: RCPT TO:<** PROBLEM Host Alert: ENT-ADM-W7-00 is DOWN **@ENT-NAG-LNX-00.MyDomain.com> (501 5.5.4 Invalid Address)
Jul 16 01:30:10 ENT-NAG-LNX-00 sSMTP[10277]: RCPT TO:<** RECOVERY Host Alert: ENT-ADM-W7-00 is UP **@ENT-NAG-LNX-00.MyDomain.com> (501 5.5.4 Invalid Address)



From Notifications in Nagios Web Interface:
----------------------------------------------------------------------------
ENT-ADM-W7-00 N/A HOST UP 07-16-2012 01:30:10 Joe notify-host-by-email PING OK - Packet loss = 0%, RTA = 0.34 ms
ENT-ADM-W7-00 N/A HOST DOWN 07-16-2012 00:46:34 Joe notify-host-by-email (Host Check Timed Out)



From Event Log in Nagios Web Interface:
----------------------------------------------------------------------------
[07-16-2012 01:30:10] HOST NOTIFICATION: Joe;ENT-ADM-W7-00;UP;notify-host-by-email;PING OK - Packet loss = 0%, RTA = 0.34 ms
[07-16-2012 00:46:34] HOST NOTIFICATION: Joe;ENT-ADM-W7-00;DOWN;notify-host-by-email;(Host Check Timed Out)


ssmtp.conf
----------------------------------------------------------------------------
#
# Config file for sSMTP sendmail
#
# The person who gets all mail for userids < 1000
# Make this empty to disable rewriting.
root=postmaster

# The place where the mail goes. The actual machine name is required no
# MX records are consulted. Commonly mailhosts are named mail.domain.com
mailhub=exchange

# Where will the mail seem to come from?
#rewriteDomain=MyDomain.com

# The full hostname
hostname=ENT-NAG-LNX-00.MyDomain.com

# Are users allowed to set their own From: address?
# YES - Allow the user to specify their own From: address
# NO - Use the system generated From: address
FromLineOverride=YES



contacts.cfg
----------------------------------------------------------------------------
define contactgroup{
contactgroup_name nt-admins
alias Windows Administrators
members nagiosadmin,Joe
}




define contact{
contact_name nagiosadmin ; Short name of user
use generic-contact ; Inherit default values from generic-contact template (defined above)
alias Nagios Admin ; Full name of user
email [email protected] ; <<***** CHANGE THIS TO YOUR EMAIL ADDRESS ******
}

# 'Joe' contact definition
define contact{
contact_name Joe
alias Joe Mama
service_notification_period 24x7
host_notification_period 24x7
service_notification_options w,u,c,r
host_notification_options d,u,r
service_notification_commands notify-service-by-email
host_notification_commands notify-host-by-email
email [email protected]
}




commands.cfg
----------------------------------------------------------------------------
# 'notify-host-by-email' command definition
define command{
command_name notify-host-by-email
command_line /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" $CONTACTEMAIL$
}

# 'notify-service-by-email' command definition
define command{
command_name notify-service-by-email
command_line /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$
}


I think that the end of this definition is causing my troubles: ..... **" $CONTACTEMAIL$

But I can't find $CONTACTEMAIL$ defined anywhere.....

HELP Me!!!!!

Re: Nagios Notifications not being delivered off server.

Posted: Mon Jul 16, 2012 5:00 pm
by agriffin
$CONTACTEMAIL$ is a standard Nagios macro derived from the email directive in your contact definitions. So that's not your problem, but I'm not sure why the correct email address isn't being substituted into the command. Try adding 'debug_level=38' to your main nagios.cfg and restart Nagios. Then take a look at your nagios.debug file after it's run for a while and see what's being substituted into your email command. If it doesn't contain any sensitive information it would be helpful if you posted that file here too.

Re: Nagios Notifications not being delivered off server.

Posted: Mon Jul 16, 2012 5:32 pm
by amc_oldsarge
Ok here is my Nagios.debug after manually creating a host alert notification:

[1342476999.367711] [004.0] [pid=27951] Original UID/GID: 0/0
[1342476999.368908] [004.0] [pid=27951] New UID/GID: 1001/1001
[1342477149.219882] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477149.219943] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477149.219962] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477159.247645] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477159.247700] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477159.247714] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477159.247915] [032.0] [pid=27954] ** Service Notification Attempt ** Host: 'ENT-ADM-W7-00', Service: 'Ping', Type: 0, Options: 0, Current State: 2, Last Notification: Wed Dec 31 18:00:00 1969
[1342477159.247947] [032.1] [pid=27954] The host is either down or unreachable, so we won't notify contacts about this service.
[1342477159.247960] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477159.248090] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477159.248113] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477159.248125] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477169.017264] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477169.017310] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477169.017324] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477169.017551] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477169.017576] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477169.017588] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477169.017856] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477169.017886] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477169.017898] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477189.059351] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477189.059404] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477189.059418] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477199.089539] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477199.089585] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477199.089599] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477199.089900] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477199.089927] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477199.089940] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477209.109913] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477209.109963] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477209.109976] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477219.161911] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477219.161964] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477219.161977] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477219.162008] [032.0] [pid=27954] ** Service Notification Attempt ** Host: 'ENT-ADM-W7-00', Service: 'Ping', Type: 0, Options: 0, Current State: 2, Last Notification: Wed Dec 31 18:00:00 1969
[1342477219.162030] [032.1] [pid=27954] The host is either down or unreachable, so we won't notify contacts about this service.
[1342477219.162041] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477229.180707] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477229.180758] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477229.180772] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477239.199540] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477239.199585] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477239.199598] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477279.018123] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477279.018178] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477279.018193] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477279.018226] [032.0] [pid=27954] ** Service Notification Attempt ** Host: 'ENT-ADM-W7-00', Service: 'Ping', Type: 0, Options: 0, Current State: 2, Last Notification: Wed Dec 31 18:00:00 1969
[1342477279.018250] [032.1] [pid=27954] The host is either down or unreachable, so we won't notify contacts about this service.
[1342477279.018262] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477289.044973] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477289.045020] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477289.045033] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477319.091423] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477319.091473] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477319.091487] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477339.182551] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477339.182602] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477339.182616] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477339.182887] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477339.182929] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477339.182942] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477339.182963] [032.0] [pid=27954] ** Service Notification Attempt ** Host: 'ENT-ADM-W7-00', Service: 'Ping', Type: 0, Options: 0, Current State: 2, Last Notification: Wed Dec 31 18:00:00 1969
[1342477339.182984] [032.1] [pid=27954] The host is either down or unreachable, so we won't notify contacts about this service.
[1342477339.182995] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477349.192081] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477349.192123] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477349.192135] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477399.011951] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477399.012008] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477399.012023] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477399.012056] [032.0] [pid=27954] ** Service Notification Attempt ** Host: 'ENT-ADM-W7-00', Service: 'Ping', Type: 0, Options: 0, Current State: 2, Last Notification: Wed Dec 31 18:00:00 1969
[1342477399.012079] [032.1] [pid=27954] The host is either down or unreachable, so we won't notify contacts about this service.
[1342477399.012092] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477409.021635] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477409.021673] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477409.021687] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477449.132430] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477449.132490] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477449.132504] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477459.170197] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477459.170251] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477459.170265] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477459.170297] [032.0] [pid=27954] ** Service Notification Attempt ** Host: 'ENT-ADM-W7-00', Service: 'Ping', Type: 0, Options: 0, Current State: 2, Last Notification: Wed Dec 31 18:00:00 1969
[1342477459.170321] [032.1] [pid=27954] The host is either down or unreachable, so we won't notify contacts about this service.
[1342477459.170333] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477459.170440] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477459.170462] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477459.170473] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477469.189536] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477469.189600] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477469.189613] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477469.189670] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477469.189690] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477469.189700] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477469.189752] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477469.189771] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477469.189780] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477489.230980] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477489.231030] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477489.231044] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477499.263737] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477499.263782] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477499.263797] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477499.264087] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477499.264111] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477499.264124] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477519.062548] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477519.062624] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477519.062643] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477519.062741] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477519.062767] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477519.062781] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477519.062805] [032.0] [pid=27954] ** Service Notification Attempt ** Host: 'ENT-ADM-W7-00', Service: 'Ping', Type: 0, Options: 0, Current State: 2, Last Notification: Wed Dec 31 18:00:00 1969
[1342477519.062829] [032.1] [pid=27954] The host is either down or unreachable, so we won't notify contacts about this service.
[1342477519.062844] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477529.081715] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477529.081789] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477529.081849] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477539.097776] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477539.097895] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477539.097913] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477579.167150] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477579.167179] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477579.167184] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477579.167194] [032.0] [pid=27954] ** Service Notification Attempt ** Host: 'ENT-ADM-W7-00', Service: 'Ping', Type: 0, Options: 0, Current State: 2, Last Notification: Wed Dec 31 18:00:00 1969
[1342477579.167200] [032.1] [pid=27954] The host is either down or unreachable, so we won't notify contacts about this service.
[1342477579.167204] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477589.184617] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477589.184678] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477589.184696] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477619.231594] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477619.231654] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477619.231671] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477639.061366] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477639.061432] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477639.061450] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477639.061528] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477639.061552] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477639.061565] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477639.061588] [032.0] [pid=27954] ** Service Notification Attempt ** Host: 'ENT-ADM-W7-00', Service: 'Ping', Type: 0, Options: 0, Current State: 2, Last Notification: Wed Dec 31 18:00:00 1969
[1342477639.061612] [032.1] [pid=27954] The host is either down or unreachable, so we won't notify contacts about this service.
[1342477639.061626] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477649.071281] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477649.071341] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477649.071359] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477699.136675] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477699.136746] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477699.136764] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477699.136800] [032.0] [pid=27954] ** Service Notification Attempt ** Host: 'ENT-ADM-W7-00', Service: 'Ping', Type: 0, Options: 0, Current State: 2, Last Notification: Wed Dec 31 18:00:00 1969
[1342477699.136844] [032.1] [pid=27954] The host is either down or unreachable, so we won't notify contacts about this service.
[1342477699.136858] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477709.146496] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477709.146556] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477709.146571] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477749.239306] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477749.239378] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477749.239396] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477759.016783] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477759.016852] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477759.016870] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477759.016906] [032.0] [pid=27954] ** Service Notification Attempt ** Host: 'ENT-ADM-W7-00', Service: 'Ping', Type: 0, Options: 0, Current State: 2, Last Notification: Wed Dec 31 18:00:00 1969
[1342477759.016932] [032.1] [pid=27954] The host is either down or unreachable, so we won't notify contacts about this service.
[1342477759.016946] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477759.017059] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477759.017082] [032.1] [pid=27954] This host is in a soft state, so we won't send a notification out.
[1342477759.017096] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477769.036322] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Dec 31 18:00:00 1969
[1342477769.036373] [032.0] [pid=27954] Notification viability test passed.
[1342477769.036391] [032.1] [pid=27954] Current notification number: 1 (incremented)
[1342477769.036407] [032.1] [pid=27954] Host notification will NOT be escalated.
[1342477769.036422] [032.1] [pid=27954] Adding normal contacts for host to notification list.
[1342477769.057726] [032.0] [pid=27954] 1 contacts were notified. Next possible notification time: Mon Jul 16 19:29:29 2012
[1342477769.057782] [032.0] [pid=27954] 1 contacts were notified.
[1342477769.057970] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Mon Jul 16 17:29:29 2012
[1342477769.057999] [032.1] [pid=27954] Its not yet time to re-notify the contacts about this host problem...
[1342477769.058022] [032.1] [pid=27954] Next acceptable notification time: Mon Jul 16 19:29:29 2012
[1342477769.058036] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477769.058095] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Mon Jul 16 17:29:29 2012
[1342477769.058117] [032.1] [pid=27954] Its not yet time to re-notify the contacts about this host problem...
[1342477769.058137] [032.1] [pid=27954] Next acceptable notification time: Mon Jul 16 19:29:29 2012
[1342477769.058150] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477769.058204] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Mon Jul 16 17:29:29 2012
[1342477769.058250] [032.1] [pid=27954] Its not yet time to re-notify the contacts about this host problem...
[1342477769.058271] [032.1] [pid=27954] Next acceptable notification time: Mon Jul 16 19:29:29 2012
[1342477769.058282] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.
[1342477789.095726] [032.0] [pid=27954] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Mon Jul 16 17:29:29 2012
[1342477789.095793] [032.1] [pid=27954] Its not yet time to re-notify the contacts about this host problem...
[1342477789.095819] [032.1] [pid=27954] Next acceptable notification time: Mon Jul 16 19:29:29 2012
[1342477789.095834] [032.0] [pid=27954] Notification viability test failed. No notification will be sent out.

Thanks for the assistance.

Re: Nagios Notifications not being delivered off server.

Posted: Wed Jul 18, 2012 2:06 pm
by agriffin
Unfortunately, that didn't seem to contain the information I hoped it would. Can you set the debug level to -1 instead of 38 and try again? That will output all possible debug information and likely produce much more output than is strictly necessary, but I'm not sure what value is needed exactly to get information on macro expansion.

Re: Nagios Notifications not being delivered off server.

Posted: Wed Jul 18, 2012 5:22 pm
by amc_oldsarge
Ok, here is debug with -1

Code: Select all

[1342648845.199090] [016.1] [pid=2467] Handling check result for service 'RTA_Graph' on host 'ENT-ADM-W7-00'...
[1342648845.199103] [001.0] [pid=2467] handle_async_service_check_result()
[1342648845.199114] [016.0] [pid=2467] ** Handling check result for service 'RTA_Graph' on host 'ENT-ADM-W7-00'...
[1342648845.199126] [016.1] [pid=2467] HOST: ENT-ADM-W7-00, SERVICE: RTA_Graph, CHECK TYPE: Active, OPTIONS: 0, SCHEDULED: Yes, RESCHEDULE: Yes, EXITED OK: Yes, RETURN CODE: 2, OUTPUT: PING CRITICAL - Packet loss = 100%|rta=500.000000ms;100.000000;500.000000;0.000000 pl=100%;20;60;0\n
[1342648845.199154] [016.1] [pid=2467] Service is in a non-OK state!
[1342648845.199166] [016.1] [pid=2467] Host is currently DOWN/UNREACHABLE.
[1342648845.199177] [016.1] [pid=2467] Assuming host is in same state as before...
[1342648845.199201] [032.0] [pid=2467] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Jul 18 15:18:45 2012
[1342648845.199217] [001.0] [pid=2467] check_host_notification_viability()
[1342648845.199228] [001.0] [pid=2467] check_time_against_period()
[1342648845.199246] [001.0] [pid=2467] check_host_dependencies()
[1342648845.199260] [032.1] [pid=2467] Its not yet time to re-notify the contacts about this host problem...
[1342648845.199279] [032.1] [pid=2467] Next acceptable notification time: Wed Jul 18 17:18:45 2012
[1342648845.199292] [032.0] [pid=2467] Notification viability test failed.  No notification will be sent out.
[1342648845.199303] [016.1] [pid=2467] Current/Max Attempt(s): 2/3
[1342648845.199314] [016.1] [pid=2467] Host isn't UP, so we won't retry the service check...
[1342648845.199326] [016.1] [pid=2467] Looking for services to check for predictive dependency checks...
[1342648845.199345] [016.1] [pid=2467] Rescheduling next check of service at Wed Jul 18 17:01:28 2012
[1342648845.199358] [001.0] [pid=2467] get_next_valid_time()
[1342648845.199371] [001.0] [pid=2467] check_time_against_period()
[1342648845.199388] [001.0] [pid=2467] schedule_service_check()
[1342648845.199408] [016.0] [pid=2467] Scheduling a non-forced, active check of service 'RTA_Graph' on host 'ENT-ADM-W7-00' @ Wed Jul 18 17:01:28 2012
[1342648845.199422] [001.0] [pid=2467] reschedule_event()
[1342648845.199433] [001.0] [pid=2467] add_event()
[1342648845.199444] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648845.199456] [064.1] [pid=2467] Making callbacks (type 20)...
[1342648845.199468] [064.1] [pid=2467] Making callbacks (type 13)...
[1342648845.199479] [064.1] [pid=2467] Making callbacks (type 20)...
[1342648845.199490] [001.0] [pid=2467] check_for_service_flapping()
[1342648845.199501] [016.1] [pid=2467] Checking service 'RTA_Graph' on host 'ENT-ADM-W7-00' for flapping...
[1342648845.199513] [016.1] [pid=2467] Service is not flapping (0.00% state change).
[1342648845.199527] [001.0] [pid=2467] check_for_host_flapping()
[1342648845.199538] [016.1] [pid=2467] Checking host 'ENT-ADM-W7-00' for flapping...
[1342648845.199550] [016.1] [pid=2467] Host is not flapping (0.00% state change).
[1342648845.199563] [016.1] [pid=2467] Deleted check result file '/usr/local/nagios/var/spool/checkresults/cCcoevR'
[1342648845.199578] [016.1] [pid=2467] Handling check result for service 'Ping' on host 'ENT-ADM-W7-00'...
[1342648845.199590] [001.0] [pid=2467] handle_async_service_check_result()
[1342648845.199601] [016.0] [pid=2467] ** Handling check result for service 'Ping' on host 'ENT-ADM-W7-00'...
[1342648845.199612] [016.1] [pid=2467] HOST: ENT-ADM-W7-00, SERVICE: Ping, CHECK TYPE: Active, OPTIONS: 0, SCHEDULED: Yes, RESCHEDULE: Yes, EXITED OK: Yes, RETURN CODE: 2, OUTPUT: PING CRITICAL - Packet loss = 100%|rta=500.000000ms;100.000000;500.000000;0.000000 pl=100%;20;60;0\n
[1342648845.199639] [016.1] [pid=2467] Service is in a non-OK state!
[1342648845.199651] [016.1] [pid=2467] Host is currently DOWN/UNREACHABLE.
[1342648845.199662] [016.1] [pid=2467] Assuming host is in same state as before...
[1342648845.199692] [032.0] [pid=2467] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 1, Last Notification: Wed Jul 18 15:18:45 2012
[1342648845.199706] [001.0] [pid=2467] check_host_notification_viability()
[1342648845.199717] [001.0] [pid=2467] check_time_against_period()
[1342648845.199734] [001.0] [pid=2467] check_host_dependencies()
[1342648845.199746] [032.1] [pid=2467] Its not yet time to re-notify the contacts about this host problem...
[1342648845.199765] [032.1] [pid=2467] Next acceptable notification time: Wed Jul 18 17:18:45 2012
[1342648845.199777] [032.0] [pid=2467] Notification viability test failed.  No notification will be sent out.
[1342648845.199788] [016.1] [pid=2467] Current/Max Attempt(s): 2/2
[1342648845.199799] [016.1] [pid=2467] Service has reached max number of rechecks, so we'll handle the error...
[1342648845.199810] [001.0] [pid=2467] check_for_service_flapping()
[1342648845.199820] [016.1] [pid=2467] Checking service 'Ping' on host 'ENT-ADM-W7-00' for flapping...
[1342648845.199832] [016.1] [pid=2467] Service is not flapping (0.00% state change).
[1342648845.199846] [001.0] [pid=2467] check_for_host_flapping()
[1342648845.199857] [016.1] [pid=2467] Checking host 'ENT-ADM-W7-00' for flapping...
[1342648845.199868] [016.1] [pid=2467] Host is not flapping (0.00% state change).
[1342648845.199881] [001.0] [pid=2467] service_notification()
[1342648845.199901] [032.0] [pid=2467] ** Service Notification Attempt ** Host: 'ENT-ADM-W7-00', Service: 'Ping', Type: 0, Options: 0, Current State: 2, Last Notification: Wed Dec 31 18:00:00 1969
[1342648845.199916] [001.0] [pid=2467] check_service_notification_viability()
[1342648845.199927] [001.0] [pid=2467] check_time_against_period()
[1342648845.199945] [001.0] [pid=2467] check_service_dependencies()
[1342648845.199957] [001.0] [pid=2467] check_host_dependencies()
[1342648845.199969] [032.1] [pid=2467] The host is either down or unreachable, so we won't notify contacts about this service.
[1342648845.199981] [032.0] [pid=2467] Notification viability test failed.  No notification will be sent out.
[1342648845.200000] [016.1] [pid=2467] Rescheduling next check of service at Wed Jul 18 17:01:29 2012
[1342648845.200012] [001.0] [pid=2467] get_next_valid_time()
[1342648845.200023] [001.0] [pid=2467] check_time_against_period()
[1342648845.200039] [001.0] [pid=2467] schedule_service_check()
[1342648845.200059] [016.0] [pid=2467] Scheduling a non-forced, active check of service 'Ping' on host 'ENT-ADM-W7-00' @ Wed Jul 18 17:01:29 2012
[1342648845.200073] [001.0] [pid=2467] reschedule_event()
[1342648845.200084] [001.0] [pid=2467] add_event()
[1342648845.200095] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648845.200107] [064.1] [pid=2467] Making callbacks (type 20)...
[1342648845.200118] [064.1] [pid=2467] Making callbacks (type 13)...
[1342648845.200129] [064.1] [pid=2467] Making callbacks (type 20)...
[1342648845.200141] [016.1] [pid=2467] Deleted check result file '/usr/local/nagios/var/spool/checkresults/cv8HyXz'
[1342648845.200155] [016.1] [pid=2467] Handling check result for service 'Memory Free' on host 'ENT-ADM-W7-00'...
[1342648845.200167] [001.0] [pid=2467] handle_async_service_check_result()
[1342648845.200177] [016.0] [pid=2467] ** Handling check result for service 'Memory Free' on host 'ENT-ADM-W7-00'...
[1342648845.200189] [016.1] [pid=2467] HOST: ENT-ADM-W7-00, SERVICE: Memory Free, CHECK TYPE: Active, OPTIONS: 0, SCHEDULED: Yes, RESCHEDULE: Yes, EXITED OK: Yes, RETURN CODE: 0, OUTPUT: No counter specified\n
[1342648845.200209] [016.1] [pid=2467] Service is OK.
[1342648845.200220] [016.1] [pid=2467] Host is NOT UP, so we'll check it to see if it recovered...
[1342648845.200232] [001.0] [pid=2467] run_async_host_check_3x()
[1342648845.200243] [016.0] [pid=2467] ** Running async check of host 'ENT-ADM-W7-00'...
[1342648845.200255] [001.0] [pid=2467] check_host_check_viability_3x()
[1342648845.200266] [001.0] [pid=2467] check_time_against_period()
[1342648845.200277] [001.0] [pid=2467] check_host_dependencies()
[1342648845.200302] [064.1] [pid=2467] Making callbacks (type 14)...
[1342648845.200314] [016.0] [pid=2467] Checking host 'ENT-ADM-W7-00'...
[1342648845.200325] [001.0] [pid=2467] adjust_host_check_attempt_3x()
[1342648845.200349] [001.0] [pid=2467] get_raw_command_line_r()
[1342648845.200365] [001.0] [pid=2467] process_macros_r()
[1342648845.200376] [2048.1] [pid=2467] **** BEGIN MACRO PROCESSING ***********
[1342648845.200387] [2048.1] [pid=2467] Processing: '$USER1$/check_ping -H $HOSTADDRESS$ -w 3000.0,80% -c 5000.0,100% -p 5'
[1342648845.200417] [2048.1] [pid=2467]   Done.  Final output: '/usr/local/nagios/libexec/check_ping -H 10.1.7.8 -w 3000.0,80% -c 5000.0,100% -p 5'
[1342648845.200430] [2048.1] [pid=2467] **** END MACRO PROCESSING *************
[1342648845.200505] [016.1] [pid=2467] Check result output will be written to '/usr/local/nagios/var/spool/checkresults/checkEAH0zq' (fd=7)
[1342648845.200576] [064.1] [pid=2467] Making callbacks (type 14)...
[1342648845.201708] [001.0] [pid=8887] process_macros_r()
[1342648845.201793] [001.0] [pid=8887] process_macros_r()
[1342648845.201862] [2048.1] [pid=8887] **** BEGIN MACRO PROCESSING ***********
[1342648845.201875] [2048.1] [pid=8887] Processing: '## http://'
[1342648845.201890] [2048.1] [pid=8887]   Done.  Final output: '## http://'
[1342648845.201903] [2048.1] [pid=8887] **** END MACRO PROCESSING *************
[1342648845.201923] [001.0] [pid=8887] process_macros_r()
[1342648845.205130] [016.1] [pid=2467] Service did not change state.
[1342648845.205236] [016.1] [pid=2467] Rescheduling next check of service at Wed Jul 18 17:05:42 2012
[1342648845.205252] [001.0] [pid=2467] get_next_valid_time()
[1342648845.205264] [001.0] [pid=2467] check_time_against_period()
[1342648845.205288] [001.0] [pid=2467] schedule_service_check()
[1342648845.205308] [016.0] [pid=2467] Scheduling a non-forced, active check of service 'Memory Free' on host 'ENT-ADM-W7-00' @ Wed Jul 18 17:05:42 2012
[1342648845.205325] [001.0] [pid=2467] reschedule_event()
[1342648845.205337] [001.0] [pid=2467] add_event()
[1342648845.205351] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648845.205363] [064.1] [pid=2467] Making callbacks (type 20)...
[1342648845.205376] [064.1] [pid=2467] Making callbacks (type 13)...
[1342648845.205387] [064.1] [pid=2467] Making callbacks (type 20)...
[1342648845.205400] [001.0] [pid=2467] check_for_service_flapping()
[1342648845.205410] [016.1] [pid=2467] Checking service 'Memory Free' on host 'ENT-ADM-W7-00' for flapping...
[1342648845.205422] [016.1] [pid=2467] Service is not flapping (0.00% state change).
[1342648845.205439] [001.0] [pid=2467] check_for_host_flapping()
[1342648845.205450] [016.1] [pid=2467] Checking host 'ENT-ADM-W7-00' for flapping...
[1342648845.205469] [016.1] [pid=2467] Host is not flapping (0.00% state change).
[1342648845.205485] [016.1] [pid=2467] Deleted check result file '/usr/local/nagios/var/spool/checkresults/cIs5hIf'
[1342648845.205504] [016.0] [pid=2467] Finished reaping 4 check results
[1342648845.205515] [001.0] [pid=2467] reap_check_results() end
[1342648845.205526] [001.0] [pid=2467] handle_timed_event() end
[1342648845.205537] [001.0] [pid=2467] reschedule_event()
[1342648845.205547] [001.0] [pid=2467] add_event()
[1342648845.205560] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648845.205579] [008.1] [pid=2467] ** Event Check Loop
[1342648845.205600] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:45 2012
[1342648845.205619] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648845.205631] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648845.205642] [001.0] [pid=2467] handle_timed_event() start
[1342648845.205653] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648845.205671] [008.0] [pid=2467] ** Timed Event ** Type: 8, Run Time: Wed Jul 18 17:00:45 2012
[1342648845.205684] [008.0] [pid=2467] ** Status Data Save Event
[1342648845.205695] [064.1] [pid=2467] Making callbacks (type 25)...
[1342648845.205706] [001.0] [pid=2467] save_status_data()
[1342648845.208054] [064.1] [pid=2467] Making callbacks (type 25)...
[1342648845.208081] [001.0] [pid=2467] handle_timed_event() end
[1342648845.208094] [001.0] [pid=2467] reschedule_event()
[1342648845.208106] [001.0] [pid=2467] add_event()
[1342648845.208118] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648845.208130] [008.1] [pid=2467] ** Event Check Loop
[1342648845.208158] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:45 2012
[1342648845.208181] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648845.208194] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648845.208206] [001.0] [pid=2467] handle_timed_event() start
[1342648845.208217] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648845.208237] [008.0] [pid=2467] ** Timed Event ** Type: 1, Run Time: Wed Jul 18 17:00:45 2012
[1342648845.208250] [008.0] [pid=2467] ** External Command Check Event
[1342648845.208261] [001.0] [pid=2467] check_for_external_commands()
[1342648845.208272] [001.0] [pid=2467] handle_timed_event() end
[1342648845.208284] [001.0] [pid=2467] reschedule_event()
[1342648845.208295] [001.0] [pid=2467] add_event()
[1342648845.208306] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648845.208317] [008.1] [pid=2467] ** Event Check Loop
[1342648845.208336] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648845.208356] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648845.208369] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648845.208381] [001.0] [pid=2467] check_for_external_commands()
[1342648845.208393] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648845.458537] [008.1] [pid=2467] ** Event Check Loop
[1342648845.458656] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648845.458680] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648845.458697] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648845.458710] [001.0] [pid=2467] check_for_external_commands()
[1342648845.458723] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648845.708868] [008.1] [pid=2467] ** Event Check Loop
[1342648845.708986] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648845.709011] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648845.709023] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648845.709036] [001.0] [pid=2467] check_for_external_commands()
[1342648845.709049] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648845.959178] [008.1] [pid=2467] ** Event Check Loop
[1342648845.959280] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648845.959303] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648845.959315] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648845.959327] [001.0] [pid=2467] check_for_external_commands()
[1342648845.959339] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648846.209484] [008.1] [pid=2467] ** Event Check Loop
[1342648846.209605] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648846.209630] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648846.209643] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648846.209655] [001.0] [pid=2467] check_for_external_commands()
[1342648846.209668] [064.1] [pid=2467] Making callbacks (type 18)...
[1342648846.209682] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648846.459823] [008.1] [pid=2467] ** Event Check Loop
[1342648846.459941] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648846.459965] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648846.459977] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648846.459990] [001.0] [pid=2467] check_for_external_commands()
[1342648846.460003] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648846.710143] [008.1] [pid=2467] ** Event Check Loop
[1342648846.710281] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648846.710306] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648846.710318] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648846.710330] [001.0] [pid=2467] check_for_external_commands()
[1342648846.710344] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648846.960488] [008.1] [pid=2467] ** Event Check Loop
[1342648846.960608] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648846.960631] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648846.960643] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648846.960656] [001.0] [pid=2467] check_for_external_commands()
[1342648846.960669] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648847.210808] [008.1] [pid=2467] ** Event Check Loop
[1342648847.210942] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648847.210982] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648847.210996] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648847.211009] [001.0] [pid=2467] check_for_external_commands()
[1342648847.211027] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648847.461167] [008.1] [pid=2467] ** Event Check Loop
[1342648847.461287] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648847.461311] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648847.461323] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648847.461336] [001.0] [pid=2467] check_for_external_commands()
[1342648847.461348] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648847.711489] [008.1] [pid=2467] ** Event Check Loop
[1342648847.711607] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648847.711631] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648847.711643] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648847.711655] [001.0] [pid=2467] check_for_external_commands()
[1342648847.711668] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648847.961843] [008.1] [pid=2467] ** Event Check Loop
[1342648847.961957] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648847.961981] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648847.961993] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648847.962005] [001.0] [pid=2467] check_for_external_commands()
[1342648847.962018] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648848.212159] [008.1] [pid=2467] ** Event Check Loop
[1342648848.212277] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648848.212301] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648848.212313] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648848.212326] [001.0] [pid=2467] check_for_external_commands()
[1342648848.212340] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648848.462482] [064.1] [pid=2467] Making callbacks (type 18)...
[1342648848.462566] [008.1] [pid=2467] ** Event Check Loop
[1342648848.462616] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648848.462638] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648848.462650] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648848.462662] [001.0] [pid=2467] check_for_external_commands()
[1342648848.462674] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648848.712820] [008.1] [pid=2467] ** Event Check Loop
[1342648848.712940] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648848.712963] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648848.712975] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648848.712987] [001.0] [pid=2467] check_for_external_commands()
[1342648848.713026] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648848.963171] [008.1] [pid=2467] ** Event Check Loop
[1342648848.963288] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648848.963311] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648848.963323] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648848.963336] [001.0] [pid=2467] check_for_external_commands()
[1342648848.963349] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648849.213496] [008.1] [pid=2467] ** Event Check Loop
[1342648849.213616] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648849.213640] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648849.213652] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648849.213664] [001.0] [pid=2467] check_for_external_commands()
[1342648849.213678] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648849.463822] [008.1] [pid=2467] ** Event Check Loop
[1342648849.463939] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648849.463963] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648849.463975] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648849.463989] [001.0] [pid=2467] check_for_external_commands()
[1342648849.464002] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648849.714150] [008.1] [pid=2467] ** Event Check Loop
[1342648849.714265] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648849.714288] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648849.714300] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648849.714313] [001.0] [pid=2467] check_for_external_commands()
[1342648849.714326] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648849.964466] [008.1] [pid=2467] ** Event Check Loop
[1342648849.964582] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648849.964606] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648849.964617] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648849.964630] [001.0] [pid=2467] check_for_external_commands()
[1342648849.964643] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648850.214781] [008.1] [pid=2467] ** Event Check Loop
[1342648850.214895] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:50 2012
[1342648850.214917] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648850.214929] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648850.214942] [001.0] [pid=2467] handle_timed_event() start
[1342648850.214954] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648850.214973] [008.0] [pid=2467] ** Timed Event ** Type: 1, Run Time: Wed Jul 18 17:00:50 2012
[1342648850.214986] [008.0] [pid=2467] ** External Command Check Event
[1342648850.214997] [001.0] [pid=2467] check_for_external_commands()
[1342648850.215009] [001.0] [pid=2467] handle_timed_event() end
[1342648850.215020] [001.0] [pid=2467] reschedule_event()
[1342648850.215031] [001.0] [pid=2467] add_event()
[1342648850.215042] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648850.215053] [008.1] [pid=2467] ** Event Check Loop
[1342648850.215071] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648850.215090] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:50 2012
[1342648850.215102] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648850.215113] [008.1] [pid=2467] Running event...
[1342648850.215124] [001.0] [pid=2467] handle_timed_event() start
[1342648850.215134] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648850.215153] [008.0] [pid=2467] ** Timed Event ** Type: 0, Run Time: Wed Jul 18 17:00:50 2012
[1342648850.215165] [008.0] [pid=2467] ** Service Check Event ==> Host: 'ENT-DC-W2K8-01', Service: 'Service Check', Options: 0, Latency: 0.215000 sec
[1342648850.215188] [001.0] [pid=2467] run_scheduled_service_check() start
[1342648850.215225] [016.0] [pid=2467] Attempting to run scheduled check of service 'Service Check' on host 'ENT-DC-W2K8-01': check options=0, latency=0.215000
[1342648850.215241] [001.0] [pid=2467] run_async_service_check()
[1342648850.215263] [001.0] [pid=2467] check_service_check_viability()
[1342648850.215275] [001.0] [pid=2467] check_time_against_period()
[1342648850.215296] [001.0] [pid=2467] check_service_dependencies()
[1342648850.215313] [064.1] [pid=2467] Making callbacks (type 13)...
[1342648850.215325] [016.0] [pid=2467] Checking service 'Service Check' on host 'ENT-DC-W2K8-01'...
[1342648850.215369] [001.0] [pid=2467] get_raw_command_line_r()
[1342648850.215390] [001.0] [pid=2467] process_macros_r()
[1342648850.215402] [2048.1] [pid=2467] **** BEGIN MACRO PROCESSING ***********
[1342648850.215413] [2048.1] [pid=2467] Processing: 'WinVNC4,TermService,SepMasterService,"Active Directory Domain Services","DNS Server"'
[1342648850.215434] [2048.1] [pid=2467]   Done.  Final output: 'WinVNC4,TermService,SepMasterService,"Active Directory Domain Services","DNS Server"'
[1342648850.215447] [2048.1] [pid=2467] **** END MACRO PROCESSING *************
[1342648850.215457] [001.0] [pid=2467] process_macros_r()
[1342648850.215468] [2048.1] [pid=2467] **** BEGIN MACRO PROCESSING ***********
[1342648850.215479] [2048.1] [pid=2467] Processing: '$USER1$/check_nt -H $HOSTADDRESS$ -p 20000 -v SERVICESTATE -l $ARG1$'
[1342648850.215517] [2048.1] [pid=2467]   Done.  Final output: '/usr/local/nagios/libexec/check_nt -H 10.1.1.1 -p 20000 -v SERVICESTATE -l WinVNC4,TermService,SepMasterService,"Active Directory Domain Services","DNS Server"'
[1342648850.215530] [2048.1] [pid=2467] **** END MACRO PROCESSING *************
[1342648850.215543] [064.1] [pid=2467] Making callbacks (type 13)...
[1342648850.215651] [016.1] [pid=2467] Check result output will be written to '/usr/local/nagios/var/spool/checkresults/checkvogvxU' (fd=7)
[1342648850.216874] [001.0] [pid=8893] process_macros_r()
[1342648850.216953] [001.0] [pid=8893] process_macros_r()
[1342648850.216968] [2048.1] [pid=8893] **** BEGIN MACRO PROCESSING ***********
[1342648850.216980] [2048.1] [pid=8893] Processing: '## http://10.1.1.39'
[1342648850.216994] [2048.1] [pid=8893]   Done.  Final output: '## http://10.1.1.39'
[1342648850.217006] [2048.1] [pid=8893] **** END MACRO PROCESSING *************
[1342648850.217025] [001.0] [pid=8893] process_macros_r()
[1342648850.217046] [001.0] [pid=8893] process_macros_r()
[1342648850.217066] [001.0] [pid=8893] process_macros_r()
[1342648850.217078] [2048.1] [pid=8893] **** BEGIN MACRO PROCESSING ***********
[1342648850.217089] [2048.1] [pid=8893] Processing: '##'
[1342648850.217102] [2048.1] [pid=8893]   Done.  Final output: '##'
[1342648850.217114] [2048.1] [pid=8893] **** END MACRO PROCESSING *************
[1342648850.217133] [001.0] [pid=8893] process_macros_r()
[1342648850.220455] [001.0] [pid=2467] handle_timed_event() end
[1342648850.220535] [008.1] [pid=2467] ** Event Check Loop
[1342648850.220587] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648850.220611] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648850.220624] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648850.220636] [001.0] [pid=2467] check_for_external_commands()
[1342648850.220649] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648850.470797] [008.1] [pid=2467] ** Event Check Loop
[1342648850.470918] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648850.470943] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648850.470955] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648850.470968] [001.0] [pid=2467] check_for_external_commands()
[1342648850.470981] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648850.721118] [008.1] [pid=2467] ** Event Check Loop
[1342648850.721235] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648850.721259] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648850.721298] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648850.721310] [001.0] [pid=2467] check_for_external_commands()
[1342648850.721323] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648850.971467] [008.1] [pid=2467] ** Event Check Loop
[1342648850.971588] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648850.971611] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648850.971623] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648850.971636] [001.0] [pid=2467] check_for_external_commands()
[1342648850.971650] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648851.221843] [008.1] [pid=2467] ** Event Check Loop
[1342648851.221963] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648851.221987] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648851.221999] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648851.222012] [001.0] [pid=2467] check_for_external_commands()
[1342648851.222025] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648851.472170] [008.1] [pid=2467] ** Event Check Loop
[1342648851.472290] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648851.472314] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648851.472325] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648851.472338] [001.0] [pid=2467] check_for_external_commands()
[1342648851.472351] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648851.722495] [008.1] [pid=2467] ** Event Check Loop
[1342648851.722613] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648851.722636] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648851.722647] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648851.722660] [001.0] [pid=2467] check_for_external_commands()
[1342648851.722673] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648851.972837] [008.1] [pid=2467] ** Event Check Loop
[1342648851.972952] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648851.972976] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648851.972988] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648851.973000] [001.0] [pid=2467] check_for_external_commands()
[1342648851.973012] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648852.223158] [008.1] [pid=2467] ** Event Check Loop
[1342648852.223286] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648852.223326] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648852.223339] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648852.223354] [001.0] [pid=2467] check_for_external_commands()
[1342648852.223367] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648852.473514] [008.1] [pid=2467] ** Event Check Loop
[1342648852.473626] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648852.473650] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648852.473661] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648852.473674] [001.0] [pid=2467] check_for_external_commands()
[1342648852.473687] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648852.723832] [008.1] [pid=2467] ** Event Check Loop
[1342648852.723944] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648852.723967] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648852.723979] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648852.723991] [001.0] [pid=2467] check_for_external_commands()
[1342648852.724004] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648852.974149] [008.1] [pid=2467] ** Event Check Loop
[1342648852.974259] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648852.974282] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648852.974320] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648852.974333] [001.0] [pid=2467] check_for_external_commands()
[1342648852.974346] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648853.224494] [008.1] [pid=2467] ** Event Check Loop
[1342648853.224609] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648853.224633] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648853.224646] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648853.224658] [001.0] [pid=2467] check_for_external_commands()
[1342648853.224671] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648853.474815] [008.1] [pid=2467] ** Event Check Loop
[1342648853.474924] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648853.474947] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648853.474959] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648853.474971] [001.0] [pid=2467] check_for_external_commands()
[1342648853.474984] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648853.725123] [008.1] [pid=2467] ** Event Check Loop
[1342648853.725234] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648853.725257] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648853.725270] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648853.725282] [001.0] [pid=2467] check_for_external_commands()
[1342648853.725295] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648853.975439] [008.1] [pid=2467] ** Event Check Loop
[1342648853.975548] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648853.975571] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648853.975583] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648853.975595] [001.0] [pid=2467] check_for_external_commands()
[1342648853.975608] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648854.225761] [008.1] [pid=2467] ** Event Check Loop
[1342648854.225924] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648854.225950] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648854.225962] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648854.225975] [001.0] [pid=2467] check_for_external_commands()
[1342648854.225989] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648854.476136] [064.1] [pid=2467] Making callbacks (type 18)...
[1342648854.476220] [008.1] [pid=2467] ** Event Check Loop
[1342648854.476272] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648854.476293] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648854.476305] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648854.476317] [001.0] [pid=2467] check_for_external_commands()
[1342648854.476330] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648854.726472] [008.1] [pid=2467] ** Event Check Loop
[1342648854.726593] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648854.726615] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648854.726627] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648854.726640] [001.0] [pid=2467] check_for_external_commands()
[1342648854.726653] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648854.976797] [008.1] [pid=2467] ** Event Check Loop
[1342648854.976914] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648854.976938] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648854.976950] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648854.976962] [001.0] [pid=2467] check_for_external_commands()
[1342648854.976975] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648855.227117] [008.1] [pid=2467] ** Event Check Loop
[1342648855.227236] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648855.227287] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648855.227299] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648855.227311] [001.0] [pid=2467] handle_timed_event() start
[1342648855.227323] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648855.227342] [008.0] [pid=2467] ** Timed Event ** Type: 5, Run Time: Wed Jul 18 17:00:55 2012
[1342648855.227355] [008.0] [pid=2467] ** Check Result Reaper
[1342648855.227366] [001.0] [pid=2467] reap_check_results() start
[1342648855.227376] [016.0] [pid=2467] Starting to reap check results.
[1342648855.227448] [016.1] [pid=2467] Starting to read check result queue '/usr/local/nagios/var/spool/checkresults'...
[1342648855.227523] [016.1] [pid=2467] Processing check result file: '/usr/local/nagios/var/spool/checkresults/c886cDN'
[1342648855.227793] [016.1] [pid=2467] Handling check result for service 'Service Check' on host 'ENT-DC-W2K8-01'...
[1342648855.227812] [001.0] [pid=2467] handle_async_service_check_result()
[1342648855.227824] [016.0] [pid=2467] ** Handling check result for service 'Service Check' on host 'ENT-DC-W2K8-01'...
[1342648855.227836] [016.1] [pid=2467] HOST: ENT-DC-W2K8-01, SERVICE: Service Check, CHECK TYPE: Active, OPTIONS: 0, SCHEDULED: Yes, RESCHEDULE: Yes, EXITED OK: Yes, RETURN CODE: 0, OUTPUT: All services are running\n
[1342648855.227867] [016.1] [pid=2467] Service is OK.
[1342648855.227879] [016.1] [pid=2467] Service did not change state.
[1342648855.227903] [016.1] [pid=2467] Rescheduling next check of service at Wed Jul 18 17:05:50 2012
[1342648855.227916] [001.0] [pid=2467] get_next_valid_time()
[1342648855.227927] [001.0] [pid=2467] check_time_against_period()
[1342648855.227951] [001.0] [pid=2467] schedule_service_check()
[1342648855.227972] [016.0] [pid=2467] Scheduling a non-forced, active check of service 'Service Check' on host 'ENT-DC-W2K8-01' @ Wed Jul 18 17:05:50 2012
[1342648855.227990] [001.0] [pid=2467] reschedule_event()
[1342648855.228001] [001.0] [pid=2467] add_event()
[1342648855.228013] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648855.228025] [064.1] [pid=2467] Making callbacks (type 20)...
[1342648855.228038] [064.1] [pid=2467] Making callbacks (type 13)...
[1342648855.228049] [064.1] [pid=2467] Making callbacks (type 20)...
[1342648855.228061] [001.0] [pid=2467] check_for_service_flapping()
[1342648855.228071] [016.1] [pid=2467] Checking service 'Service Check' on host 'ENT-DC-W2K8-01' for flapping...
[1342648855.228084] [016.1] [pid=2467] Service is not flapping (0.00% state change).
[1342648855.228104] [001.0] [pid=2467] check_for_host_flapping()
[1342648855.228115] [016.1] [pid=2467] Checking host 'ENT-DC-W2K8-01' for flapping...
[1342648855.228130] [016.1] [pid=2467] Host is not flapping (0.00% state change).
[1342648855.228145] [016.1] [pid=2467] Deleted check result file '/usr/local/nagios/var/spool/checkresults/c886cDN'
[1342648855.228158] [016.0] [pid=2467] Finished reaping 1 check results
[1342648855.228170] [001.0] [pid=2467] reap_check_results() end
[1342648855.228180] [001.0] [pid=2467] handle_timed_event() end
[1342648855.228194] [001.0] [pid=2467] reschedule_event()
[1342648855.228205] [001.0] [pid=2467] add_event()
[1342648855.228216] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648855.228228] [008.1] [pid=2467] ** Event Check Loop
[1342648855.228248] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648855.228268] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648855.228280] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648855.228291] [001.0] [pid=2467] handle_timed_event() start
[1342648855.228302] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648855.228321] [008.0] [pid=2467] ** Timed Event ** Type: 8, Run Time: Wed Jul 18 17:00:55 2012
[1342648855.228334] [008.0] [pid=2467] ** Status Data Save Event
[1342648855.228345] [064.1] [pid=2467] Making callbacks (type 25)...
[1342648855.228357] [001.0] [pid=2467] save_status_data()
[1342648855.230666] [064.1] [pid=2467] Making callbacks (type 25)...
[1342648855.230694] [001.0] [pid=2467] handle_timed_event() end
[1342648855.230707] [001.0] [pid=2467] reschedule_event()
[1342648855.230719] [001.0] [pid=2467] add_event()
[1342648855.230731] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648855.230743] [008.1] [pid=2467] ** Event Check Loop
[1342648855.230772] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:00:55 2012
[1342648855.230794] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648855.230806] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648855.230818] [001.0] [pid=2467] handle_timed_event() start
[1342648855.230829] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648855.230849] [008.0] [pid=2467] ** Timed Event ** Type: 1, Run Time: Wed Jul 18 17:00:55 2012
[1342648855.230862] [008.0] [pid=2467] ** External Command Check Event
[1342648855.230874] [001.0] [pid=2467] check_for_external_commands()
[1342648855.230885] [001.0] [pid=2467] handle_timed_event() end
[1342648855.230896] [001.0] [pid=2467] reschedule_event()
[1342648855.230907] [001.0] [pid=2467] add_event()
[1342648855.230918] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648855.230930] [008.1] [pid=2467] ** Event Check Loop
[1342648855.230949] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648855.230968] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:00:55 2012
[1342648855.230980] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648855.230992] [008.1] [pid=2467] Running event...
[1342648855.231003] [001.0] [pid=2467] handle_timed_event() start
[1342648855.231014] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648855.231033] [008.0] [pid=2467] ** Timed Event ** Type: 12, Run Time: Wed Jul 18 17:00:55 2012
[1342648855.231046] [008.0] [pid=2467] ** Host Check Event ==> Host: 'ENT-ADM-W7-00', Options: 0, Latency: 0.231000 sec
[1342648855.231066] [001.0] [pid=2467] perform_scheduled_host_check()
[1342648855.231078] [001.0] [pid=2467] run_scheduled_host_check_3x()
[1342648855.231089] [016.0] [pid=2467] Attempting to run scheduled check of host 'ENT-ADM-W7-00': check options=0, latency=0.231000
[1342648855.231104] [001.0] [pid=2467] run_async_host_check_3x()
[1342648855.231122] [016.0] [pid=2467] ** Running async check of host 'ENT-ADM-W7-00'...
[1342648855.231134] [001.0] [pid=2467] check_host_check_viability_3x()
[1342648855.231145] [001.0] [pid=2467] check_time_against_period()
[1342648855.231157] [001.0] [pid=2467] check_host_dependencies()
[1342648855.231169] [016.1] [pid=2467] A check of this host is already being executed, so we'll pass for the moment...
[1342648855.231181] [016.1] [pid=2467] Unable to run scheduled host check at this time
[1342648855.231193] [001.0] [pid=2467] get_next_valid_time()
[1342648855.231224] [016.1] [pid=2467] Rescheduled next host check for Wed Jul 18 17:05:55 2012
[1342648855.231237] [064.1] [pid=2467] Making callbacks (type 19)...
[1342648855.231249] [001.0] [pid=2467] schedule_host_check()
[1342648855.231268] [016.0] [pid=2467] Scheduling a non-forced, active check of host 'ENT-ADM-W7-00' @ Wed Jul 18 17:05:55 2012
[1342648855.231285] [001.0] [pid=2467] reschedule_event()
[1342648855.231297] [001.0] [pid=2467] add_event()
[1342648855.231308] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648855.231319] [064.1] [pid=2467] Making callbacks (type 19)...
[1342648855.231331] [001.0] [pid=2467] handle_timed_event() end
[1342648855.231342] [008.1] [pid=2467] ** Event Check Loop
[1342648855.231362] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648855.231382] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648855.231394] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648855.231407] [001.0] [pid=2467] check_for_external_commands()
[1342648855.231418] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648855.481560] [008.1] [pid=2467] ** Event Check Loop
[1342648855.481703] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648855.481728] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648855.481740] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648855.481753] [001.0] [pid=2467] check_for_external_commands()
[1342648855.481766] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648855.731909] [008.1] [pid=2467] ** Event Check Loop
[1342648855.732028] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648855.732052] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648855.732064] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648855.732076] [001.0] [pid=2467] check_for_external_commands()
[1342648855.732089] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648855.982230] [008.1] [pid=2467] ** Event Check Loop
[1342648855.982346] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648855.982369] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648855.982381] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648855.982393] [001.0] [pid=2467] check_for_external_commands()
[1342648855.982406] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648856.232569] [008.1] [pid=2467] ** Event Check Loop
[1342648856.232759] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648856.232803] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648856.232828] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648856.232851] [001.0] [pid=2467] check_for_external_commands()
[1342648856.232876] [064.1] [pid=2467] Making callbacks (type 18)...
[1342648856.232901] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648856.483059] [008.1] [pid=2467] ** Event Check Loop
[1342648856.483179] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648856.483203] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648856.483215] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648856.483227] [001.0] [pid=2467] check_for_external_commands()
[1342648856.483240] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648856.733388] [008.1] [pid=2467] ** Event Check Loop
[1342648856.733511] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648856.733534] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648856.733546] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648856.733559] [001.0] [pid=2467] check_for_external_commands()
[1342648856.733572] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648856.983718] [008.1] [pid=2467] ** Event Check Loop
[1342648856.983835] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648856.983858] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648856.983870] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648856.983883] [001.0] [pid=2467] check_for_external_commands()
[1342648856.983897] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648857.234043] [008.1] [pid=2467] ** Event Check Loop
[1342648857.234175] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648857.234217] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648857.234231] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648857.234245] [001.0] [pid=2467] check_for_external_commands()
[1342648857.234258] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648857.484404] [008.1] [pid=2467] ** Event Check Loop
[1342648857.484522] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648857.484545] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648857.484556] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648857.484569] [001.0] [pid=2467] check_for_external_commands()
[1342648857.484582] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648857.734748] [008.1] [pid=2467] ** Event Check Loop
[1342648857.734867] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648857.734891] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648857.734902] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648857.734915] [001.0] [pid=2467] check_for_external_commands()
[1342648857.734928] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648857.985073] [008.1] [pid=2467] ** Event Check Loop
[1342648857.985190] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648857.985213] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648857.985225] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648857.985236] [001.0] [pid=2467] check_for_external_commands()
[1342648857.985249] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648858.235387] [008.1] [pid=2467] ** Event Check Loop
[1342648858.235503] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648858.235527] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648858.235540] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648858.235552] [001.0] [pid=2467] check_for_external_commands()
[1342648858.235566] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648858.485707] [008.1] [pid=2467] ** Event Check Loop
[1342648858.485857] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648858.485885] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648858.485897] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648858.485910] [001.0] [pid=2467] check_for_external_commands()
[1342648858.485923] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648858.736060] [008.1] [pid=2467] ** Event Check Loop
[1342648858.736175] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648858.736198] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648858.736209] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648858.736221] [001.0] [pid=2467] check_for_external_commands()
[1342648858.736234] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648858.986373] [008.1] [pid=2467] ** Event Check Loop
[1342648858.986493] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648858.986515] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648858.986527] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648858.986539] [001.0] [pid=2467] check_for_external_commands()
[1342648858.986552] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648859.236700] [008.1] [pid=2467] ** Event Check Loop
[1342648859.236817] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648859.236839] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648859.236851] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648859.236863] [001.0] [pid=2467] check_for_external_commands()
[1342648859.236876] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648859.487015] [008.1] [pid=2467] ** Event Check Loop
[1342648859.487132] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648859.487157] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648859.487169] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648859.487182] [001.0] [pid=2467] check_for_external_commands()
[1342648859.487194] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648859.737341] [008.1] [pid=2467] ** Event Check Loop
[1342648859.737458] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648859.737481] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648859.737492] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648859.737504] [001.0] [pid=2467] check_for_external_commands()
[1342648859.737518] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648859.987692] [008.1] [pid=2467] ** Event Check Loop
[1342648859.987813] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648859.987837] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648859.987848] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648859.987860] [001.0] [pid=2467] check_for_external_commands()
[1342648859.987874] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648860.238021] [008.1] [pid=2467] ** Event Check Loop
[1342648860.238140] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:00 2012
[1342648860.238163] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648860.238175] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648860.238188] [001.0] [pid=2467] handle_timed_event() start
[1342648860.238200] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648860.238219] [008.0] [pid=2467] ** Timed Event ** Type: 1, Run Time: Wed Jul 18 17:01:00 2012
[1342648860.238232] [008.0] [pid=2467] ** External Command Check Event
[1342648860.238243] [001.0] [pid=2467] check_for_external_commands()
[1342648860.238254] [001.0] [pid=2467] handle_timed_event() end
[1342648860.238265] [001.0] [pid=2467] reschedule_event()
[1342648860.238276] [001.0] [pid=2467] add_event()
[1342648860.238287] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648860.238299] [064.1] [pid=2467] Making callbacks (type 18)...
[1342648860.238311] [008.1] [pid=2467] ** Event Check Loop

Re: Nagios Notifications not being delivered off server.

Posted: Wed Jul 18, 2012 5:26 pm
by amc_oldsarge
Part deux

Code: Select all

[1342648860.238328] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648860.238348] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648860.238359] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648860.238371] [001.0] [pid=2467] check_for_external_commands()
[1342648860.238382] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648860.488528] [008.1] [pid=2467] ** Event Check Loop
[1342648860.488648] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648860.488671] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648860.488683] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648860.488695] [001.0] [pid=2467] check_for_external_commands()
[1342648860.488708] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648860.738848] [008.1] [pid=2467] ** Event Check Loop
[1342648860.738964] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648860.738987] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648860.738999] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648860.739010] [001.0] [pid=2467] check_for_external_commands()
[1342648860.739023] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648860.989166] [008.1] [pid=2467] ** Event Check Loop
[1342648860.989285] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648860.989309] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648860.989321] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648860.989334] [001.0] [pid=2467] check_for_external_commands()
[1342648860.989346] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648861.239490] [008.1] [pid=2467] ** Event Check Loop
[1342648861.239607] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648861.239629] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648861.239641] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648861.239653] [001.0] [pid=2467] check_for_external_commands()
[1342648861.239666] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648861.489791] [008.1] [pid=2467] ** Event Check Loop
[1342648861.489888] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648861.489895] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648861.489897] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648861.489919] [001.0] [pid=2467] check_for_external_commands()
[1342648861.489923] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648861.740053] [008.1] [pid=2467] ** Event Check Loop
[1342648861.740172] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648861.740194] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648861.740206] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648861.740218] [001.0] [pid=2467] check_for_external_commands()
[1342648861.740231] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648861.990376] [008.1] [pid=2467] ** Event Check Loop
[1342648861.990494] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648861.990517] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648861.990529] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648861.990541] [001.0] [pid=2467] check_for_external_commands()
[1342648861.990554] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648862.240695] [008.1] [pid=2467] ** Event Check Loop
[1342648862.240829] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648862.240872] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648862.240885] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648862.240902] [001.0] [pid=2467] check_for_external_commands()
[1342648862.240915] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648862.491056] [008.1] [pid=2467] ** Event Check Loop
[1342648862.491174] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648862.491198] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648862.491210] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648862.491222] [001.0] [pid=2467] check_for_external_commands()
[1342648862.491235] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648862.741381] [008.1] [pid=2467] ** Event Check Loop
[1342648862.741499] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648862.741522] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648862.741534] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648862.741545] [001.0] [pid=2467] check_for_external_commands()
[1342648862.741559] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648862.991700] [008.1] [pid=2467] ** Event Check Loop
[1342648862.991818] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648862.991842] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648862.991853] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648862.991866] [001.0] [pid=2467] check_for_external_commands()
[1342648862.991881] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648863.242028] [008.1] [pid=2467] ** Event Check Loop
[1342648863.242145] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648863.242168] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648863.242179] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648863.242192] [001.0] [pid=2467] check_for_external_commands()
[1342648863.242205] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648863.492338] [008.1] [pid=2467] ** Event Check Loop
[1342648863.492453] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648863.492476] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648863.492488] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648863.492500] [001.0] [pid=2467] check_for_external_commands()
[1342648863.492513] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648863.742650] [008.1] [pid=2467] ** Event Check Loop
[1342648863.742764] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648863.742788] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648863.742799] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648863.742839] [001.0] [pid=2467] check_for_external_commands()
[1342648863.742853] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648863.992999] [008.1] [pid=2467] ** Event Check Loop
[1342648863.993117] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648863.993140] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648863.993152] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648863.993164] [001.0] [pid=2467] check_for_external_commands()
[1342648863.993178] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648864.243323] [008.1] [pid=2467] ** Event Check Loop
[1342648864.243441] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648864.243464] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648864.243475] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648864.243488] [001.0] [pid=2467] check_for_external_commands()
[1342648864.243501] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648864.493642] [008.1] [pid=2467] ** Event Check Loop
[1342648864.493760] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648864.493783] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648864.493795] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648864.493869] [001.0] [pid=2467] check_for_external_commands()
[1342648864.493882] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648864.744027] [008.1] [pid=2467] ** Event Check Loop
[1342648864.744144] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648864.744167] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648864.744179] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648864.744191] [001.0] [pid=2467] check_for_external_commands()
[1342648864.744204] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648864.994345] [008.1] [pid=2467] ** Event Check Loop
[1342648864.994462] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648864.994485] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648864.994496] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648864.994509] [001.0] [pid=2467] check_for_external_commands()
[1342648864.994522] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.244668] [008.1] [pid=2467] ** Event Check Loop
[1342648865.244788] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648865.244811] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648865.244823] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648865.244835] [001.0] [pid=2467] handle_timed_event() start
[1342648865.244847] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.244866] [008.0] [pid=2467] ** Timed Event ** Type: 6, Run Time: Wed Jul 18 17:01:05 2012
[1342648865.244878] [008.0] [pid=2467] ** Orphaned Host and Service Check Event
[1342648865.244890] [001.0] [pid=2467] check_for_orphaned_hosts()
[1342648865.244901] [001.0] [pid=2467] check_for_orphaned_services()
[1342648865.244918] [001.0] [pid=2467] handle_timed_event() end
[1342648865.244929] [001.0] [pid=2467] reschedule_event()
[1342648865.244939] [001.0] [pid=2467] add_event()
[1342648865.244950] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.244961] [008.1] [pid=2467] ** Event Check Loop
[1342648865.244980] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648865.244999] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648865.245010] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648865.245021] [001.0] [pid=2467] handle_timed_event() start
[1342648865.245032] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.245050] [008.0] [pid=2467] ** Timed Event ** Type: 10, Run Time: Wed Jul 18 17:01:05 2012
[1342648865.245061] [008.0] [pid=2467] ** Service Result Freshness Check Event
[1342648865.245099] [001.0] [pid=2467] check_service_result_freshness()
[1342648865.245111] [016.1] [pid=2467] Checking the freshness of service check results...
[1342648865.245122] [001.0] [pid=2467] handle_timed_event() end
[1342648865.245133] [001.0] [pid=2467] reschedule_event()
[1342648865.245143] [001.0] [pid=2467] add_event()
[1342648865.245154] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.245165] [008.1] [pid=2467] ** Event Check Loop
[1342648865.245184] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648865.245203] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648865.245214] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648865.245225] [001.0] [pid=2467] handle_timed_event() start
[1342648865.245236] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.245254] [008.0] [pid=2467] ** Timed Event ** Type: 5, Run Time: Wed Jul 18 17:01:05 2012
[1342648865.245266] [008.0] [pid=2467] ** Check Result Reaper
[1342648865.245277] [001.0] [pid=2467] reap_check_results() start
[1342648865.245287] [016.0] [pid=2467] Starting to reap check results.
[1342648865.245322] [016.1] [pid=2467] Starting to read check result queue '/usr/local/nagios/var/spool/checkresults'...
[1342648865.245391] [016.0] [pid=2467] Finished reaping 0 check results
[1342648865.245405] [001.0] [pid=2467] reap_check_results() end
[1342648865.245416] [001.0] [pid=2467] handle_timed_event() end
[1342648865.245426] [001.0] [pid=2467] reschedule_event()
[1342648865.245437] [001.0] [pid=2467] add_event()
[1342648865.245448] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.245459] [008.1] [pid=2467] ** Event Check Loop
[1342648865.245479] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648865.245500] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648865.245512] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648865.245523] [001.0] [pid=2467] handle_timed_event() start
[1342648865.245534] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.245552] [008.0] [pid=2467] ** Timed Event ** Type: 8, Run Time: Wed Jul 18 17:01:05 2012
[1342648865.245565] [008.0] [pid=2467] ** Status Data Save Event
[1342648865.245576] [064.1] [pid=2467] Making callbacks (type 25)...
[1342648865.245587] [001.0] [pid=2467] save_status_data()
[1342648865.247794] [064.1] [pid=2467] Making callbacks (type 25)...
[1342648865.247817] [001.0] [pid=2467] handle_timed_event() end
[1342648865.247830] [001.0] [pid=2467] reschedule_event()
[1342648865.247841] [001.0] [pid=2467] add_event()
[1342648865.247853] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.247865] [008.1] [pid=2467] ** Event Check Loop
[1342648865.247891] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:05 2012
[1342648865.247913] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648865.247925] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648865.247937] [001.0] [pid=2467] handle_timed_event() start
[1342648865.247948] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.247967] [008.0] [pid=2467] ** Timed Event ** Type: 1, Run Time: Wed Jul 18 17:01:05 2012
[1342648865.247980] [008.0] [pid=2467] ** External Command Check Event
[1342648865.247992] [001.0] [pid=2467] check_for_external_commands()
[1342648865.248003] [001.0] [pid=2467] handle_timed_event() end
[1342648865.248014] [001.0] [pid=2467] reschedule_event()
[1342648865.248025] [001.0] [pid=2467] add_event()
[1342648865.248036] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.248047] [008.1] [pid=2467] ** Event Check Loop
[1342648865.248066] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648865.248086] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:05 2012
[1342648865.248098] [008.1] [pid=2467] Current/Max Service Checks: 0/0
[1342648865.248110] [008.1] [pid=2467] Running event...
[1342648865.248121] [001.0] [pid=2467] handle_timed_event() start
[1342648865.248143] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.248164] [008.0] [pid=2467] ** Timed Event ** Type: 0, Run Time: Wed Jul 18 17:01:05 2012
[1342648865.248176] [008.0] [pid=2467] ** Service Check Event ==> Host: 'ENT-ADM-W7-00', Service: 'Memory Util.', Options: 0, Latency: 0.248000 sec
[1342648865.248193] [001.0] [pid=2467] run_scheduled_service_check() start
[1342648865.248205] [016.0] [pid=2467] Attempting to run scheduled check of service 'Memory Util.' on host 'ENT-ADM-W7-00': check options=0, latency=0.248000
[1342648865.248221] [001.0] [pid=2467] run_async_service_check()
[1342648865.248233] [001.0] [pid=2467] check_service_check_viability()
[1342648865.248245] [001.0] [pid=2467] check_time_against_period()
[1342648865.248265] [001.0] [pid=2467] check_service_dependencies()
[1342648865.248282] [064.1] [pid=2467] Making callbacks (type 13)...
[1342648865.248294] [016.0] [pid=2467] Checking service 'Memory Util.' on host 'ENT-ADM-W7-00'...
[1342648865.248327] [001.0] [pid=2467] get_raw_command_line_r()
[1342648865.248344] [001.0] [pid=2467] process_macros_r()
[1342648865.248356] [2048.1] [pid=2467] **** BEGIN MACRO PROCESSING ***********
[1342648865.248368] [2048.1] [pid=2467] Processing: '75'
[1342648865.248383] [2048.1] [pid=2467]   Done.  Final output: '75'
[1342648865.248395] [2048.1] [pid=2467] **** END MACRO PROCESSING *************
[1342648865.248406] [001.0] [pid=2467] process_macros_r()
[1342648865.248417] [2048.1] [pid=2467] **** BEGIN MACRO PROCESSING ***********
[1342648865.248428] [2048.1] [pid=2467] Processing: '90'
[1342648865.248440] [2048.1] [pid=2467]   Done.  Final output: '90'
[1342648865.248452] [2048.1] [pid=2467] **** END MACRO PROCESSING *************
[1342648865.248463] [001.0] [pid=2467] process_macros_r()
[1342648865.248474] [2048.1] [pid=2467] **** BEGIN MACRO PROCESSING ***********
[1342648865.248484] [2048.1] [pid=2467] Processing: '$USER1$/check_nt -H $HOSTADDRESS$ -p 20000 -v MEMUSE -w $ARG1$ -c $ARG2$'
[1342648865.248530] [2048.1] [pid=2467]   Done.  Final output: '/usr/local/nagios/libexec/check_nt -H 10.1.7.8 -p 20000 -v MEMUSE -w 75 -c 90'
[1342648865.248543] [2048.1] [pid=2467] **** END MACRO PROCESSING *************
[1342648865.248556] [064.1] [pid=2467] Making callbacks (type 13)...
[1342648865.248619] [016.1] [pid=2467] Check result output will be written to '/usr/local/nagios/var/spool/checkresults/checkd2kLzC' (fd=7)
[1342648865.249888] [001.0] [pid=8898] process_macros_r()
[1342648865.249974] [001.0] [pid=8898] process_macros_r()
[1342648865.249989] [2048.1] [pid=8898] **** BEGIN MACRO PROCESSING ***********
[1342648865.250001] [2048.1] [pid=8898] Processing: '## http://'
[1342648865.250016] [2048.1] [pid=8898]   Done.  Final output: '## http://'
[1342648865.250028] [2048.1] [pid=8898] **** END MACRO PROCESSING *************
[1342648865.250047] [001.0] [pid=8898] process_macros_r()
[1342648865.250069] [001.0] [pid=8898] process_macros_r()
[1342648865.250090] [001.0] [pid=8898] process_macros_r()
[1342648865.250103] [2048.1] [pid=8898] **** BEGIN MACRO PROCESSING ***********
[1342648865.250114] [2048.1] [pid=8898] Processing: '##'
[1342648865.250127] [2048.1] [pid=8898]   Done.  Final output: '##'
[1342648865.250139] [2048.1] [pid=8898] **** END MACRO PROCESSING *************
[1342648865.250158] [001.0] [pid=8898] process_macros_r()
[1342648865.253400] [001.0] [pid=2467] handle_timed_event() end
[1342648865.253475] [008.1] [pid=2467] ** Event Check Loop
[1342648865.253521] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648865.253543] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648865.253555] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648865.253567] [001.0] [pid=2467] check_for_external_commands()
[1342648865.253580] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.503708] [008.1] [pid=2467] ** Event Check Loop
[1342648865.503819] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648865.503843] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648865.503882] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648865.503895] [001.0] [pid=2467] check_for_external_commands()
[1342648865.503908] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648865.754047] [008.1] [pid=2467] ** Event Check Loop
[1342648865.754159] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648865.754183] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648865.754195] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648865.754207] [001.0] [pid=2467] check_for_external_commands()
[1342648865.754220] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648866.004369] [008.1] [pid=2467] ** Event Check Loop
[1342648866.004484] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648866.004507] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648866.004519] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648866.004532] [001.0] [pid=2467] check_for_external_commands()
[1342648866.004545] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648866.254667] [008.1] [pid=2467] ** Event Check Loop
[1342648866.254742] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648866.254749] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648866.254752] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648866.254756] [001.0] [pid=2467] check_for_external_commands()
[1342648866.254760] [064.1] [pid=2467] Making callbacks (type 18)...
[1342648866.254763] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648866.504887] [064.1] [pid=2467] Making callbacks (type 18)...
[1342648866.504974] [008.1] [pid=2467] ** Event Check Loop
[1342648866.505024] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648866.505046] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648866.505059] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648866.505071] [001.0] [pid=2467] check_for_external_commands()
[1342648866.505083] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648866.755228] [008.1] [pid=2467] ** Event Check Loop
[1342648866.755349] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648866.755373] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648866.755385] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648866.755397] [001.0] [pid=2467] check_for_external_commands()
[1342648866.755411] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648867.005559] [008.1] [pid=2467] ** Event Check Loop
[1342648867.005680] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648867.005704] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648867.005717] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648867.005729] [001.0] [pid=2467] check_for_external_commands()
[1342648867.005743] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648867.255884] [008.1] [pid=2467] ** Event Check Loop
[1342648867.256026] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648867.256065] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648867.256079] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648867.256092] [001.0] [pid=2467] check_for_external_commands()
[1342648867.256106] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648867.506237] [008.1] [pid=2467] ** Event Check Loop
[1342648867.506358] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648867.506382] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648867.506394] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648867.506407] [001.0] [pid=2467] check_for_external_commands()
[1342648867.506420] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648867.756567] [008.1] [pid=2467] ** Event Check Loop
[1342648867.756715] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648867.756739] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648867.756751] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648867.756764] [001.0] [pid=2467] check_for_external_commands()
[1342648867.756777] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648868.006922] [008.1] [pid=2467] ** Event Check Loop
[1342648868.007041] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648868.007065] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648868.007077] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648868.007090] [001.0] [pid=2467] check_for_external_commands()
[1342648868.007103] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648868.257251] [008.1] [pid=2467] ** Event Check Loop
[1342648868.257371] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648868.257395] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648868.257406] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648868.257421] [001.0] [pid=2467] check_for_external_commands()
[1342648868.257435] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648868.507564] [008.1] [pid=2467] ** Event Check Loop
[1342648868.507685] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648868.507709] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648868.507722] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648868.507734] [001.0] [pid=2467] check_for_external_commands()
[1342648868.507747] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648868.757893] [008.1] [pid=2467] ** Event Check Loop
[1342648868.758014] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648868.758037] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648868.758050] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648868.758062] [001.0] [pid=2467] check_for_external_commands()
[1342648868.758075] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648869.008212] [008.1] [pid=2467] ** Event Check Loop
[1342648869.008332] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648869.008356] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648869.008368] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648869.008381] [001.0] [pid=2467] check_for_external_commands()
[1342648869.008394] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648869.258529] [008.1] [pid=2467] ** Event Check Loop
[1342648869.258644] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648869.258668] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648869.258679] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648869.258692] [001.0] [pid=2467] check_for_external_commands()
[1342648869.258705] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648869.508837] [008.1] [pid=2467] ** Event Check Loop
[1342648869.508955] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648869.508981] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648869.508998] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648869.509012] [001.0] [pid=2467] check_for_external_commands()
[1342648869.509025] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648869.759167] [008.1] [pid=2467] ** Event Check Loop
[1342648869.759289] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648869.759313] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648869.759325] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648869.759337] [001.0] [pid=2467] check_for_external_commands()
[1342648869.759350] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648870.009492] [008.1] [pid=2467] ** Event Check Loop
[1342648870.009638] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:10 2012
[1342648870.009662] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648870.009674] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648870.009687] [001.0] [pid=2467] handle_timed_event() start
[1342648870.009699] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648870.009718] [008.0] [pid=2467] ** Timed Event ** Type: 1, Run Time: Wed Jul 18 17:01:10 2012
[1342648870.009731] [008.0] [pid=2467] ** External Command Check Event
[1342648870.009742] [001.0] [pid=2467] check_for_external_commands()
[1342648870.009754] [001.0] [pid=2467] handle_timed_event() end
[1342648870.009765] [001.0] [pid=2467] reschedule_event()
[1342648870.009785] [001.0] [pid=2467] add_event()
[1342648870.009844] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648870.009861] [008.1] [pid=2467] ** Event Check Loop
[1342648870.009882] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648870.009901] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648870.009913] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648870.009925] [001.0] [pid=2467] check_for_external_commands()
[1342648870.009936] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648870.260080] [008.1] [pid=2467] ** Event Check Loop
[1342648870.260200] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648870.260224] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648870.260236] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648870.260249] [001.0] [pid=2467] check_for_external_commands()
[1342648870.260262] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648870.510393] [008.1] [pid=2467] ** Event Check Loop
[1342648870.510514] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648870.510537] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648870.510549] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648870.510562] [001.0] [pid=2467] check_for_external_commands()
[1342648870.510575] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648870.760721] [008.1] [pid=2467] ** Event Check Loop
[1342648870.760841] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648870.760864] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648870.760876] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648870.760889] [001.0] [pid=2467] check_for_external_commands()
[1342648870.760902] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648871.011039] [008.1] [pid=2467] ** Event Check Loop
[1342648871.011158] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648871.011181] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648871.011193] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648871.011206] [001.0] [pid=2467] check_for_external_commands()
[1342648871.011219] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648871.261361] [008.1] [pid=2467] ** Event Check Loop
[1342648871.261480] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648871.261504] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648871.261516] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648871.261528] [001.0] [pid=2467] check_for_external_commands()
[1342648871.261541] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648871.511678] [008.1] [pid=2467] ** Event Check Loop
[1342648871.511799] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648871.511822] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648871.511834] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648871.511847] [001.0] [pid=2467] check_for_external_commands()
[1342648871.511859] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648871.762032] [008.1] [pid=2467] ** Event Check Loop
[1342648871.762153] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648871.762176] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648871.762188] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648871.762201] [001.0] [pid=2467] check_for_external_commands()
[1342648871.762214] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648872.012361] [008.1] [pid=2467] ** Event Check Loop
[1342648872.012479] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648872.012504] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648872.012516] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648872.012529] [001.0] [pid=2467] check_for_external_commands()
[1342648872.012542] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648872.262688] [064.1] [pid=2467] Making callbacks (type 18)...
[1342648872.262783] [008.1] [pid=2467] ** Event Check Loop
[1342648872.262865] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648872.262887] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648872.262900] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648872.262913] [001.0] [pid=2467] check_for_external_commands()
[1342648872.262925] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648872.513056] [008.1] [pid=2467] ** Event Check Loop
[1342648872.513177] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648872.513200] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648872.513212] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648872.513225] [001.0] [pid=2467] check_for_external_commands()
[1342648872.513237] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648872.763384] [008.1] [pid=2467] ** Event Check Loop
[1342648872.763507] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648872.763531] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648872.763543] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648872.763555] [001.0] [pid=2467] check_for_external_commands()
[1342648872.763569] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648873.013709] [008.1] [pid=2467] ** Event Check Loop
[1342648873.013863] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648873.013891] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:13 2012
[1342648873.013903] [008.1] [pid=2467] Current/Max Service Checks: 1/0
[1342648873.013915] [008.1] [pid=2467] Running event...
[1342648873.013926] [001.0] [pid=2467] handle_timed_event() start
[1342648873.013938] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648873.013957] [008.0] [pid=2467] ** Timed Event ** Type: 0, Run Time: Wed Jul 18 17:01:13 2012
[1342648873.013970] [008.0] [pid=2467] ** Service Check Event ==> Host: 'ENT-DC-W2K8-01', Service: 'Swap File', Options: 0, Latency: 0.013000 sec
[1342648873.013995] [001.0] [pid=2467] run_scheduled_service_check() start
[1342648873.014006] [016.0] [pid=2467] Attempting to run scheduled check of service 'Swap File' on host 'ENT-DC-W2K8-01': check options=0, latency=0.013000
[1342648873.014022] [001.0] [pid=2467] run_async_service_check()
[1342648873.014043] [001.0] [pid=2467] check_service_check_viability()
[1342648873.014055] [001.0] [pid=2467] check_time_against_period()
[1342648873.014079] [001.0] [pid=2467] check_service_dependencies()
[1342648873.014098] [064.1] [pid=2467] Making callbacks (type 13)...
[1342648873.014110] [016.0] [pid=2467] Checking service 'Swap File' on host 'ENT-DC-W2K8-01'...
[1342648873.014139] [001.0] [pid=2467] get_raw_command_line_r()
[1342648873.014159] [001.0] [pid=2467] process_macros_r()
[1342648873.014171] [2048.1] [pid=2467] **** BEGIN MACRO PROCESSING ***********
[1342648873.014182] [2048.1] [pid=2467] Processing: '75'
[1342648873.014197] [2048.1] [pid=2467]   Done.  Final output: '75'
[1342648873.014236] [2048.1] [pid=2467] **** END MACRO PROCESSING *************
[1342648873.014247] [001.0] [pid=2467] process_macros_r()
[1342648873.014258] [2048.1] [pid=2467] **** BEGIN MACRO PROCESSING ***********
[1342648873.014269] [2048.1] [pid=2467] Processing: '90'
[1342648873.014282] [2048.1] [pid=2467]   Done.  Final output: '90'
[1342648873.014293] [2048.1] [pid=2467] **** END MACRO PROCESSING *************
[1342648873.014304] [001.0] [pid=2467] process_macros_r()
[1342648873.014315] [2048.1] [pid=2467] **** BEGIN MACRO PROCESSING ***********
[1342648873.014325] [2048.1] [pid=2467] Processing: '$USER1$/check_nt -H $HOSTADDRESS$ -p 20000 -v COUNTER -l "\\Paging File(_Total)\\% Usage","Paging File usage is %.2f %%" -w $ARG1$ -c $ARG2$'
[1342648873.014376] [2048.1] [pid=2467]   Done.  Final output: '/usr/local/nagios/libexec/check_nt -H 10.1.1.1 -p 20000 -v COUNTER -l "\\Paging File(_Total)\\% Usage","Paging File usage is %.2f %%" -w 75 -c 90'
[1342648873.014389] [2048.1] [pid=2467] **** END MACRO PROCESSING *************
[1342648873.014401] [064.1] [pid=2467] Making callbacks (type 13)...
[1342648873.014501] [016.1] [pid=2467] Check result output will be written to '/usr/local/nagios/var/spool/checkresults/checkRdTH6P' (fd=7)
[1342648873.015703] [001.0] [pid=8903] process_macros_r()
[1342648873.015790] [001.0] [pid=8903] process_macros_r()
[1342648873.015804] [2048.1] [pid=8903] **** BEGIN MACRO PROCESSING ***********
[1342648873.015816] [2048.1] [pid=8903] Processing: '## http://10.1.1.39'
[1342648873.015831] [2048.1] [pid=8903]   Done.  Final output: '## http://10.1.1.39'
[1342648873.015843] [2048.1] [pid=8903] **** END MACRO PROCESSING *************
[1342648873.015862] [001.0] [pid=8903] process_macros_r()
[1342648873.015883] [001.0] [pid=8903] process_macros_r()
[1342648873.015905] [001.0] [pid=8903] process_macros_r()
[1342648873.015917] [2048.1] [pid=8903] **** BEGIN MACRO PROCESSING ***********
[1342648873.015928] [2048.1] [pid=8903] Processing: '##'
[1342648873.015940] [2048.1] [pid=8903]   Done.  Final output: '##'
[1342648873.015952] [2048.1] [pid=8903] **** END MACRO PROCESSING *************
[1342648873.015972] [001.0] [pid=8903] process_macros_r()
[1342648873.019286] [001.0] [pid=2467] handle_timed_event() end
[1342648873.019363] [008.1] [pid=2467] ** Event Check Loop
[1342648873.019415] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648873.019438] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:18 2012
[1342648873.019452] [008.1] [pid=2467] Current/Max Service Checks: 2/0
[1342648873.019465] [001.0] [pid=2467] check_for_external_commands()
[1342648873.019478] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648873.269616] [008.1] [pid=2467] ** Event Check Loop
[1342648873.269743] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648873.269768] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:18 2012
[1342648873.269781] [008.1] [pid=2467] Current/Max Service Checks: 2/0
[1342648873.269793] [001.0] [pid=2467] check_for_external_commands()
[1342648873.269853] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648873.519984] [008.1] [pid=2467] ** Event Check Loop
[1342648873.520102] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648873.520127] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:18 2012
[1342648873.520139] [008.1] [pid=2467] Current/Max Service Checks: 2/0
[1342648873.520151] [001.0] [pid=2467] check_for_external_commands()
[1342648873.520165] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648873.770296] [008.1] [pid=2467] ** Event Check Loop
[1342648873.770410] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648873.770433] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:18 2012
[1342648873.770445] [008.1] [pid=2467] Current/Max Service Checks: 2/0
[1342648873.770457] [001.0] [pid=2467] check_for_external_commands()
[1342648873.770471] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648874.020628] [008.1] [pid=2467] ** Event Check Loop
[1342648874.020743] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648874.020767] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:18 2012
[1342648874.020779] [008.1] [pid=2467] Current/Max Service Checks: 2/0
[1342648874.020791] [001.0] [pid=2467] check_for_external_commands()
[1342648874.020804] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648874.270925] [008.1] [pid=2467] ** Event Check Loop
[1342648874.271033] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648874.271056] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:18 2012
[1342648874.271068] [008.1] [pid=2467] Current/Max Service Checks: 2/0
[1342648874.271080] [001.0] [pid=2467] check_for_external_commands()
[1342648874.271093] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648874.521224] [008.1] [pid=2467] ** Event Check Loop
[1342648874.521337] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648874.521361] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:18 2012
[1342648874.521373] [008.1] [pid=2467] Current/Max Service Checks: 2/0
[1342648874.521385] [001.0] [pid=2467] check_for_external_commands()
[1342648874.521398] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648874.771529] [008.1] [pid=2467] ** Event Check Loop
[1342648874.771651] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648874.771675] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:18 2012
[1342648874.771688] [008.1] [pid=2467] Current/Max Service Checks: 2/0
[1342648874.771702] [001.0] [pid=2467] check_for_external_commands()
[1342648874.771716] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648875.021851] [008.1] [pid=2467] ** Event Check Loop
[1342648875.021967] [008.1] [pid=2467] Next High Priority Event Time: Wed Jul 18 17:01:15 2012
[1342648875.021993] [008.1] [pid=2467] Next Low Priority Event Time:  Wed Jul 18 17:01:18 2012
[1342648875.022007] [008.1] [pid=2467] Current/Max Service Checks: 2/0
[1342648875.022021] [001.0] [pid=2467] handle_timed_event() start
[1342648875.022034] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648875.022055] [008.0] [pid=2467] ** Timed Event ** Type: 5, Run Time: Wed Jul 18 17:01:15 2012
[1342648875.022069] [008.0] [pid=2467] ** Check Result Reaper
[1342648875.022081] [001.0] [pid=2467] reap_check_results() start
[1342648875.022093] [016.0] [pid=2467] Starting to reap check results.
[1342648875.022164] [016.1] [pid=2467] Starting to read check result queue '/usr/local/nagios/var/spool/checkresults'...
[1342648875.022241] [016.1] [pid=2467] Processing check result file: '/usr/local/nagios/var/spool/checkresults/cc1AcSB'
[1342648875.022506] [016.1] [pid=2467] Processing check result file: '/usr/local/nagios/var/spool/checkresults/cAOeiUy'
[1342648875.022698] [016.1] [pid=2467] Processing check result file: '/usr/local/nagios/var/spool/checkresults/ctLawJ3'
[1342648875.022879] [016.1] [pid=2467] Handling check result for service 'Memory Util.' on host 'ENT-ADM-W7-00'...
[1342648875.022898] [001.0] [pid=2467] handle_async_service_check_result()
[1342648875.022911] [016.0] [pid=2467] ** Handling check result for service 'Memory Util.' on host 'ENT-ADM-W7-00'...
[1342648875.022924] [016.1] [pid=2467] HOST: ENT-ADM-W7-00, SERVICE: Memory Util., CHECK TYPE: Active, OPTIONS: 0, SCHEDULED: Yes, RESCHEDULE: Yes, EXITED OK: Yes, RETURN CODE: 0, OUTPUT: Memory usage: total:32728.89 Mb - used: 7646.74 Mb (23%) - free: 25082.15 Mb (77%) | 'Memory usage'=7646.74Mb;24546.67;29456.00;0.00;32728.89\n
[1342648875.022962] [016.1] [pid=2467] Service is OK.
[1342648875.022976] [016.1] [pid=2467] Host is NOT UP, so we'll check it to see if it recovered...
[1342648875.022989] [001.0] [pid=2467] run_async_host_check_3x()
[1342648875.023005] [016.0] [pid=2467] ** Running async check of host 'ENT-ADM-W7-00'...
[1342648875.023018] [001.0] [pid=2467] check_host_check_viability_3x()
[1342648875.023055] [001.0] [pid=2467] check_time_against_period()
[1342648875.023068] [001.0] [pid=2467] check_host_dependencies()
[1342648875.023080] [016.1] [pid=2467] A check of this host is already being executed, so we'll pass for the moment...
[1342648875.023092] [016.1] [pid=2467] Service experienced a HARD RECOVERY.
[1342648875.023120] [001.0] [pid=2467] process_macros_r()
[1342648875.023133] [2048.1] [pid=2467] **** BEGIN MACRO PROCESSING ***********
[1342648875.023144] [2048.1] [pid=2467] Processing: 'SERVICE ALERT: ENT-ADM-W7-00;Memory Util.;$SERVICESTATE$;$SERVICESTATETYPE$;$SERVICEATTEMPT$;Memory usage: total:32728.89 Mb - used: 7646.74 Mb (23%) - free: 25082.15 Mb (77%)
'
[1342648875.023175] [2048.1] [pid=2467]   Done.  Final output: 'SERVICE ALERT: ENT-ADM-W7-00;Memory Util.;OK;HARD;1;Memory usage: total:32728.89 Mb - used: 7646.74 Mb (23%) - free: 25082.15 Mb (77%)
'
[1342648875.023188] [2048.1] [pid=2467] **** END MACRO PROCESSING *************
[1342648875.023341] [064.1] [pid=2467] Making callbacks (type 9)...
[1342648875.023360] [001.0] [pid=2467] check_for_service_flapping()
[1342648875.023380] [016.1] [pid=2467] Checking service 'Memory Util.' on host 'ENT-ADM-W7-00' for flapping...
[1342648875.023394] [001.0] [pid=2467] check_for_host_flapping()
[1342648875.023406] [016.1] [pid=2467] Checking host 'ENT-ADM-W7-00' for flapping...
[1342648875.023423] [016.1] [pid=2467] Host is not flapping (0.00% state change).
[1342648875.023443] [001.0] [pid=2467] service_notification()
[1342648875.023469] [032.0] [pid=2467] ** Service Notification Attempt ** Host: 'ENT-ADM-W7-00', Service: 'Memory Util.', Type: 0, Options: 0, Current State: 0, Last Notification: Wed Dec 31 18:00:00 1969
[1342648875.023485] [001.0] [pid=2467] check_service_notification_viability()
[1342648875.023500] [001.0] [pid=2467] check_time_against_period()
[1342648875.023526] [001.0] [pid=2467] check_service_dependencies()
[1342648875.023541] [001.0] [pid=2467] check_host_dependencies()
[1342648875.023553] [032.1] [pid=2467] We shouldn't notify about this recovery.
[1342648875.023565] [032.0] [pid=2467] Notification viability test failed.  No notification will be sent out.
[1342648875.023577] [001.0] [pid=2467] handle_service_event()
[1342648875.023589] [064.1] [pid=2467] Making callbacks (type 30)...
[1342648875.023610] [001.0] [pid=2467] run_global_service_event_handler()
[1342648875.023625] [001.0] [pid=2467] check_for_external_commands()
[1342648875.023647] [016.1] [pid=2467] Rescheduling next check of service at Wed Jul 18 17:06:05 2012
[1342648875.023661] [001.0] [pid=2467] get_next_valid_time()
[1342648875.023673] [001.0] [pid=2467] check_time_against_period()
[1342648875.023691] [001.0] [pid=2467] schedule_service_check()
[1342648875.023711] [016.0] [pid=2467] Scheduling a non-forced, active check of service 'Memory Util.' on host 'ENT-ADM-W7-00' @ Wed Jul 18 17:06:05 2012
[1342648875.023728] [001.0] [pid=2467] reschedule_event()
[1342648875.023740] [001.0] [pid=2467] add_event()
[1342648875.023752] [064.1] [pid=2467] Making callbacks (type 8)...
[1342648875.023764] [064.1] [pid=2467] Making callbacks (type 20)...
[1342648875.023779] [064.1] [pid=2467] Making callbacks (type 13)...
[1342648875.023790] [064.1] [pid=2467] Making callbacks (type 20)...
[1342648875.023803] [016.1] [pid=2467] Deleted check result file '/usr/local/nagios/var/spool/checkresults/cAOeiUy'
[1342648875.023820] [016.1] [pid=2467] Handling check result for host 'ENT-ADM-W7-00'...
[1342648875.023832] [001.0] [pid=2467] handle_async_host_check_result_3x()
[1342648875.023844] [016.1] [pid=2467] ** Handling async check result for host 'ENT-ADM-W7-00'...
[1342648875.023871] [001.0] [pid=2467] process_host_check_result_3x()
[1342648875.023884] [016.1] [pid=2467] HOST: ENT-ADM-W7-00, ATTEMPT=1/10, CHECK TYPE=ACTIVE, STATE TYPE=HARD, OLD STATE=1, NEW STATE=0
[1342648875.023898] [016.1] [pid=2467] Host was DOWN/UNREACHABLE.
[1342648875.023909] [016.1] [pid=2467] Host experienced a HARD recovery (it's now UP).
[1342648875.023921] [016.1] [pid=2467] Propagating checks to parent host(s)...
[1342648875.023943] [016.1] [pid=2467] Propagating checks to child host(s)...
[1342648875.023955] [016.1] [pid=2467] Pre-handle_host_state() Host: ENT-ADM-W7-00, Attempt=1/10, Type=HARD, Final State=0
[1342648875.023968] [001.0] [pid=2467] handle_host_state()
[1342648875.023979] [001.0] [pid=2467] obsessive_compulsive_host_check_processor()
[1342648875.024001] [001.0] [pid=2467] process_macros_r()
[1342648875.024013] [2048.1] [pid=2467] **** BEGIN MACRO PROCESSING ***********
[1342648875.024024] [2048.1] [pid=2467] Processing: 'HOST ALERT: ENT-ADM-W7-00;$HOSTSTATE$;$HOSTSTATETYPE$;$HOSTATTEMPT$;PING WARNING - Packet loss = 82%, RTA = 0.59 ms
'
[1342648875.024054] [2048.1] [pid=2467]   Done.  Final output: 'HOST ALERT: ENT-ADM-W7-00;UP;HARD;1;PING WARNING - Packet loss = 82%, RTA = 0.59 ms
'
[1342648875.024067] [2048.1] [pid=2467] **** END MACRO PROCESSING *************
[1342648875.024172] [064.1] [pid=2467] Making callbacks (type 9)...
[1342648875.024193] [001.0] [pid=2467] check_pending_flex_host_downtime()
[1342648875.024218] [032.0] [pid=2467] ** Host Notification Attempt ** Host: 'ENT-ADM-W7-00', Type: 0, Options: 0, Current State: 0, Last Notification: Wed Dec 31 18:00:00 1969
[1342648875.024233] [001.0] [pid=2467] check_host_notification_viability()
[1342648875.024245] [001.0] [pid=2467] check_time_against_period()
[1342648875.024263] [001.0] [pid=2467] check_host_dependencies()
[1342648875.024279] [032.0] [pid=2467] Notification viability test passed.
[1342648875.024290] [032.1] [pid=2467] Current notification number: 2 (incremented)
[1342648875.024303] [001.0] [pid=2467] create_notification_list_from_host()
[1342648875.024314] [001.0] [pid=2467] should_host_notification_be_escalated()
[1342648875.024325] [032.1] [pid=2467] Host notification will NOT be escalated.
[1342648875.024340] [032.1] [pid=2467] Adding normal contacts for host to notification list.
[1342648875.024351] [001.0] [pid=2467] add_notification() start
[1342648875.024362] [001.0] [pid=2467] find_notification() start
[1342648875.024375] [064.1] [pid=2467] Making callbacks (type 12)...
[1342648875.024399] [001.0] [pid=2467] notify_contact_of_host()
[1342648875.024410] [001.0] [pid=2467] check_contact_host_notification_viability()
[1342648875.024421] [001.0] [pid=2467] check_time_against_period()
[1342648875.024440] [064.1] [pid=2467] Making callbacks (type 27)...
[1342648875.024456] [064.1] [pid=2467] Making callbacks (type 28)...
[1342648875.024474] [001.0] [pid=2467] get_raw_command_line_r()
[1342648875.024490] [001.0] [pid=2467] process_macros_r()
[1342648875.024502] [2048.1] [pid=2467] **** BEGIN MACRO PROCESSING ***********
[1342648875.024513] [2048.1] [pid=2467] Processing: '/usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" $CONTACTEMAIL$'
[1342648875.024587] [2048.1] [pid=2467]   Done.  Final output: '/usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: RECOVERY\nHost: ENT-ADM-W7-00\nState: UP\nAddress: 10.1.7.8\nInfo: PING WARNING - Packet loss = 82%, RTA = 0.59 ms\n\nDate/Time: Wed Jul 18 17:01:15 CDT 2012\n" | /bin/mail -s "** RECOVERY Host Alert: ENT-ADM-W7-00 is UP **" [email protected]'
[1342648875.024601] [2048.1] [pid=2467] **** END MACRO PROCESSING *************
[1342648875.024617] [001.0] [pid=2467] process_macros_r()
[1342648875.024631] [2048.1] [pid=2467] **** BEGIN MACRO PROCESSING ***********
[1342648875.024642] [2048.1] [pid=2467] Processing: 'HOST NOTIFICATION: joe;ENT-ADM-W7-00;$HOSTSTATE$;notify-host-by-email;$HOSTOUTPUT$
'
[1342648875.024666] [2048.1] [pid=2467]   Done.  Final output: 'HOST NOTIFICATION: joe;ENT-ADM-W7-00;UP;notify-host-by-email;PING WARNING - Packet loss = 82%, RTA = 0.59 ms
'
[1342648875.024679] [2048.1] [pid=2467] **** END MACRO PROCESSING *************
[1342648875.024769] [064.1] [pid=2467] Making callbacks (type 9)...
[1342648875.024786] [001.0] [pid=2467] my_system_r()
[1342648875.024808] [256.1] [pid=2467] Running command '/usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: RECOVERY\nHost: ENT-ADM-W7-00\nState: UP\nAddress: 10.1.7.8\nInfo: PING WARNING - Packet loss = 82%, RTA = 0.59 ms\n\nDate/Time: Wed Jul 18 17:01:15 CDT 2012\n" | /bin/mail -s "** RECOVERY Host Alert: ENT-ADM-W7-00 is UP **" [email protected]'...
[1342648875.024836] [064.1] [pid=2467] Making callbacks (type 10)...
[1342648875.026101] [001.0] [pid=8907] process_macros_r()
[1342648875.026187] [001.0] [pid=8907] process_macros_r()
[1342648875.026203] [2048.1] [pid=8907] **** BEGIN MACRO PROCESSING ***********
[1342648875.026215] [2048.1] [pid=8907] Processing: '## http://'
[1342648875.026229] [2048.1] [pid=8907]   Done.  Final output: '## http://'
[1342648875.026242] [2048.1] [pid=8907] **** END MACRO PROCESSING *************
[1342648875.026261] [001.0] [pid=8907] process_macros_r()
[1342648875.047947] [256.1] [pid=2467] Execution time=0.023 sec, early timeout=0, result=1, output=(null)
[1342648875.048035] [064.1] [pid=2467] Making callbacks (type 10)...
[1342648875.048080] [064.1] [pid=2467] Making callbacks (type 28)...
[1342648875.048097] [064.1] [pid=2467] Making callbacks (type 27)...
[1342648875.048116] [001.0] [pid=2467] get_next_host_notification_time()
[1342648875.048166] [032.0] [pid=2467] 1 contacts were notified.  Next possible notification time: Wed Jul 18 19:01:15 2012
[1342648875.048180] [032.0] [pid=2467] 1 contacts were notified.
[1342648875.048194] [064.1] [pid=2467] Making callbacks (type 12)...
[1342648875.048206] [064.1] [pid=2467] Making callbacks (type 19)...
[1342648875.048218] [001.0] [pid=2467] handle_host_event()
[1342648875.048229] [064.1] [pid=2467] Making callbacks (type 30)...
[1342648875.048246] [001.0] [pid=2467] run_global_host_event_handler()
[1342648875.048258] [001.0] [pid=2467] check_for_external_commands()
[1342648875.048270] [016.1] [pid=2467] Post-handle_host_state() Host: ENT-ADM-W7-00, Attempt=1/10, Type=HARD, Final State=0
[1342648875.048284] [001.0] [pid=2467] check_for_host_flapping()
[1342648875.048294] [016.1] [pid=2467] Checking host 'ENT-ADM-W7-00' for flapping...
[1342648875.048317] [016.1] [pid=2467] Rescheduling next check of host at Wed Jul 18 17:06:15 2012
[1342648875.048330] [001.0] [pid=2467] get_next_valid_time()
[1342648875.048341] [001.0] [pid=2467] schedule_host_check()
[1342648875.048361] [016.0] [pid=2467] Scheduling a non-forced, active check of host 'ENT-ADM-W7-00' @ Wed Jul 18 17:06:15 2012
[1342648875.048390] [064.1] [pid=2467] Making callbacks (type 19)...
[1342648875.048403] [064.1] [pid=2467] Making callbacks (type 19)...
[1342648875.048417] [016.1] [pid=2467] ** Async check result for host 'ENT-ADM-W7-00' handled: new state=0
[1342648875.048431] [064.1] [pid=2467] Making callbacks (type 14)...
[1342648875.048443] [016.1] [pid=2467] Deleted check result file '/usr/local/nagios/var/spool/checkresults/cc1AcSB'

Re: Nagios Notifications not being delivered off server.

Posted: Thu Jul 19, 2012 10:35 am
by agriffin
Okay, so macro expansion seems to be working fine. This is the exact command it expanded to and then executed without any errors:

Code: Select all

/usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: RECOVERY\nHost: ENT-ADM-W7-00\nState: UP\nAddress: 10.1.7.8\nInfo: PING WARNING - Packet loss = 82%, RTA = 0.59 ms\n\nDate/Time: Wed Jul 18 17:01:15 CDT 2012\n" | /bin/mail -s "** RECOVERY Host Alert: ENT-ADM-W7-00 is UP **" [email protected]
Can you try executing that from the command line to see whether it works?