Page 1 of 1
Freshness threshold not working during apply configuration
Posted: Thu Jan 12, 2017 3:54 am
by WillemDH
Hello,
I've seen it multiple times on different passive services. When a passive check is received with a configured freshness during an apply configuration, the freshness is not working. This seems like a bug to me and makes the freshness directive unreliable.
Grtz
Willem
Re: Freshness threshold not working during apply configurati
Posted: Thu Jan 12, 2017 10:33 am
by rkennedy
Hey Willem - When you mention that the freshness is not working, are you referring to it skipping the active check that should be executing? We'll have to do some testing on our end, but just trying to understand what you're experiencing exactly.
Re: Freshness threshold not working during apply configurati
Posted: Mon Jan 16, 2017 3:41 am
by WillemDH
Clint,
It is indeed like you say, the active check (check_dummy) is not executing, resulting in the critical state not resetting. We have two apply configurations configured with Rundeck, one at 07:00 and one at 19:00. I've got some passive services set up with a freshness threshold of 60 seconds which multiple passive checks a day. When I notice these services popup in the open service problem for longer then a minute it's always at 07:00 and 19:00 or duirng a manual apply.
Grtz
Willem
Re: Freshness threshold not working during apply configurati
Posted: Mon Jan 16, 2017 12:22 pm
by ssax
What do you have set for the services in your /usr/local/nagios/var/objects.cache OR /var/nagiosramdisk/objects.cache? I'm wondering about the Retain status information (retain_status_information) and Retain non-status information (retain_nonstatus_information) settings.
Thank you
Re: Freshness threshold not working during apply configurati
Posted: Wed Jan 18, 2017 5:48 pm
by WillemDH
Sean,
Here you go:
cat /var/nagiosramdisk/objects.cache | sed -rn "/define service \{/{:a;N;/}/{/.host_name.cashpc0003/p;d};ba}"
This is one of the passive services giving me issues:
Code: Select all
define service {
host_name cashpc0003
service_description EVT_Cash_Quota
display_name EVT_System
check_period xi_timeperiod_24x7
check_command check_dummy!0!"Dummy check passed"!!!!!!
contacts nagiosadmin,nagiosadmin
contact_groups cg_dummy
notification_period xi_timeperiod_24x7
initial_state o
importance 0
check_interval 1440.000000
retry_interval 1.000000
max_check_attempts 1
is_volatile 0
parallelize_check 1
active_checks_enabled 1
passive_checks_enabled 1
obsess 1
event_handler_enabled 1
low_flap_threshold 0.000000
high_flap_threshold 0.000000
flap_detection_enabled 0
flap_detection_options a
freshness_threshold 60
check_freshness 1
notification_options a
notifications_enabled 1
notification_interval 1440.000000
first_notification_delay 0.000000
stalking_options o,w,u,c
process_perf_data 0
icon_image windowseventlog.png
retain_status_information 1
retain_nonstatus_information 1
_XIWIZARD windowseventlog
}
Grtz
Willem
Re: Freshness threshold not working during apply configurati
Posted: Thu Jan 19, 2017 4:34 pm
by tgriep
How many host and service checks are setup on this server and what version of XI is it running.
For a test, can you increase the freshness threshold to 120 seconds and see if the issue goes away or is reduced?
I am thinking that the checks that are causing the issue may be close to the 60 seconds threshold and then the Apply Config is run and during the time it takes to run that, the 60 seconds are up and the active check doesn't happen.
Also, do you see any messages in the /usr/local/nagios/var/nagios.log file at those times?
Re: Freshness threshold not working during apply configurati
Posted: Thu Jan 19, 2017 5:17 pm
by WillemDH
+- 21k hosts and svces on 5.4. I also think exactly what you are saying is happening. But I still consider this a bug... We expect the freshness to just do the work its configured to do... Enfin I'll set it higher, but that is not exactly what we need. These checks aren't supposed to show up on our dashboards. They just need to send an email alert when coming in, and then reset immediately.
Re: Freshness threshold not working during apply configurati
Posted: Fri Jan 20, 2017 10:44 am
by tgriep
Thanks for trying this out for us. Changing those settings and the issue is resolves or lessens, would give us a hint on where to start to troubleshoot this.