Active Service Execution Time Doubled

[brdr]

Hi,

We are using Nagios XI 2014R2.3.

Earlier this month we had seen our 'Active Service Execution Time' almost doubled one morning. Although avg execution time remained about the same. Is there perhaps a database query that exist to capture the services execution from slowest to faster, or even top 10 slowest services? Thanks.

Hourly snapshots of nagiostats below.....

Total Services: 2159
Active Service Latency: 0.000 / 1.049 / 0.486 sec
Active Service Execution Time: 0.001 / 15.276 / 0.281 sec
Mon Feb 2 06:00:01 EST 2015
Total Services: 2159
Active Service Latency: 0.000 / 1.062 / 0.486 sec
Active Service Execution Time: 0.001 / 15.025 / 0.292 sec
Mon Feb 2 07:00:01 EST 2015
Total Services: 2159
Active Service Latency: 0.000 / 1.043 / 0.475 sec
Active Service Execution Time: 0.001 / 15.023 / 0.283 sec
Mon Feb 2 08:00:01 EST 2015
Total Services: 2159
Active Service Latency: 0.000 / 1.017 / 0.491 sec
Active Service Execution Time: 0.001 / 15.023 / 0.297 sec
Mon Feb 2 09:00:01 EST 2015
Total Services: 2159
Active Service Latency: 0.000 / 1.050 / 0.489 sec
Active Service Execution Time: 0.001 / 15.280 / 0.277 sec
Total Services: 2159
Active Service Latency: 0.000 / 7.436 / 0.494 sec
Active Service Execution Time: 0.001 / 15.026 / 0.269 sec
Mon Feb 2 10:00:01 EST 2015
Mon Feb 2 11:00:01 EST 2015
Total Services: 2159
Active Service Latency: 0.000 / 8.354 / 0.490 sec
Active Service Execution Time: 0.001 / 30.123 / 0.309 sec
Total Services: 2155
Active Service Latency: 0.000 / 7.436 / 0.492 sec
Active Service Execution Time: 0.001 / 28.889 / 0.202 sec
Mon Feb 2 12:00:01 EST 2015
Mon Feb 2 13:00:01 EST 2015
Total Services: 2155
Active Service Latency: 0.000 / 7.436 / 0.492 sec
Active Service Execution Time: 0.001 / 29.003 / 0.235 sec

[cmerchant]

I do not know of individual statistics for the service checks. Was there a change to the service check interval, or the retry check interval? Was there any outage that increased the rate of your service checks.

Could you share with us a screen shot of your Admin --> System Information --> Monitoring Engine Status.

[brdr]

screen shot attached.

I don't recall any service check interval / retry interval change, or any outage change. Another user may have added a new service check that points to increase in the service execution time.

[cmerchant]

Looking at the sceduled events over time graph, shows an uneven number of active service checks in the first part of the 15 min. interval.

The active service time average is good. The latency is an issue because of the unevenness of scheduled checks. I would have you look at

http://assets.nagios.com/downloads/nagi ... ios-XI.pdf

If you would like to PM your profile to me, we can take a look at what might of changed.