Nagios Average Services Check Latency is very high

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
Locked
satheesh19870819
Posts: 43
Joined: Wed Mar 21, 2018 1:11 pm

Nagios Average Services Check Latency is very high

Post by satheesh19870819 »

Hello All,

I am facing heavy Latency for host/services in Nagios . It is around 400 s all the time . Max concurrent check is already set as 80000

max_concurrent_checks=80000

Also i have 16 core and 32 GB server config and the CPU load is always around 10 to 15 not more than 15 . So resource should not be an issue .

Kindly help me to fine tune the performance where 450 s of latency is very high .

Please find below details

Code: Select all

 
Nagios Core 3.5.1
Copyright (c) 2009-2011 Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 08-30-2013
License: GPL

Website: http://www.nagios.org
Timing information on object configuration processing is listed
below.  You can use this information to see if precaching your
object configuration would be useful.

Object Config Source: Config files (uncached)

OBJECT CONFIG PROCESSING TIMES      (* = Potential for precache savings with -u option)
----------------------------------
Read:                 0.892682 sec
Resolve:              0.264231 sec  *
Recomb Contactgroups: 0.012484 sec  *
Recomb Hostgroups:    0.010538 sec  *
Dup Services:         0.237377 sec  *
Recomb Servicegroups: 0.009139 sec  *
Duplicate:            0.128406 sec  *
Inherit:              0.047183 sec  *
Recomb Contacts:      0.000000 sec  *
Sort:                 0.000000 sec  *
Register:             0.497005 sec
Free:                 0.111037 sec
                      ============
TOTAL:                2.210083 sec  * = 0.709359 sec (32.10%) estimated savings


RETENTION DATA TIMES
----------------------------------
Read and Process:     4.237136 sec
                      ============
TOTAL:                4.237136 sec


Timing information on configuration verification is listed below.

CONFIG VERIFICATION TIMES          (* = Potential for speedup with -x option)
----------------------------------
Object Relationships: 0.227714 sec
Circular Paths:       0.003508 sec  *
Misc:                 0.037706 sec
                      ============
TOTAL:                0.268928 sec  * = 0.003508 sec (1.3%) estimated savings


EVENT SCHEDULING TIMES
-------------------------------------
Get service info:        0.141716 sec
Get host info info:      0.001811 sec
Get service params:      0.000013 sec
Schedule service times:  0.425668 sec
Schedule service events: 26.723265 sec
Get host params:         0.000001 sec
Schedule host times:     0.019399 sec
Schedule host events:    9.577614 sec
                         ============
TOTAL:                   36.889487 sec


Projected scheduling information for host and service checks
is listed below.  This information assumes that you are going
to start running Nagios with your current config files.

HOST SCHEDULING INFORMATION
---------------------------
Total hosts:                     4044
Total scheduled hosts:           4044
Host inter-check delay method:   SMART
Average host check interval:     386.53 sec
Host inter-check delay:          0.10 sec
Max host check spread:           30 min
First scheduled check:           Sun May 20 23:49:55 2018
Last scheduled check:            Mon May 21 23:00:00 2018


SERVICE SCHEDULING INFORMATION
-------------------------------
Total services:                     58601
Total scheduled services:           58165
Service inter-check delay method:   SMART
Average service check interval:     17560.65 sec
Inter-check delay:                  0.03 sec
Interleave factor method:           SMART
Average services per host:          14.49
Service interleave factor:          15
Max service check spread:           30 min
First scheduled check:              Sun May 20 23:51:55 2018
Last scheduled check:               Sun May 27 16:30:00 2018


CHECK PROCESSING INFORMATION
----------------------------
Check result reaper interval:       3 sec
Max concurrent service checks:      80000


PERFORMANCE SUGGESTIONS
-----------------------
I have no suggestions - things look okay.
Last edited by tmcdonald on Mon May 21, 2018 10:07 am, edited 1 time in total.
Reason: Please use [code][/code] tags around long output
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: Nagios Average Services Check Latency is very high

Post by mcapra »

If you're looking for cheap performance gains, I would strongly suggest upgrading to Nagios Core 4+. Almost any tuning done on this system would likely be outvalued by simply upgrading to a more modern version of Nagios Core.

Here's a similar thread:
https://support.nagios.com/forum/viewto ... =7&t=48454
Former Nagios employee
https://www.mcapra.com/
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: Nagios Average Services Check Latency is very high

Post by tmcdonald »

Thanks for the assist, @mcapra! OP, please let us know if you need further assistance.
Former Nagios employee
Locked