NSCA: big delay delay processing servicechecks

[sebastiaopburnay]

Hmm, okay so this is happening 24/7 correct? This error with processing the check results, and also the large latency between the received result, and it actually being displayed in the cgi's.

Are you noticing a large influx of files in:

Code: Select all

/var/spool/checkresults

Yes, it happens all the time
That folder (in my case /usr/local/nagios/var/spool/checkresults has 35 files with strange names, but all (except two) have a size of zero.

Or piling up in:

Code: Select all

/tmp

The only two folders on my filesystem named tmp are /tmp and /var/tmp and they are both empty...

Anyway, I've been doing some further analysis to log files and I've found out entries on syslog like:

Code: Select all

Feb 20 20:39:23 SRVCARTMSCORE01 ndo2db: Message sent to queue.
Feb 20 20:39:23 SRVCARTMSCORE01 ndo2db: Warning: queue send error, retrying...

That lead me to posts in other forums such as:
1 - http://www.monitoring-portal.org/wbb/in ... adID=25673,
2 - http://support.nagios.com/forum/viewtop ... 43&p=84026 and
3 - http://support.nagios.com/wiki/index.ph ... 3.x_Issues

Rhey suggest that ndomod might unable Nagios' to do its work.

while 1st post just suggests downgrading NDOUtils,
Posts 1 and 2 suggest it has something to do with settings on /etc/sysctl.conf, but I'm not comfortable enough to do those changes without understanding them

[lmiltchev]

Run the following commands and show us the output:

Code: Select all

grep "for i in" /etc/init.d/nagios
cat /etc/sysctl.conf

[sebastiaopburnay]

Run the following commands and show us the output:

Code: Select all

grep "for i in" /etc/init.d/nagios

Code: Select all

 for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30; do

cat /etc/sysctl.conf[/code]

/etc/sysctl.conf is all commented entries

Code: Select all

root@MyServer:~#  grep ^[^#] /etc/sysctl.conf 
 
 
 
root@MyServer:~# 

With further investigation of syslog and consultation of NDOUtils README file, I've defined the following values:

Code: Select all

root@myServer:~# sysctl -p
# msgmnb - product of (msgmax  * msgmni )
kernel.msgmnb = 393740288
# msgmax - suggested by NDOUtils README file
kernel.msgmax = 65536
# msgmni - left as it was
kernel.msgmni = 6008

So far I've detected no ndomod error/warning messages on syslog. And the servicechecks seem to be synched .
Yet, I want to observe teh server's behaviour for a few days prior to closing this post.

[sreinhardt]

Cool, let us know if a few days how it continues to go!

[sebastiaopburnay]

The kernel queue values have definitely solved those errors I was facing.

I do not recall how to set as solved, but I've marked it with a [v] for success

Thank you all for the time and knowledge shared.

This post will serve as knowlege article for my future implmenmtations