Page 3 of 3

Re: gearman - a lot JOBs waiting

Posted: Wed Apr 06, 2016 5:24 pm
by tmcdonald
The queue may very well build up if a lot of checks are being run, but it should go back down after a bit. Is the problem that the queue never clears, or that it sometimes isn't at 0?

Can you also address the other questions from @bheden's last post? In particular if you increased the results worker in the NEB config.

Re: gearman - a lot JOBs waiting

Posted: Thu Apr 07, 2016 5:17 pm
by bosecorp
I did it in the worker config

the problem is that sometimes the queue builds up and stays high and doesn't go down for a considerable amount of time. it doesn't seem to be able to catch up

Re: gearman - a lot JOBs waiting

Posted: Fri Apr 08, 2016 9:20 am
by bheden
Open up your NEB config, (/etc/mod_gearman/mod_gearman_neb.conf for ModGearman v1 or /etc/mod_gearman2/module.conf for ModGearman v2) and find the line that says

Code: Select all

result_workers=1
Change this line to

Code: Select all

result_workers=2
Let us know if this stops the hang ups. Thanks.

Re: gearman - a lot JOBs waiting

Posted: Fri Apr 08, 2016 6:12 pm
by bosecorp
I already had that setting. I am having at some point in the past I made that change.


for the last two days the queues have been stable

Re: gearman - a lot JOBs waiting

Posted: Mon Apr 11, 2016 1:16 pm
by tmcdonald
It's hard to say what's causing the queue to go up, but generally this will happen when there is a network outage and many hosts/services go down at once. This causes checks to be run more frequently, usually from every 5 minutes down to every minute. Do the jumps in the queue coincide with any network events you can identify?

Re: gearman - a lot JOBs waiting

Posted: Tue Apr 12, 2016 8:01 pm
by bosecorp
I haven't been good at capturing the different events. i will start..

queue has been good for the last few days.

this is so weird

Re: gearman - a lot JOBs waiting

Posted: Wed Apr 13, 2016 9:26 am
by rkennedy
Sounds good - let us know if there is any further assistance you need from us.