Gearman Load Balancer Configuration

[rajasegar]

Anytime

Sometimes I need to go and play with things to truly learn how they work. Sometimes the official documentation doesn't make a lot of sense lol.

One more question, is there a reason why the nagios supplied installation scripts is using the latest version for server install and old version for the worker?

Master server

Code: Select all

[nagios@nagiosprodxi1 perfdata]$ check_gearman -V
check_gearman: version 1.5.0b1 running on libgearman 1.1.8

This is the worker

Code: Select all

[nagios@nagiosprodcol1 ~]$ check_gearman -V
check_gearman: version 1.4_nagios4 running on libgearman 0.25

Instruction used were from the Nagios PDF file, Integrating_Mod_Gearman_with_Nagios_XI.pdf

Code: Select all

Server Installation (Nagios XI 2014 / Core 4.x)
Note: This step is completed on the local Nagios system(s), for a new installation:
cd /tmp
For 64-bit CentOS/RHEL 6 Nagios servers:
wget http://assets.nagios.com/downloads/mod_gearman/rpms/mod_gearman-1.5.0b1-1.el6.x86_64.rpm


Worker Installation (For use with a Master Server Running XI 2014 / Core 4.x)
Note: This step is completed on the remote worker system(s):
cd /tmp
wget http://assets.nagios.com/downloads/nagiosxi/scripts/ModGearmanFullinstallVersionCore4.sh
chmod +x ModGearmanFullinstallVersionCore4.sh
./ModGearmanFullinstallVersionCore4.sh

[lmiltchev]

One more question, is there a reason why the nagios supplied installation scripts is using the latest version for server install and old version for the worker?

Short answer - while upgrading the version for the workers is possible, the steps (procedures) for doing this haven't been tested/documented, yet. I assume that it will take some time before we do this, because this is not a priority (Mod Gearman used to be a lot more popular with Nagios Core 3.x, not so much with Core 4.x.)

[rajasegar]

One more question, is there a reason why the nagios supplied installation scripts is using the latest version for server install and old version for the worker?

Short answer - while upgrading the version for the workers is possible, the steps (procedures) for doing this haven't been tested/documented, yet. I assume that it will take some time before we do this, because this is not a priority (Mod Gearman used to be a lot more popular with Nagios Core 3.x, not so much with Core 4.x.)

This is just introducing another variable for issues due to version mismatch.

[rajasegar]

Configured 2 worker servers to handle hostgroup_LOAD_BALANCER_MSB

Code: Select all

2015-04-02 11:59:32  -  localhost:4730  -  v1.1.8

 Queue Name                  | Worker Available | Jobs Waiting | Jobs Running
------------------------------------------------------------------------------
 check_results               |               2  |           0  |           0
 eventhandler                |             166  |           0  |           0
 host                        |              67  |           0  |           3
 hostgroup_LOAD_BALANCER_MSB |              99  |           0  |           0
 service                     |              67  |           0  |          11
 worker_nagiosprodcol1       |               1  |           0  |           0
 worker_nagiosprodcol2       |               1  |           0  |           0
 worker_nagiosprodxi1        |               1  |           0  |           0
------------------------------------------------------------------------------

It is working fine now.
Not sure eventhandler, check_results etc config is sufficient or not.

No improvement in latency but the load is a lot lower now.
Events graphs seems to be the same pattern.

2015-04-02_12-02-14.png

[cmerchant]

No improvement in latency but the load is a lot lower now.
Events graphs seems to be the same pattern.

Comparing the performance from your other thread, from a day earlier,
http://support.nagios.com/forum/viewtop ... 10#p132811

You went from Max Service Latency of 6.29 to 2.65 seconds
Avg Service Latency of 0.60 to 0.53 seconds

Service Check Execution Max Time from 100.57 to 60.04 seconds
Avg Service Check Execution Time from 1.14 to 1.05 seconds

I would say it was an improvement. Percentage wise, how much did you offload to the load balance mod-gearman workers?

[rajasegar]

No improvement in latency but the load is a lot lower now.
Events graphs seems to be the same pattern.

Comparing the performance from your other thread, from a day earlier,
http://support.nagios.com/forum/viewtop ... 10#p132811

You went from Max Service Latency of 6.29 to 2.65 seconds
Avg Service Latency of 0.60 to 0.53 seconds

Service Check Execution Max Time from 100.57 to 60.04 seconds
Avg Service Check Execution Time from 1.14 to 1.05 seconds

I would say it was an improvement. Percentage wise, how much did you offload to the load balance mod-gearman workers?

About 20% of the servers.

Host and service execution latency average have always been around 0.51 - 0.60 seconds

[cmerchant]

Having the checks offloaded to a mod-gearman worker server(s) does not necessarily speed up the individual checks (unless you moved them to faster CPU(s)), it would improve the latency time (queue time) if you increased the number of workers. Can we go ahead and close this thread?

[rajasegar]

Having the checks offloaded to a mod-gearman worker server(s) does not necessarily speed up the individual checks (unless you moved them to faster CPU(s)), it would improve the latency time (queue time) if you increased the number of workers. Can we go ahead and close this thread?

Ok. Please close the thread.