Page 2 of 5
Re: gearmand failed to load module
Posted: Fri Jan 30, 2015 12:12 pm
by emartine
Yes it is running.
debug=0
logfile=/var/log/mod_gearman/mod_gearman_worker.log
server=127.0.0.1:4730
eventhandler=yes
services=yes
hosts=yes
encryption=yes
key=sameasislistedinmod_gearman_neb.conf
job_timeout=60
min-worker=5
max-worker=50
idle-timeout=30
max-jobs=1000
spawn-rate=1
fork_on_exec=no
load_limit1=0
load_limit5=0
load_limit15=0
show_error_output=yes
enable_embedded_perl=on
use_embedded_perl_implicitly=off
use_perl_cache=on
p1_file=/usr/share/mod_gearman/mod_gearman_p1.pl
workaround_rc_25=off
Re: gearmand failed to load module
Posted: Fri Jan 30, 2015 12:31 pm
by tgriep
Which one on these workers are you having problems with?
worker_nagiosxi-tst or worker_gearmanworker-tst
Which server is the gearmand server running on?
Re: gearmand failed to load module
Posted: Fri Jan 30, 2015 10:59 pm
by emartine
the worker_nagiosxi-tst. THis same server runs gearmand.
Re: gearmand failed to load module
Posted: Mon Feb 02, 2015 9:49 am
by lmiltchev
You could try changing this line:
to this:
Technically it's the same, but "localhost" is used by default in the config, so give it a try.
Also, you can temporarily start debugging:
Restart the server, worker and nagios, and check the "/var/log/mod_gearman/mod_gearman_neb.log" for clues.
Re: gearmand failed to load module
Posted: Mon Feb 02, 2015 12:28 pm
by emartine
OK.
I was able to get both workers working setting the configuration files:
Remote worker :/etc/mod_gearman/mod_gearman_worker.conf --> server=<IPADDRESS>:4730
Local server worker :/etc/mod_gearman/mod_gearman_worker.conf --> server=<IPADDRESS>:4730
Local server gearmand: /etc/mod_gearman/mod_gearman_neb.conf --> server=localhost:4730
Local server gearmand sysconfig: /etc/sysconfig/gearmand --> OPTIONS="-L <IPADDRESS>"
Both workers are now running and gearmand is running but I am still unable to submit commands in Nagios (I still get - your request was not processed in a timely manner - message).
Not sure if this is relevant or not but I am getting this error from the mod_gearman_neb.log This was before 11 and I have restarted gearmand several times since then.
tail -f /var/log/mod_gearman/mod_gearman_neb.log
[2015-02-02 09:07:49][2369][ERROR] sending job to gearmand failed: flush(GEARMAN_COULD_NOT_CONNECT) localhost:127 -> libgearman/connection.cc:745
[2015-02-02 09:29:00][7211][ERROR] sending job to gearmand failed: flush(GEARMAN_COULD_NOT_CONNECT) localhost:127 -> libgearman/connection.cc:745
[2015-02-02 09:54:41][13142][ERROR] sending job to gearmand failed: flush(GEARMAN_COULD_NOT_CONNECT) localhost:127 -> libgearman/connection.cc:745
[2015-02-02 09:55:32][13438][ERROR] sending job to gearmand failed: flush(GEARMAN_COULD_NOT_CONNECT) localhost:127 -> libgearman/connection.cc:745
[2015-02-02 09:57:57][2286][ERROR] sending job to gearmand failed: flush(GEARMAN_COULD_NOT_CONNECT) localhost:127 -> libgearman/connection.cc:745
[2015-02-02 10:32:47][10333][ERROR] sending job to gearmand failed: flush(GEARMAN_COULD_NOT_CONNECT) localhost:127 -> libgearman/connection.cc:745
[2015-02-02 10:58:39][16314][ERROR] sending job to gearmand failed: flush(GEARMAN_COULD_NOT_CONNECT) localhost:127 -> libgearman/connection.cc:745
[2015-02-02 10:59:18][16674][ERROR] sending job to gearmand failed: flush(GEARMAN_COULD_NOT_CONNECT) localhost:127 -> libgearman/connection.cc:745
Re: gearmand failed to load module
Posted: Mon Feb 02, 2015 1:10 pm
by scottwilkerson
is gearmand running on the XI server?
Can you run gearman_top on the XI server?
Re: gearmand failed to load module
Posted: Mon Feb 02, 2015 1:19 pm
by tgriep
Try removing this from the sysconfig file on the gearmand server /etc/sysconfig/gearmand --> OPTIONS="-L <IPADDRESS>"
Them restart the gearman server by running
Re: gearmand failed to load module
Posted: Mon Feb 02, 2015 1:23 pm
by emartine
Gearmand is running per my previous post.
Queue Name | Worker Available | Jobs Waiting | Jobs Running
--------------------------------------------------------------------------------------
eventhandler | 21 | 0 | 0
host | 21 | 0 | 0
service | 21 | 0 | 0
worker_nagiosxi-tst | 1 | 0 | 0
worker_nxitst01dar | 1 | 0 | 0
--------------------------------------------------------------------------------------
Re: gearmand failed to load module
Posted: Mon Feb 02, 2015 1:29 pm
by emartine
The worker log seems like it doesn't have checks going through.
==> /var/log/mod_gearman/mod_gearman_worker.log <==
[2015-02-02 12:09:35][2128][INFO ] no checks in 2minutes, restarting all workers
[2015-02-02 12:11:36][2128][INFO ] no checks in 2minutes, restarting all workers
[2015-02-02 12:13:37][2128][INFO ] no checks in 2minutes, restarting all workers
[2015-02-02 12:15:38][2128][INFO ] no checks in 2minutes, restarting all workers
[2015-02-02 12:17:39][2128][INFO ] no checks in 2minutes, restarting all workers
[2015-02-02 12:19:40][2128][INFO ] no checks in 2minutes, restarting all workers
[2015-02-02 12:21:41][2128][INFO ] no checks in 2minutes, restarting all workers
[2015-02-02 12:23:42][2128][INFO ] no checks in 2minutes, restarting all workers
[2015-02-02 12:25:43][2128][INFO ] no checks in 2minutes, restarting all workers
[2015-02-02 12:27:44][2128][INFO ] no checks in 2minutes, restarting all workers
Re: gearmand failed to load module
Posted: Mon Feb 02, 2015 1:35 pm
by emartine
Seems like everything is running ok.