Mode Gearman

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
FranckB1
Posts: 25
Joined: Mon Feb 19, 2018 3:06 am

Mode Gearman

Post by FranckB1 »

Hello,

I'm srtuggling with running Nagios in Mod_Gearman.

Before going deeper in details with my configuration files (mod_gearman_neb.conf and mod_gearman_worker.conf), I would like to submit on the forum the issue I got when I run the following command : su nagios -c /usr/bin/mod_gearman_worker start (in order to launch the worker on my Nagios Server).

The first output line is :
[2018-04-05 14:34:06][30683][ERROR] please specify at least one server

As a result, in my view, there is no launched worker process.
If I issue the command "ps -e|greap gearman", I just get one line with the "gearmand" process, which is not the worker...

And finally, in my GUI (namely Eyes Of Networks), all my hosts are DOWN, with the same message for each one :
(host check orphaned, is the mod-gearman worker on queue 'hostgroup_hostgroup2-servers' running?)
This seems to be logical, considering the problem above...

Maybe does someone know the origin of this mess, and why do I get the message "please specify at least one server" when attempting to launch the worker ?

Thanks for help.
FranckB1
Posts: 25
Joined: Mon Feb 19, 2018 3:06 am

Re: Mode Gearman

Post by FranckB1 »

As I keep on searching, I found another way to launch the worker :
by issuing the command "systemctl start mod_gearman_worker"

It started to run, my hosts appears UP (what a miracle !), and after a while they went back to DOWN...
This time with a different output message : (host check orphaned, is the mod-gearman worker on queue 'hostgroups_hostgroup1-servers' running?)

Where hostgroup1-servers is the group in which I put the localhost in...

Thanks for help.
npolovenko
Support Tech
Posts: 3457
Joined: Mon May 15, 2017 5:00 pm

Re: Mode Gearman

Post by npolovenko »

Hello, @ FranckB1. So as of right now your mod gearman server and workers are on the same server, correct? Please run the following command to see if mod gearman worker is still running:

Code: Select all

systemctl status mod_gearman_worker
Please upload all the mod gearman logs which can usually be found in

Code: Select all

/var/log/mod_gearman2/
What's the output of gearman_top2 ?
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Mode Gearman

Post by scottwilkerson »

What guide did you follow to setup mod_gearman?

there are so many different configuration options we would need to know all the versions of everything you are using and you would also need to share both your mod_gearman_neb.conf and mod_gearman_worker.conf
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
FranckB1
Posts: 25
Joined: Mon Feb 19, 2018 3:06 am

Re: Mode Gearman

Post by FranckB1 »

Hi Scott,

Thank for your response.

I followed a guide from nicolargo blog. I don't know if it could help you, it's a french written blog (I'm a french guy ;) ...)
=>https://blog.nicolargo.com/2013/09/dist ... arman.html

Maybe you can advise me a better guide (don't matter if it's an english one).

Below the output of the systemctl status mod_gearman_worker command :

Code: Select all

● mod_gearman_worker.service - Mod-Gearman Worker
   Loaded: loaded (/usr/lib/systemd/system/mod_gearman_worker.service; enabled;                  vendor preset: enabled)
   Active: active (running) since jeu. 2018-04-05 17:35:24 CEST; 14h ago
     Docs: http://mod-gearman.org/docs.html
  Process: 9808 ExecStart=/usr/bin/mod_gearman_worker -d --config=/etc/mod_gearm                 an/mod_gearman_worker.conf --pidfile=/var/mod_gearman/mod_gearman_worker.pid (co                 de=exited, status=0/SUCCESS)
 Main PID: 9809 (mod_gearman_wor)
   CGroup: /system.slice/mod_gearman_worker.service
           ├─ 9809 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...
           ├─32600 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...
           ├─32613 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...
           ├─32614 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...
           ├─32615 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...
           ├─32616 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...
           └─32617 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...

avril 05 17:35:24 eon2 systemd[1]: Starting Mod-Gearman Worker...
avril 05 17:35:24 eon2 systemd[1]: PID file /var/mod_gearman/mod_gearman_wo...t.
avril 05 17:35:24 eon2 systemd[1]: Started Mod-Gearman Worker.
Hint: Some lines were ellipsized, use -l to show in full.
Concerning software versions :
Nagios is 3.5.1
Mod_gearman 1.5.5

Let me know if you want more info about versions.

See attached log an config files...
Attachments
mod_gearman_worker.conf
(5.72 KiB) Downloaded 365 times
mod_gearman_neb.log
(427.63 KiB) Downloaded 365 times
mod_gearman_neb.conf
(5.7 KiB) Downloaded 350 times
FranckB1
Posts: 25
Joined: Mon Feb 19, 2018 3:06 am

Re: Mode Gearman

Post by FranckB1 »

And finally the mod_gearman_worker.log....
Attachments
mod_gearman_worker.log
(172.96 KiB) Downloaded 340 times
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Mode Gearman

Post by scottwilkerson »

I can see other checks going through fine.

I wonder if it has to do with how localhost is handled because it is handled differently because usually people don't want the localhost check to run on the worker, becaus ethat would actually report the workers state, NOT the nagios servers state.

These versions of XI and mod_gearman are several years old so I am a bit rusty remembering how the settings work, but this all looks correct.
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
FranckB1
Posts: 25
Joined: Mon Feb 19, 2018 3:06 am

Re: Mode Gearman

Post by FranckB1 »

Hi Scott,

Thank you for responding to me.
usually people don't want the localhost check to run on the worker
I've followed the instruction of the guide I've found, I don't know if it's relevant but in this documentation (https://assets.nagios.com/downloads/nag ... ios_XI.pdf, they also talk about running gearmand and mod_gearman_worker on the same host.

So, in order to have a clear head on this monday morning, I'll try to sum'up my configuration (which is still not running !)

Thus, on the main server, gearmand and mod_gearman_worker are running and configured to point on the localhost.
2 hostgroups configured in Nagios, one group for local hosts (hostgroup1) and another one for remote hosts (hostgroup2).

The event broker is set up, but only on the main server
=> /usr/lib64/mod_gearman/mod_gearman.o config=/etc/mod_gearman/mod_gearman_neb.conf

On the secondary server, only mod_gearman_worker is running, and configured, this time, for pointing on the main server.

One group is configured, hostgroup2, which is the same as hostgroup2 on the main server.

One final detail : in the mod_gearman_neb.conf on the main server, the parameter queue_custom_variable is set to WORKER.
And in the hosts.cfg file (still of the main server), each host has this kind of line in its definition :
_WORKER hostgroup_hostgroup1-servers #for the local host
_WORKER hostgroup_hostgroup2-servers #for the remote hosts.

That's it !

Thanks for help.
FranckB1
Posts: 25
Joined: Mon Feb 19, 2018 3:06 am

Re: Mode Gearman

Post by FranckB1 »

Another thing that seems weird to me.

I'm trying to run Nagios in mod_gearman because I need a distributed architecture, and above all, many of the hosts I want to supervise will be protected by a firewall, or hidden behind a single public IP.

The more I read posts and documentations on the internet, the less I'm sure that I could use mod_gearman to meet that requirement, as if all the IP of the supervised hosts would have to be known on the central server.

I know this hypothesis could seem to be dumb, but when I try to create a remot host in the Nagios central server configuration file, without configuring a basic check and so providing a valid IP address for this host, nothing work...
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: Mode Gearman

Post by tgriep »

One suggestion, I would remove the hostgroup2-servers hostgroup from the Gearman Worker that is running on the Nagios server.
If that is the group you want to run on the remote worker, you do not want it to run the checks on the local server.

Your last post about the distributed architecture, you have to think of the checks being run on the remote gearman worker and the IP address that have to be checked have to be accessible from that system.
What you want to do is setup the Hostgroup to only run the checks you want on that remote worker and also set these 2 options to no.

Code: Select all

services=no
hosts=no
That way, the remote worker will only run the checks in the group and not any checks from the generic hosts and services queue and that way you should not have any issues.

Does that help out?
Be sure to check out our Knowledgebase for helpful articles and solutions!
Locked