Mode Gearman

An open discussion forum for obtaining help with Nagios Core. Nagios Core users of all experience levels are welcome here. Subforum have been created for the discussion of Nagios Core and Nagios Plugin development.

NOTE: The SourceForge.net mailing lists have been deprecated in favor of this forum in order to expedite support and provide additional features not available on the old mailing list.

Mode Gearman

Postby FranckB1 » Thu Apr 05, 2018 8:06 am

Hello,

I'm srtuggling with running Nagios in Mod_Gearman.

Before going deeper in details with my configuration files (mod_gearman_neb.conf and mod_gearman_worker.conf), I would like to submit on the forum the issue I got when I run the following command : su nagios -c /usr/bin/mod_gearman_worker start (in order to launch the worker on my Nagios Server).

The first output line is :
[2018-04-05 14:34:06][30683][ERROR] please specify at least one server

As a result, in my view, there is no launched worker process.
If I issue the command "ps -e|greap gearman", I just get one line with the "gearmand" process, which is not the worker...

And finally, in my GUI (namely Eyes Of Networks), all my hosts are DOWN, with the same message for each one :
(host check orphaned, is the mod-gearman worker on queue 'hostgroup_hostgroup2-servers' running?)
This seems to be logical, considering the problem above...

Maybe does someone know the origin of this mess, and why do I get the message "please specify at least one server" when attempting to launch the worker ?

Thanks for help.
FranckB1
 
Posts: 18
Joined: Mon Feb 19, 2018 3:06 am

Re: Mode Gearman

Postby FranckB1 » Thu Apr 05, 2018 10:53 am

As I keep on searching, I found another way to launch the worker :
by issuing the command "systemctl start mod_gearman_worker"

It started to run, my hosts appears UP (what a miracle !), and after a while they went back to DOWN...
This time with a different output message : (host check orphaned, is the mod-gearman worker on queue 'hostgroups_hostgroup1-servers' running?)

Where hostgroup1-servers is the group in which I put the localhost in...

Thanks for help.
FranckB1
 
Posts: 18
Joined: Mon Feb 19, 2018 3:06 am

Re: Mode Gearman

Postby npolovenko » Thu Apr 05, 2018 12:36 pm

Hello, @ FranckB1. So as of right now your mod gearman server and workers are on the same server, correct? Please run the following command to see if mod gearman worker is still running:
Code: Select all
systemctl status mod_gearman_worker

Please upload all the mod gearman logs which can usually be found in
Code: Select all
/var/log/mod_gearman2/

What's the output of gearman_top2 ?
User avatar
npolovenko
Support Tech
 
Posts: 1518
Joined: Mon May 15, 2017 5:00 pm

Re: Mode Gearman

Postby scottwilkerson » Thu Apr 05, 2018 12:39 pm

What guide did you follow to setup mod_gearman?

there are so many different configuration options we would need to know all the versions of everything you are using and you would also need to share both your mod_gearman_neb.conf and mod_gearman_worker.conf
User avatar
scottwilkerson
CTO
 
Posts: 8921
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises

Re: Mode Gearman

Postby FranckB1 » Fri Apr 06, 2018 3:42 am

Hi Scott,

Thank for your response.

I followed a guide from nicolargo blog. I don't know if it could help you, it's a french written blog (I'm a french guy ;) ...)
=>https://blog.nicolargo.com/2013/09/distributed-monitoring-nagios-laide-mod-gearman.html

Maybe you can advise me a better guide (don't matter if it's an english one).

Below the output of the systemctl status mod_gearman_worker command :
Code: Select all
● mod_gearman_worker.service - Mod-Gearman Worker
   Loaded: loaded (/usr/lib/systemd/system/mod_gearman_worker.service; enabled;                  vendor preset: enabled)
   Active: active (running) since jeu. 2018-04-05 17:35:24 CEST; 14h ago
     Docs: http://mod-gearman.org/docs.html
  Process: 9808 ExecStart=/usr/bin/mod_gearman_worker -d --config=/etc/mod_gearm                 an/mod_gearman_worker.conf --pidfile=/var/mod_gearman/mod_gearman_worker.pid (co                 de=exited, status=0/SUCCESS)
Main PID: 9809 (mod_gearman_wor)
   CGroup: /system.slice/mod_gearman_worker.service
           ├─ 9809 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...
           ├─32600 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...
           ├─32613 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...
           ├─32614 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...
           ├─32615 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...
           ├─32616 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...
           └─32617 /usr/bin/mod_gearman_worker -d --config=/etc/mod_gearman/m...

avril 05 17:35:24 eon2 systemd[1]: Starting Mod-Gearman Worker...
avril 05 17:35:24 eon2 systemd[1]: PID file /var/mod_gearman/mod_gearman_wo...t.
avril 05 17:35:24 eon2 systemd[1]: Started Mod-Gearman Worker.
Hint: Some lines were ellipsized, use -l to show in full.


Concerning software versions :
Nagios is 3.5.1
Mod_gearman 1.5.5

Let me know if you want more info about versions.

See attached log an config files...
Attachments
mod_gearman_worker.conf
(5.72 KiB) Downloaded 13 times
mod_gearman_neb.log
(427.63 KiB) Downloaded 12 times
mod_gearman_neb.conf
(5.7 KiB) Downloaded 11 times
FranckB1
 
Posts: 18
Joined: Mon Feb 19, 2018 3:06 am

Re: Mode Gearman

Postby FranckB1 » Fri Apr 06, 2018 3:45 am

And finally the mod_gearman_worker.log....
Attachments
mod_gearman_worker.log
(172.96 KiB) Downloaded 11 times
FranckB1
 
Posts: 18
Joined: Mon Feb 19, 2018 3:06 am

Re: Mode Gearman

Postby scottwilkerson » Fri Apr 06, 2018 9:03 am

I can see other checks going through fine.

I wonder if it has to do with how localhost is handled because it is handled differently because usually people don't want the localhost check to run on the worker, becaus ethat would actually report the workers state, NOT the nagios servers state.

These versions of XI and mod_gearman are several years old so I am a bit rusty remembering how the settings work, but this all looks correct.
User avatar
scottwilkerson
CTO
 
Posts: 8921
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises

Re: Mode Gearman

Postby FranckB1 » Mon Apr 09, 2018 4:02 am

Hi Scott,

Thank you for responding to me.

usually people don't want the localhost check to run on the worker

I've followed the instruction of the guide I've found, I don't know if it's relevant but in this documentation (https://assets.nagios.com/downloads/nagiosxi/docs/Integrating_Mod_Gearman_with_Nagios_XI.pdf, they also talk about running gearmand and mod_gearman_worker on the same host.

So, in order to have a clear head on this monday morning, I'll try to sum'up my configuration (which is still not running !)

Thus, on the main server, gearmand and mod_gearman_worker are running and configured to point on the localhost.
2 hostgroups configured in Nagios, one group for local hosts (hostgroup1) and another one for remote hosts (hostgroup2).

The event broker is set up, but only on the main server
=> /usr/lib64/mod_gearman/mod_gearman.o config=/etc/mod_gearman/mod_gearman_neb.conf

On the secondary server, only mod_gearman_worker is running, and configured, this time, for pointing on the main server.

One group is configured, hostgroup2, which is the same as hostgroup2 on the main server.

One final detail : in the mod_gearman_neb.conf on the main server, the parameter queue_custom_variable is set to WORKER.
And in the hosts.cfg file (still of the main server), each host has this kind of line in its definition :
_WORKER hostgroup_hostgroup1-servers #for the local host
_WORKER hostgroup_hostgroup2-servers #for the remote hosts.

That's it !

Thanks for help.
FranckB1
 
Posts: 18
Joined: Mon Feb 19, 2018 3:06 am

Re: Mode Gearman

Postby FranckB1 » Mon Apr 09, 2018 8:06 am

Another thing that seems weird to me.

I'm trying to run Nagios in mod_gearman because I need a distributed architecture, and above all, many of the hosts I want to supervise will be protected by a firewall, or hidden behind a single public IP.

The more I read posts and documentations on the internet, the less I'm sure that I could use mod_gearman to meet that requirement, as if all the IP of the supervised hosts would have to be known on the central server.

I know this hypothesis could seem to be dumb, but when I try to create a remot host in the Nagios central server configuration file, without configuring a basic check and so providing a valid IP address for this host, nothing work...
FranckB1
 
Posts: 18
Joined: Mon Feb 19, 2018 3:06 am

Re: Mode Gearman

Postby tgriep » Mon Apr 09, 2018 4:52 pm

One suggestion, I would remove the hostgroup2-servers hostgroup from the Gearman Worker that is running on the Nagios server.
If that is the group you want to run on the remote worker, you do not want it to run the checks on the local server.

Your last post about the distributed architecture, you have to think of the checks being run on the remote gearman worker and the IP address that have to be checked have to be accessible from that system.
What you want to do is setup the Hostgroup to only run the checks you want on that remote worker and also set these 2 options to no.
Code: Select all
services=no
hosts=no


That way, the remote worker will only run the checks in the group and not any checks from the generic hosts and services queue and that way you should not have any issues.

Does that help out?
Be sure to check out our Knowledgebase for helpful articles and solutions!
User avatar
tgriep
Madmin
 
Posts: 6551
Joined: Thu Oct 30, 2014 9:02 am

Next

Return to Nagios Core

Who is online

Users browsing this forum: iam33 and 25 guests