Page 1 of 3

mod-gearman |showing false device down

Posted: Mon Apr 29, 2019 4:49 am
by progressive.nagiosXI
Hi Team,

sometime nagiosxi showing devices down but only for the devices monitor through mod-gearman .

host output :-host check orphaned ,is the mod-gearman worker on queue hostgroup running

CPU also normal at that time in Nagiosxi.

the issue is resolved only after nagiosxi server restart(init 6).


Thanks

Re: mod-gearman |showing false device down

Posted: Mon Apr 29, 2019 2:13 pm
by tgriep
The message that you are receiving usually means that the Gearman worker is not returning the data for a check or that the Gearman server is not passing it on to the Nagios Process.

To trouble shoot this, you would have to enable debugging for Mod Gearman and check the log files.

Edit the /etc/mod_gearman2/module.conf file on the Gearman server (Nagios) and enable debugging by changing the following from

Code: Select all

debug=0
to

Code: Select all

debug=1
Save the file and restart the Gearman server

Code: Select all

service gearmand restart

On the Gearman Worker, edit the
/etc/mod_gearman2/worker.conf file and change the following from

Code: Select all

debug=0
to

Code: Select all

debug=1
Save the file and restart the Gearman worker.

Code: Select all

service mod-gearman2-worker restart
Check the log files in the /var/log/mod_gearman2 folder on the Gearman Worker and Gearman server for any errors when the Orphan message is displayed in XI.

Re: mod-gearman |showing false device down

Posted: Wed May 01, 2019 2:34 am
by progressive.nagiosXI
Hi,

we successfully done given steps in workers.but getting gearmand restart failed in nagiosxi server.

Note:when we check status of gearmand service it always show in failed state and in htop command gearmand service running as per polling interval.


Thanks

Re: mod-gearman |showing false device down

Posted: Wed May 01, 2019 8:49 am
by tgriep
It looks like there could be a permission problem for the file / folder where the gearmand server is trying to save the pip file.
Check the /var/run/gearmand folder to see if it exists and that the gearmand user account can create files in it.

Also, check the /var/log/gearmand/gearmand.log file for any errors and either fix them or post them here so we can view them.

You can try running this command as root to force the gearmand daemon to shutdown.

Code: Select all

killall -9 gearmand
Then start it up

Code: Select all

service gearmand start
Then check to see if it is running.

Re: mod-gearman |showing false device down

Posted: Thu May 02, 2019 12:20 am
by progressive.nagiosXI
Hi ,

1==> /var/run/gearmand folder not found.
2==> after pkill -9 gearmand all gearmand process stopped and also not showing in HTOP command.
3==>service gearmand start showing failed .
4==>after above steps nagios misbehaving and showing blue screen in Nagvis and gearmand process still not showing in HTOP as its showing before kill command (running as per polling time).
5==> after init 6 everything ok.

Thanks

Re: mod-gearman |showing false device down

Posted: Thu May 02, 2019 8:40 am
by tgriep
After the reboot, I want to verify, is the gearmand.pid file in the /var/run/gearmand folder?
If not, run the following commands as root to create the folder and set the permissions.

Code: Select all

mkdir /var/run/gearmand
chown gearmand.gearmand /var/run/gearmand
chmod 755 /var/run/gearmand
Then restart the gearmand server and verify that the gearmand.pid file exists.

If it works, let us know and also if everything else is working as well.

Re: mod-gearman |showing false device down

Posted: Thu May 02, 2019 11:38 am
by progressive.nagiosXI
Hi ,

Please check attached command logs running

gearmand folder/dir auto removed after init 6.

Thanks

Re: mod-gearman |showing false device down

Posted: Thu May 02, 2019 2:28 pm
by tgriep
Edit the gearmand init script

Code: Select all

/etc/init.d/gearmand
Change of the occurrences of this

Code: Select all

/var/run/gearmand/gearmand.pid
to this

Code: Select all

/var/run/gearmand.pid
Save the change when the gearmand daemon starts, it will be able to create the pid file in the /var/run folder instead.

Re: mod-gearman |showing false device down

Posted: Fri May 03, 2019 11:51 am
by progressive.nagiosXI
Hi

Edit the gearmand init script done
from :- /var/run/gearmand/gearmand.pid
with :- /var/run/gearmand.pid

but still not able to start/restart gearmand service.

Thanks

Re: mod-gearman |showing false device down

Posted: Fri May 03, 2019 1:38 pm
by ssax
Please do this instead:

Edit the gearmand init script:

Code: Select all

/etc/init.d/gearmand
Change of the occurrences of the gearman.pid to this:

Code: Select all

/var/run/gearmand/gearmand.pid
Save the change when the gearmand daemon starts, it will be able to create the pid file in the /var/run/gearmand folder instead.