Page 1 of 2

mod_gearman in NagiosXi 5.7.1

Posted: Thu Jun 25, 2020 4:14 am
by alessandrotosi
Hi everyone,
I upgraded to a test environment. Unfortunately the mod_gearman configuration seems not to work. Is this mode still supported?
I think the problem is in the broker configuration in the nagios.cfg file but if set to use gearman mode the process engine remains stopped.

Workers seem to continue working but the results are not reported.

Re: mod_gearman in NagiosXi 5.7.1

Posted: Thu Jun 25, 2020 2:52 pm
by benjaminsmith
Hi,

It's still supported, however, there's a known issue right now in 5.7.1 with Mod Gearman, please open the nagios configuration file and make sure that ndo is loaded before Mod Gearman, and let me know if the issue is resolved.

Code: Select all

broker_module=/usr/local/nagios/bin/ndo.so /usr/local/nagios/etc/ndo.cfg
broker_module=/usr/lib64/mod_gearman/mod_gearman_nagios4.o config=/etc/mod_gearman/module.conf eventhandler=no

Re: mod_gearman in NagiosXi 5.7.1

Posted: Fri Jun 26, 2020 5:51 am
by alessandrotosi
thanks,
before ndo it came after gearman. I tried to modify as indicated but nothing has changed

Re: mod_gearman in NagiosXi 5.7.1

Posted: Fri Jun 26, 2020 1:49 pm
by lmiltchev
Have you tried restarting the modgearman daemon, worker, and nagios service after making the changes in the config file?

Re: mod_gearman in NagiosXi 5.7.1

Posted: Mon Jun 29, 2020 1:06 am
by alessandrotosi
Yes, of course

Re: mod_gearman in NagiosXi 5.7.1

Posted: Mon Jun 29, 2020 5:17 am
by YannPingot
Hello,

We've also upgraded NagiosXI to the 5.7.1 in our test environment and gearman is not working anymore either.

I've tried compiling the latest version just to be sure, the module seems to be loaded fine but checks are not handled (hostgroups based).
As mentioned i've tried to load ndo before but no luck.

I think the new ndo3 is the faulty one here, and i can't find any documentation about it except a small readme in the zip sources, not really useful.

NDO is also outputting a lot of errors in nagios.log (both collations and charset in ndo.cfg are UTF8) :

Code: Select all

[1593425475] NDO-3: ndo_return = 1 (Incorrect string value: '\xE2\x97\x8F cr...' for column 'output' at row 1)
[1593425475] NDO-3: ndo_handle_service_status(ndo-handlers.c:953): Unable to execute statement
[1593425497] NDO-3: ndo_return = 1 (Incorrect string value: '\xE2\x97\x8F ht...' for column 'output' at row 1)
[1593425497] NDO-3: ndo_handle_service_status(ndo-handlers.c:953): Unable to execute statement
[1593425522] NDO-3: ndo_return = 1 (Incorrect string value: '\xE2\x97\x8F ht...' for column 'output' at row 1)
[1593425522] NDO-3: ndo_handle_service_status(ndo-handlers.c:953): Unable to execute statement
[1593425522] NDO-3: ndo_return = 1 (Incorrect string value: '\xE2\x97\x8F ht...' for column 'output' at row 1)
[1593425522] NDO-3: ndo_handle_service_check(ndo-handlers.c:439): Unable to execute statement
[1593425522] NDO-3: ndo_return = 1 (Incorrect string value: '\xE2\x97\x8F ht...' for column 'output' at row 1)
[1593425522] NDO-3: ndo_handle_service_status(ndo-handlers.c:953): Unable to execute statement
Thanks,

Re: mod_gearman in NagiosXi 5.7.1

Posted: Mon Jun 29, 2020 1:50 pm
by swolf
Hi @alessandrotosi and @YannPingot,

Developer here. Thanks for reporting this, that's useful information for us. For what it's worth, I agree with your assessment that it's NDO 3 that needs to behave better.

If possible, please send me a private message with your system profile attached. You can download this by logging into Nagios XI and navigating to Admin->System Config->System Profile. The trick that @benjaminsmith shared (switching the broker_module lines) has worked on all of the lab machines that I've seen, so hopefully with the profile we'll be able to set up a system that reproduces the same problem you're both seeing.

Once you've sent me the profile, feel free to temporarily revert back to using NDO 2 until we have this fixed. If your XI system has internet access, here's the easiest way to downgrade:

In your terminal, run the following commands:

Code: Select all

systemctl stop nagios
cd /tmp
rm -rf /tmp/nagiosxi
wget https://assets.nagios.com/downloads/nagiosxi/5/xi-5.6.14.tar.gz
tar zxf xi-5.6.14.tar.gz
cd /tmp/nagiosxi
./init.sh
cd /tmp/nagiosxi/subcomponents/ndoutils
./install
systemctl enable ndo2db
systemctl start ndo2db
If you use CentOS 6, or some older version of Ubuntu/Debian, you will need to change the systemd commands to the equivalent commands in whatever service manager is being used. Let me know if you need help with that.

Then, in nagios.cfg, find the commented NDO 2 broker module line, and uncomment it. By default, it looks like this:

Code: Select all

#broker_module=/usr/local/nagios/bin/ndomod.o config_file=/usr/local/nagios/etc/ndomod.cfg
Then, find the NDO 3 broker module line, and comment it. It should look close to this:

Code: Select all

broker_module=/usr/local/nagios/bin/ndo.so /usr/local/nagios/etc/ndo.cfg
Last, restart the nagios service:

Code: Select all

service nagios restart
Sorry for the inconvenience you've had so far. Hopefully we'll be able to resolve this soon.

Re: mod_gearman in NagiosXi 5.7.1

Posted: Tue Jun 30, 2020 11:25 am
by alessandrotosi
Thank you,

I tried to go back as indicated in the previous post but the procedure fails due to the configuration of the database installed on a separate machine (Offloading MySQL to a remote server)

Re: mod_gearman in NagiosXi 5.7.1

Posted: Tue Jun 30, 2020 5:08 pm
by swolf
@alessandrotosi,

I can confirm that I've received your profile. I haven't had a chance to set up a system using the configuration, but I'll get back to you (hopefully tomorrow).

As for the failed NDO downgrade, I'm not sure why it would fail for an offloaded database. After the install, please verify that your ndo2db.cfg settings are as they were before the upgrade. Also, if you were running ndo2db on the remote database's server, then you try to restart the ndo2db service on the remote database machine instead of on the Nagios XI server.

If you're still having issues with the downgrade, let me know of any specific error messages you're seeing.

@YannPingot,

Another technician says that he's seen errors similar to what you provided earlier, and that it's possible that the database schema upgrade failed. We would like you to try to rerun the schema upgrade manually. You can run the following commands as root:

Code: Select all

cd /tmp
rm -rf nagiosxi xi*.tar.gz
wget https://assets.nagios.com/downloads/nagiosxi/5/xi-5.7.1.tar.gz
tar xzf xi-5.7.1.tar.gz
cd /tmp/nagiosxi/subcomponents/ndo/ndo-3.0.1/db
mysql -u root -pnagiosxi nagios <upgrade-from-2.1.3.sql
mysql -u root -pnagiosxi nagios <upgrade-from-3.0.0.sql
systemctl restart nagios
If you're using an offloaded database, or if you've configured a custom password, make sure to change the "mysql" commands' options to take those settings into account. Let us know if this resolves your issue or if you see errors (either during the schema change or after upgrading).

Re: mod_gearman in NagiosXi 5.7.1

Posted: Wed Jul 01, 2020 3:47 am
by YannPingot
Hi @swolf,

I've passed again each sql script, i have an error with the first one (upgrade-from-2.1.3.sql), maybe this is the cause since it's trying to push a VARCHAR(1023), this script halts :

Code: Select all

Enter password:
INFO: Column already exists.
INFO: Column already exists.
ERROR 1071 (42000) at line 146: Specified key was too long; max key length is 1000 bytes
I can try reducing the length below 1000 but it may have an impact elsewhere ?

The other one just consists of few "Column already exists" that shouldn't be a problem.

Thanks,