sucessful migration - mysql issues

[ssax]

I would still follow the commands to see if that resolves it for you on the 5.7.1 system.

Once nagios is stopped, restart mysql/mariadb as well to kill the existing connections.

[veehexx]

reverting to ndo2db has fixed both sql load issue and the 'no data to display' in the hosts&service status screens has gone and is correctly showing the expected list of Hosts & Services.

almost there!

1 last issue remaing now... Monitoring Engine Process is stuck in a stopped state. Obviously no checks are being done with the process in a stopped state but it wont start via the webUI.

EDIT: ME Process took a long time to wake up after restarting after the ndo2db downgrade - maybe 15 minutes. gave VM a reboot and while it seems to take longer to report on this process status than a v5.6.x release, the webUI only took 2 minutes to update the status.

there does seem to be something off there though. Take a look at the following screenshot. Process start time was immediately after reboot, however it the webUI only turned to green status at about 9:42:30; 90-120secs after the service actually started. Until 9:45:30, the process state (and related pages like birdseye) remained as red/stopped state.

MEprocess.PNG

it currently looks like everything is working. i just need to update my nrpe settings for ssl handshake (i already know a fix) and i should be good if it continues to remain stable.

one last question - upon the next release of nagios (5.7.2?), what do i need to do for these NDO changes? should i manually revert the nagios.cfg file to ndo3 and perform upgrade or will the updater handle this itself?

update: same delay on the ME Process when you apply configuration. status page says it's not running and took 2m20secs for it to update and show green/running. process start time aligns with immediately after apply configuration completed (2m20sec ago).

[ssax]

That's normal, when you apply config it writes out the DB changes to the file system then restarts the nagios service, which then starts updating all objects the DB through (NDO2DB), once that's done it will calculate limited user permissions/CCM permissions which could take up to a few minutes on larger systems.

I think once you upgrade to 5.7.2 (once 5.7.2 is released) if it doesn't upgrade that for you, you should be able to do this after upgrading:

Code: Select all

systemctl stop nagios
systemctl stop ndo2db
cd /tmp
rm -rf /tmp/nagiosxi
wget https://assets.nagios.com/downloads/nagiosxi/5/xi-5.7.2.tar.gz
tar zxf xi-5.7.2.tar.gz
cd /tmp/nagiosxi
./init.sh
cd /tmp/nagiosxi/subcomponents/ndo
./install
systemctl disable ndo2db
rm -f /etc/init.d/ndo2db
rm -f /usr/lib/systemd/system/ndo2db.service
systemctl daemon-reload
systemctl start nagios

[veehexx]

we've had the issue reoccur on 5.7.2 update

re-ran through saxx's solution on page3 (https://support.nagios.com/forum/viewto ... 01#p312201 - specifically quotes 2 through 5) and after 20-30minutes nagios became functional again

guess theres still issues with NDO3, and nice the downgrade to ndo2db worked a second time.

Just bringing this to your attention that it's still an issue for us and needed re-fixing. We're back online working so no support needed, just an FYI

[ssax]

Some systems have issues on 5.7.2 (not everyone has the issue), the only thing you can do is downgrade via the instructions at this point in time.

The developers are actively working it and we expect a fix to be released in 5.7.3.

I apologize for any inconvenience.

Thank you for reporting that you needed to do that.

Do you see this error in your /usr/local/nagios/var/nagios.log when it was failing?

Code: Select all

NDO-3: Query failed in ndo_empty_queue_timed_event

[veehexx]

yes, seeing those. Were in the /usr/local/nagios/var/archives/ location.

[bpennin]

we've had the issue reoccur on 5.7.2 update

re-ran through saxx's solution on page3 (https://support.nagios.com/forum/viewto ... 01#p312201 - specifically quotes 2 through 5) and after 20-30minutes nagios became functional again

guess theres still issues with NDO3, and nice the downgrade to ndo2db worked a second time.

Just bringing this to your attention that it's still an issue for us and needed re-fixing. We're back online working so no support needed, just an FYI

Thanks for sharing all of this information as I am still sitting at 5.7.1 and will be updating to 5.7.2. Much appreciated!

[bpennin]

Curious if this was an upgrade from 5.6.x to 5.7 and if the problem exists with both upgrades and clean installs from scratch? I am at the point of loading my 5.7 server up but will blow it away if the clean install is a better way to go. My server was a restore/upgrade so I could pull all the scripts and commands we created on a previous Nagios server. Thanks again!

[veehexx]

Curious if this was an upgrade from 5.6.x to 5.7 and if the problem exists with both upgrades and clean installs from scratch? I am at the point of loading my 5.7 server up but will blow it away if the clean install is a better way to go. My server was a restore/upgrade so I could pull all the scripts and commands we created on a previous Nagios server. Thanks again!

we went fresh install.
from nagios's pre-configured centos6 & xi VM we deployed years ago, to our own centos8stream + nagiosXI install as per the install docs. i figured C8S is the way to go as its a rolling releas so should avoid us having to migrate in the future.

[benjaminsmith]

Hi @veehexx,

Thanks for sharing that information. We should have a patch out shortly for the error you were seeing and appreciate your patience.