Non stop issues since upgrading to 5.7.3

[rajasegar]

Since upgrading to 5.7.3 the server is always running out of memory.
12 cores, 16GB memory, 1254 hosts with 8909 services.
It is crashing for no reason also.

ipcs -q is always empty, the monitoring seems to be running fine though.

Code: Select all

[nagios@myucbrnagiapp05 temp]$ ipcs -q
------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages

Please advise how to revert back to 5.6.14.
I have no confidence on the stability of 5.7.3

Moderator's Note: The profile has been shared with the support team but has been removed from the public forum.


Thank you.

[benjaminsmith]

Hi,

Thank you for the profile and sorry to hear about the upgrade issues. I did find the following in the nagios log.

Code: Select all

[1601462102] NDO-3: Unable to prepare statement for query (32): Table './nagios/nagios_contactnotifications' is marked as crashed and last (automatic?) repair failed
[1601462102] NDO-3: Error preparing statements
[1601462102] NDO-3: ndo_handle_contact_notification(ndo-handlers.c:1094): Could not reconnect to MySQL database

The presence of crashed database is causing an issue with database updates, please run the repair script and let me know if you notice an improvement.

Code: Select all

/usr/local/nagiosxi/scripts/repair_databases.sh

If you're still having stability issues, it's safe to downgrade to the previous version of ndo. Here are instructions.

Code: Select all

[code]
systemctl stop nagios
cd /tmp
rm -rf /tmp/nagiosxi
wget https://assets.nagios.com/downloads/nagiosxi/5/xi-5.6.14.tar.gz
tar zxf xi-5.6.14.tar.gz
cd /tmp/nagiosxi/subcomponents/ndoutils
./install
systemctl enable ndo2db

Then edit your /usr/local/nagios/etc/nagios.cfg and make sure this line is uncommented:

Code: Select all

broker_module=/usr/local/nagios/bin/ndomod.o config_file=/usr/local/nagios/etc/ndomod.cfg

Make sure this line is commented:

Code: Select all

#broker_module=/usr/local/nagios/bin/ndo.so /usr/local/nagios/etc/ndo.cfg

Then start the nagios service:

Code: Select all

systemctl start nagios

[/code]

[rajasegar]

Hi,

Thanks for the instructions. Will try it out today and revert back on status.

[benjaminsmith]

Hi,

Thanks for the instructions. Will try it out today and revert back on status.

Your welcome. Just let us know if you have any questions.

[rajasegar]

Hi,

Thanks for the instructions. Will try it out today and revert back on status.

Your welcome. Just let us know if you have any questions.

The instance seems to be fine now. Please close the thread.

[scottwilkerson]

Hi,

Thanks for the instructions. Will try it out today and revert back on status.

Your welcome. Just let us know if you have any questions.

The instance seems to be fine now. Please close the thread.

Great!

Closing thread