Nagios XI SQL Errors after ESX host patching

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
Locked
krobertson71
Posts: 444
Joined: Tue Feb 11, 2014 10:16 pm

Nagios XI SQL Errors after ESX host patching

Post by krobertson71 »

***UPDATE** Host Checks are fine. From what you probably can already tell from below, it's just related to service checks only.

My Nagios instance sits on a dedicated ESX host. They brought down the Nagios XI test vm (r2.0) and patched the ESX Host. Well when it came back up I am getting this all over the place.
SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired
I tried to run the repair_database.sh as the nagios user but got this error:
[nagios@nagiast01 scripts]$ ./repair_databases.sh
mysqld (pid 1438) is running...
mysqld (pid 1438) is running...
DATABASE: nagios
TABLE:
./repairmysql.sh: line 47: pushd: /var/lib/mysql/nagios: Permission denied
ERROR: Could not change to dir: /var/lib/mysql/nagios
mysqld (pid 1438) is running...
DATABASE: nagiosql
TABLE:
./repairmysql.sh: line 47: pushd: /var/lib/mysql/nagiosql: Permission denied
ERROR: Could not change to dir: /var/lib/mysql/nagiosql
Stopping ndo2db: head: cannot open `/usr/local/nagios/var/ndo2db.lock' for reading: No such file or directory
done.
Starting ndo2db: done.
Running configuration check...done.
Stopping nagios: done.
Starting nagios: done.
This did not correct anything.

HELP!
sreinhardt
-fno-stack-protector
Posts: 4366
Joined: Mon Nov 19, 2012 12:10 pm

Re: Nagios XI SQL Errors after ESX host patching

Post by sreinhardt »

Unfortunately not, but you were doing the correct steps. Only difference is that, that script should be run as root. My guess would be that they used vmware acpi shutdown, which does not play nice with mysql.
Nagios-Plugins maintainer exclusively, unless you have other C language bugs with open-source nagios projects, then I am happy to help! Please pm or use other communication to alert me to issues as I no longer track the forum.
krobertson71
Posts: 444
Joined: Tue Feb 11, 2014 10:16 pm

Re: Nagios XI SQL Errors after ESX host patching

Post by krobertson71 »

sreinhardt wrote:Unfortunately not, but you were doing the correct steps. Only difference is that, that script should be run as root. My guess would be that they used vmware acpi shutdown, which does not play nice with mysql.
Problem fixed!

They gave me sudo permissions to stop mysqld and run all the scripts as root in /usr/local/nagiosxi/scripts.

Question though.

For future situations like this, if I shutdown the mysqld server and nagios will it then be ok for them to stop the VM via the VMWARE acpi and bring it back up again the same way?
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: Nagios XI SQL Errors after ESX host patching

Post by tmcdonald »

Ideally you should be doing a "shutdown -h now" or "reboot" command, but if you cannot I would do this first:

Code: Select all

service httpd stop
service nagios stop
service npcd stop
service mysqld stop
service postgresql stop
That should shut down everything in the right order.
Former Nagios employee
krobertson71
Posts: 444
Joined: Tue Feb 11, 2014 10:16 pm

Re: Nagios XI SQL Errors after ESX host patching

Post by krobertson71 »

Thank you!

You can close this thread.
Locked