Page 1 of 1

Nagios XI SQL Errors after ESX host patching

Posted: Wed Dec 10, 2014 2:36 pm
by krobertson71
***UPDATE** Host Checks are fine. From what you probably can already tell from below, it's just related to service checks only.

My Nagios instance sits on a dedicated ESX host. They brought down the Nagios XI test vm (r2.0) and patched the ESX Host. Well when it came back up I am getting this all over the place.
SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired
I tried to run the repair_database.sh as the nagios user but got this error:
[nagios@nagiast01 scripts]$ ./repair_databases.sh
mysqld (pid 1438) is running...
mysqld (pid 1438) is running...
DATABASE: nagios
TABLE:
./repairmysql.sh: line 47: pushd: /var/lib/mysql/nagios: Permission denied
ERROR: Could not change to dir: /var/lib/mysql/nagios
mysqld (pid 1438) is running...
DATABASE: nagiosql
TABLE:
./repairmysql.sh: line 47: pushd: /var/lib/mysql/nagiosql: Permission denied
ERROR: Could not change to dir: /var/lib/mysql/nagiosql
Stopping ndo2db: head: cannot open `/usr/local/nagios/var/ndo2db.lock' for reading: No such file or directory
done.
Starting ndo2db: done.
Running configuration check...done.
Stopping nagios: done.
Starting nagios: done.
This did not correct anything.

HELP!

Re: Nagios XI SQL Errors after ESX host patching

Posted: Wed Dec 10, 2014 2:55 pm
by sreinhardt
Unfortunately not, but you were doing the correct steps. Only difference is that, that script should be run as root. My guess would be that they used vmware acpi shutdown, which does not play nice with mysql.

Re: Nagios XI SQL Errors after ESX host patching

Posted: Wed Dec 10, 2014 3:44 pm
by krobertson71
sreinhardt wrote:Unfortunately not, but you were doing the correct steps. Only difference is that, that script should be run as root. My guess would be that they used vmware acpi shutdown, which does not play nice with mysql.
Problem fixed!

They gave me sudo permissions to stop mysqld and run all the scripts as root in /usr/local/nagiosxi/scripts.

Question though.

For future situations like this, if I shutdown the mysqld server and nagios will it then be ok for them to stop the VM via the VMWARE acpi and bring it back up again the same way?

Re: Nagios XI SQL Errors after ESX host patching

Posted: Wed Dec 10, 2014 3:47 pm
by tmcdonald
Ideally you should be doing a "shutdown -h now" or "reboot" command, but if you cannot I would do this first:

Code: Select all

service httpd stop
service nagios stop
service npcd stop
service mysqld stop
service postgresql stop
That should shut down everything in the right order.

Re: Nagios XI SQL Errors after ESX host patching

Posted: Wed Dec 10, 2014 7:01 pm
by krobertson71
Thank you!

You can close this thread.