Need to restore from backup

[tomdchi]

Something has gone wrong with our Nagios server and it is not able to connect to MySQL and the web UI (ournagiosserver.com/nagiosxi/) just shows a blank page. This happened sometime just after it was upgraded on the 19th.

Messages log:

Code: Select all

Mar 25 11:25:20 localhost nagios: ndomod: Successfully reconnected to data sink!  0 items lost, 31 queued items to flush.
Mar 25 11:25:20 localhost nagios: ndomod: Successfully flushed 31 queued items to data sink.
Mar 25 11:25:20 localhost ndo2db: Error: Could not connect to MySQL database: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (111)
Mar 25 11:25:20 localhost ndo2db: Error: Could not connect to MySQL database: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (111)
Mar 25 11:25:20 localhost nagios: ndomod: Error writing to data sink!  Some output may get lost...
Mar 25 11:25:20 localhost nagios: ndomod: Please check remote ndo2db log, database connection or SSL Parameters

Linux version is 2.6.32-431.1.2.0.1.el6.x86_64. Not sure exact version of Nagios but it was just updated recently to the latest version.

Backups were made prior to upgrade using the following commands:

Code: Select all

tar cjf /root/pre-upgrade-backup/usr-local-nagios.tar.bz2 /usr/local/nagios
tar cjf /root/pre-upgrade-backup/usr-local-nagiosxi.tar.bz2 /usr/local/nagiosxi
pg_dump -U nagiosxi -c -F p nagiosxi | bzip2 -c > /root/pre-upgrade-backup/pg_nagiosxi.sql.bz2
mysqldump -u root -pnagiosxi nagios | bzip2 -c > /root/pre-upgrade-backup/my_nagios.sql.bz2
mysqldump -u root -pnagiosxi nagiosql | bzip2 -c > /root/pre-upgrade-backup/my_nagiosql.sql.bz2

How would I:
1. Figure out if MySQL is completely hosed and needs to be reinstalled?
2. If MySQL OK, how do I restore the backups?

Would it be easier to do a fresh install then restore the backups?

[lmiltchev]

Is mysqld running at all?

Code: Select all

service mysqld status

You can try:

Code: Select all

service mysqld stop
rm -f /var/lib/mysql/mysql.sock
service mysqld start

Is there any reason why you are not using the steps, outlined in our official documentation for backing up and restoring your Nagios XI server?

http://assets.nagios.com/downloads/nagi ... ing_XI.pdf

[tomdchi]

MySQL was not running and did start. There is still something wrong though.

I am seeing many errors in the web UI:

SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired SQL: SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and should be repaired

The backup wasn't a full server backup but a database backup before upgrade. Instructions for this were contained in the Nagios provided document XI_Upgrade_Instructions.pdf.

[tomdchi]

found document for repairing database and things appear to be working normally again.
Thanks for the help.

[lmiltchev]

I am glad I could help!