Databse Error!!!

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
Locked
ponnpr7
Posts: 90
Joined: Fri Jul 28, 2017 3:55 pm

Databse Error!!!

Post by ponnpr7 »

Databse Error
A database connection error has been detected, please follow the repair prompt below. If the issue persists, please contact Nagios support.
Run the following from the CLI as root to attempt to repair the DB:

/usr/local/nagiosxi/scripts/repair_databases.sh



Code: Select all

[root@nagiosxi scripts]# ./repair_databases.sh
touch: cannot touch `/usr/local/nagiosxi/scripts/repair_databases.lock': Read-only file system
DATABASE: nagios
TABLE:
/usr/local/nagiosxi/scripts/repairmysql.sh: line 39: tmp_xi_vars.cfg: Read-only file system
mysqld dead but pid file exists
rm: cannot remove `/var/lib/mysql/mysql.sock': Read-only file system
/var/lib/mysql/nagios /usr/local/nagiosxi/scripts
Stopping mysqld:                                           [FAILED]
myisamchk: error: 30 when opening MyISAM-table 'nagios_acknowledgements.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_commands.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_commenthistory.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_comments.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_configfiles.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_configfilevariables.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_conninfo.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_contact_addresses.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_contactgroup_members.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_contactgroups.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_contact_notificationcommands.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_contactnotificationmethods.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_contactnotifications.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_contacts.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_contactstatus.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_customvariables.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_customvariablestatus.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_dbversion.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_downtimehistory.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_eventhandlers.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_externalcommands.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_flappinghistory.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_hostchecks.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_host_contactgroups.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_host_contacts.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_hostdependencies.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_hostescalation_contactgroups.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_hostescalation_contacts.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_hostescalations.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_hostgroup_members.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_hostgroups.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_host_parenthosts.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_hosts.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_hoststatus.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_instances.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_logentries.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_notifications.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_objects.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_processevents.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_programstatus.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_runtimevariables.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_scheduleddowntime.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_servicechecks.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_service_contactgroups.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_service_contacts.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_servicedependencies.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_serviceescalation_contactgroups.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_serviceescalation_contacts.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_serviceescalations.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_servicegroup_members.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_servicegroups.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_service_parentservices.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_services.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_servicestatus.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_statehistory.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_systemcommands.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_timedeventqueue.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_timedevents.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_timeperiods.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'nagios_timeperiod_timeranges.MYI'
Starting mysqld:                                           [FAILED]
/usr/local/nagiosxi/scripts

===============
REPAIR COMPLETE
===============
DATABASE: nagiosql
TABLE:
/usr/local/nagiosxi/scripts/repairmysql.sh: line 39: tmp_xi_vars.cfg: Read-only file system
mysqld dead but pid file exists
rm: cannot remove `/var/lib/mysql/mysql.sock': Read-only file system
/var/lib/mysql/nagiosql /usr/local/nagiosxi/scripts
Stopping mysqld:                                           [FAILED]
myisamchk: error: 30 when opening MyISAM-table 'tbl_command.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_contactgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_contact.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_contacttemplate.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_domain.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_hostdependency.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_hostescalation.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_hostextinfo.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_hostgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_host.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_hosttemplate.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_info.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkContactgroupToContactgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkContactgroupToContact.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkContacttemplateToCommandHost.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkContacttemplateToCommandService.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkContacttemplateToContactgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkContacttemplateToContacttemplate.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkContacttemplateToVariabledefinition.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkContactToCommandHost.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkContactToCommandService.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkContactToContactgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkContactToContacttemplate.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkContactToVariabledefinition.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostdependencyToHost_DH.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostdependencyToHostgroup_DH.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostdependencyToHostgroup_H.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostdependencyToHost_H.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostescalationToContactgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostescalationToContact.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostescalationToHostgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostescalationToHost.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostgroupToHostgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostgroupToHost.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHosttemplateToContactgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHosttemplateToContact.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHosttemplateToHostgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHosttemplateToHost.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHosttemplateToHosttemplate.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHosttemplateToVariabledefinition.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostToContactgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostToContact.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostToHostgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostToHost.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostToHosttemplate.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkHostToVariabledefinition.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicedependencyToHost_DH.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicedependencyToHostgroup_DH.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicedependencyToHostgroup_H.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicedependencyToHost_H.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicedependencyToService_DS.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicedependencyToService_S.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServiceescalationToContactgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServiceescalationToContact.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServiceescalationToHostgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServiceescalationToHost.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServiceescalationToService.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicegroupToServicegroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicegroupToService.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicetemplateToContactgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicetemplateToContact.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicetemplateToHostgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicetemplateToHost.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicetemplateToServicegroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicetemplateToServicetemplate.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServicetemplateToVariabledefinition.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServiceToContactgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServiceToContact.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServiceToHostgroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServiceToHost.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServiceToServicegroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServiceToServicetemplate.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkServiceToVariabledefinition.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_lnkTimeperiodToTimeperiod.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_logbook.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_mainmenu.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_servicedependency.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_serviceescalation.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_serviceextinfo.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_servicegroup.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_service.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_servicetemplate.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_session_locks.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_session.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_settings.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_submenu.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_timedefinition.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_timeperiod.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_user.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'tbl_variabledefinition.MYI'
Starting mysqld:                                           [FAILED]
/usr/local/nagiosxi/scripts

===============
REPAIR COMPLETE
===============
DATABASE: nagiosxi
TABLE:
/usr/local/nagiosxi/scripts/repairmysql.sh: line 39: tmp_xi_vars.cfg: Read-only file system
mysqld dead but pid file exists
rm: cannot remove `/var/lib/mysql/mysql.sock': Read-only file system
/var/lib/mysql/nagiosxi /usr/local/nagiosxi/scripts
Stopping mysqld:                                           [FAILED]
myisamchk: error: 30 when opening MyISAM-table 'xi_auditlog.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'xi_commands.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'xi_eventqueue.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'xi_events.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'xi_incidents.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'xi_meta.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'xi_options.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'xi_sysstat.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'xi_usermeta.MYI'

---------

myisamchk: error: 30 when opening MyISAM-table 'xi_users.MYI'
Starting mysqld:                                           [FAILED]
/usr/local/nagiosxi/scripts

===============
REPAIR COMPLETE
===============
ndo2db was not running... could not stop
Starting ndo2db:rm: cannot remove `/usr/local/nagiosxi/var/subsys/ndo2db': Read-only file system
rm: cannot remove `/usr/local/nagios/var/ndo.sock': Read-only file system
touch: cannot touch `/usr/local/nagios/var/ndo2db.lock': Read-only file system
chown: changing ownership of `/usr/local/nagios/var/ndo2db.lock': Read-only file system
Could not bind socket: Address already in use
touch: cannot touch `/usr/local/nagiosxi/var/subsys/ndo2db': Read-only file system
chown: changing ownership of `/usr/local/nagiosxi/var/subsys/ndo2db': Read-only file system
 done.
Running configuration check...
mktemp: failed to create file via template `/tmp/.configtest.XXXXXXXX': Read-only file system
/etc/init.d/nagios: line 90: : No such file or directory
grep: : No such file or directory
grep: : No such file or directory
/etc/init.d/nagios: line 110: /usr/local/nagios/var/nagios.configtest: Read-only file system
/etc/init.d/nagios: line 111: /usr/local/nagios/var/nagios.configtest: Read-only file system
chmod: changing permissions of `/usr/local/nagios/var/nagios.configtest': Read-only file system
chown: changing ownership of `/usr/local/nagios/var/nagios.configtest': Read-only file system
cat: : No such file or directory

=======================
nagios database repair FAILED, please check output above!
nagiosql database repair FAILED, please check output above!
nagiosxi database repair FAILED, please check output above!
Thanks,
Ravi Ponnaiah
Top
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: Databse Error!!!

Post by mcapra »

Uhh, did someone change your filesystem to be read-only?

MyISAM is going to have a heck of a time maintaining indices if it can't write to disk which is going to cause all sorts of upstream problems within Nagios XI and elsewhere on the system. I'd consult with the Linux administrator of this system to fix that problem first and foremost. It's quite possible the filesystem was damaged and the OS made it read-only.
Former Nagios employee
https://www.mcapra.com/
Top
ponnpr7
Posts: 90
Joined: Fri Jul 28, 2017 3:55 pm

Re: Databse Error!!!

Post by ponnpr7 »

How if I boot the VM from backup.
Top
User avatar
tgriep
Madmin
Posts: 9190
Joined: Thu Oct 30, 2014 9:02 am

Re: Databse Error!!!

Post by tgriep »

The servers hard drive seems to be in read-only mode so the system is generating the errors as it cannot update the data it needs to write to the drive.

If could be caused by the drive being full and to check that, run the following as root to see if there is sufficient space.

Code: Select all

df -h
If there is some free space, the simplest way to remount the drives as read-write is to reboot the server.
On reboot, the system may fix the filesystem errors that caused it to go in to read-only mode.

If it boots up, run the database repair script and see if it finishes and that the system starts to function normally.
Be sure to check out our Knowledgebase for helpful articles and solutions!
Top
ponnpr7
Posts: 90
Joined: Fri Jul 28, 2017 3:55 pm

Re: Databse Error!!!

Post by ponnpr7 »

The servers hard drive seems to be in read-only mode

How/where do I see that?

Ravi
Top
User avatar
tgriep
Madmin
Posts: 9190
Joined: Thu Oct 30, 2014 9:02 am

Re: Databse Error!!!

Post by tgriep »

In your first post, the output of the repair has error messages like this.
touch: cannot touch `/usr/local/nagiosxi/scripts/repair_databases.lock': Read-only file system
That is where I seen those messages.
Be sure to check out our Knowledgebase for helpful articles and solutions!
Top
Locked

Return to “Nagios XI”