Error during upgrade - backup phase

mcockram · Post by **mcockram** » Mon Aug 23, 2021 3:15 pm

Was upgrading to the latest version and during the backup, received the error

mysqldump: Got error: 1017: Can't find file: nagios_notifications
(errno: 2 - No such file or directory when using LOCK TABLES

dchurch · Post by **dchurch** » Mon Aug 23, 2021 4:02 pm

What is the output from the following command?

Code: Select all

ls -la /var/lib/mysql/nagios

mcockram · Post by **mcockram** » Tue Aug 24, 2021 7:19 am

Code: Select all

# ls -la /var/lib/mysql/nagios
total 2539032
drwxr-x---. 2 mysql mysql       8192 Aug 24 07:26 .
drwxr-xr-x. 9 mysql mysql       4096 Aug 24 05:31 ..
-rw-r-----. 1 mysql mysql      12605 Apr 20 14:15 nagios_acknowled_352.sdi
-rw-r-----  1 mysql mysql     171144 Aug 24 08:00 nagios_acknowledgements.MYD
-rw-r-----  1 mysql mysql      36864 Aug 24 08:00 nagios_acknowledgements.MYI
-rw-r-----. 1 mysql mysql       5702 Apr 20 14:15 nagios_commands_353.sdi
-rw-r-----  1 mysql mysql      14508 Aug 23 23:15 nagios_commands.MYD
-rw-r-----  1 mysql mysql       7168 Aug 24 07:25 nagios_commands.MYI
-rw-r-----. 1 mysql mysql      16596 Apr 20 14:15 nagios_commenthi_354.sdi
-rw-r-----  1 mysql mysql     756300 Aug 24 08:05 nagios_commenthistory.MYD
-rw-r-----  1 mysql mysql      57344 Aug 24 08:05 nagios_commenthistory.MYI
-rw-r-----. 1 mysql mysql      14941 Apr 20 14:15 nagios_comments_355.sdi
-rw-r-----  1 mysql mysql       7376 Aug 24 08:05 nagios_comments.MYD
-rw-r-----  1 mysql mysql       3072 Aug 24 08:05 nagios_comments.MYI
-rw-r-----. 1 mysql mysql       5974 Apr 20 14:15 nagios_configfil_356.sdi
-rw-r-----. 1 mysql mysql       9681 Apr 20 14:15 nagios_configfil_357.sdi
-rw-r-----  1 mysql mysql         44 Aug 23 23:15 nagios_configfiles.MYD
-rw-r-----  1 mysql mysql       9216 Aug 24 07:25 nagios_configfiles.MYI
-rw-r-----  1 mysql mysql       7116 Aug 23 23:15 nagios_configfilevariables.MYD
-rw-r-----  1 mysql mysql       4096 Aug 24 07:25 nagios_configfilevariables.MYI
-rw-r-----. 1 mysql mysql       6682 Apr 20 14:15 nagios_contact_a_364.sdi
-rw-r-----  1 mysql mysql        440 Aug 23 23:15 nagios_contact_addresses.MYD
-rw-r-----  1 mysql mysql       3072 Aug 24 07:25 nagios_contact_addresses.MYI
-rw-r-----. 1 mysql mysql       6764 Apr 20 14:15 nagios_contactgr_358.sdi
-rw-r-----. 1 mysql mysql       4869 Apr 20 14:15 nagios_contactgr_359.sdi
-rw-r-----  1 mysql mysql        540 Aug 23 23:15 nagios_contactgroup_members.MYD
-rw-r-----  1 mysql mysql       3072 Aug 24 07:25 nagios_contactgroup_members.MYI
-rw-r-----  1 mysql mysql        236 Aug 23 23:15 nagios_contactgroups.MYD
-rw-r-----  1 mysql mysql       3072 Aug 24 07:25 nagios_contactgroups.MYI
-rw-r-----. 1 mysql mysql       7686 Apr 20 14:15 nagios_contact_n_365.sdi
-rw-r-----. 1 mysql mysql       8965 Apr 20 14:18 nagios_contactno_544.sdi
-rw-r-----. 1 mysql mysql      10465 Apr 20 14:18 nagios_contactno_545.sdi
-rw-r-----  1 mysql mysql        924 Aug 23 23:15 nagios_contact_notificationcommands.MYD
-rw-r-----  1 mysql mysql       9216 Aug 24 07:25 nagios_contact_notificationcommands.MYI
-rw-r-----  1 mysql mysql   26939860 Aug 24 08:17 nagios_contactnotificationmethods.MYD
-rw-r-----  1 mysql mysql   14265344 Aug 24 08:17 nagios_contactnotificationmethods.MYI
-rw-r-----  1 mysql mysql   11114796 Aug 24 08:17 nagios_contactnotifications.MYD
-rw-r-----  1 mysql mysql   17740800 Aug 24 08:17 nagios_contactnotifications.MYI
-rw-r-----. 1 mysql mysql      23210 Apr 20 14:15 nagios_contacts_362.sdi
-rw-r-----  1 mysql mysql       4000 Aug 24 07:25 nagios_contacts.MYD
-rw-r-----  1 mysql mysql       3072 Aug 24 07:25 nagios_contacts.MYI
-rw-r-----. 1 mysql mysql      10452 Apr 20 14:15 nagios_contactst_363.sdi
-rw-r-----  1 mysql mysql        882 Aug 23 23:15 nagios_contactstatus.MYD
-rw-r-----  1 mysql mysql       3072 Aug 24 07:25 nagios_contactstatus.MYI
-rw-r-----. 1 mysql mysql      12900 Apr 20 14:15 nagios_customvar_366.sdi
-rw-r-----. 1 mysql mysql      12884 Apr 20 14:15 nagios_customvar_367.sdi
-rw-r-----  1 mysql mysql      59984 Aug 23 23:15 nagios_customvariables.MYD
-rw-r-----  1 mysql mysql      72704 Aug 24 07:25 nagios_customvariables.MYI
-rw-r-----  1 mysql mysql      63996 Aug 23 23:15 nagios_customvariablestatus.MYD
-rw-r-----  1 mysql mysql      64512 Aug 24 07:25 nagios_customvariablestatus.MYI
-rw-r-----. 1 mysql mysql       2521 Apr 20 14:15 nagios_dbversion_368.sdi
-rw-r-----  1 mysql mysql         20 Jul 15 21:45 nagios_dbversion.MYD
-rw-r-----  1 mysql mysql       1024 Jul 15 21:50 nagios_dbversion.MYI
-rw-r-----. 1 mysql mysql      18309 Apr 20 14:15 nagios_downtimeh_369.sdi
-rw-r-----  1 mysql mysql          0 Jul 15 21:45 nagios_downtimehistory.MYD
-rw-r-----  1 mysql mysql       1024 Aug 24 07:25 nagios_downtimehistory.MYI
-rw-r-----. 1 mysql mysql      17950 Apr 20 14:15 nagios_eventhand_370.sdi
-rw-r-----  1 mysql mysql      34480 Aug 24 08:17 nagios_eventhandlers.MYD
-rw-r-----  1 mysql mysql       5120 Aug 24 08:17 nagios_eventhandlers.MYI
-rw-r-----. 1 mysql mysql       7614 Apr 20 14:15 nagios_externalc_371.sdi
-rw-r-----  1 mysql mysql        320 Aug 24 08:00 nagios_externalcommands.MYD
-rw-r-----  1 mysql mysql       2048 Aug 24 08:00 nagios_externalcommands.MYI
-rw-r-----. 1 mysql mysql      11618 Apr 20 14:15 nagios_flappingh_372.sdi
-rw-r-----  1 mysql mysql      87497 Aug 24 08:05 nagios_flappinghistory.MYD
-rw-r-----  1 mysql mysql      17408 Aug 24 08:05 nagios_flappinghistory.MYI
-rw-r-----. 1 mysql mysql      21885 Apr 20 14:15 nagios_hostcheck_373.sdi
-rw-r-----  1 mysql mysql     237128 Aug 24 08:17 nagios_hostchecks.MYD
-rw-r-----  1 mysql mysql      31744 Aug 24 08:17 nagios_hostchecks.MYI
-rw-r-----. 1 mysql mysql       4861 Apr 20 14:15 nagios_host_cont_382.sdi
-rw-r-----. 1 mysql mysql       4901 Apr 20 14:15 nagios_host_cont_383.sdi
-rw-r-----  1 mysql mysql       5625 Aug 23 23:15 nagios_host_contactgroups.MYD
-rw-r-----  1 mysql mysql      14336 Aug 24 07:25 nagios_host_contactgroups.MYI
-rw-r-----  1 mysql mysql         30 Aug 23 23:15 nagios_host_contacts.MYD
-rw-r-----  1 mysql mysql       3072 Aug 24 07:25 nagios_host_contacts.MYI
-rw-r-----. 1 mysql mysql      10922 Apr 20 14:15 nagios_hostdepen_374.sdi
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_hostdependencies.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 24 07:25 nagios_hostdependencies.MYI
-rw-r-----. 1 mysql mysql      10724 Apr 20 14:15 nagios_hostescal_375.sdi
-rw-r-----. 1 mysql mysql       4902 Apr 20 14:15 nagios_hostescal_376.sdi
-rw-r-----. 1 mysql mysql       4931 Apr 20 14:15 nagios_hostescal_377.sdi
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_hostescalation_contactgroups.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 24 07:25 nagios_hostescalation_contactgroups.MYI
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_hostescalation_contacts.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 24 07:25 nagios_hostescalation_contacts.MYI
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_hostescalations.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 24 07:25 nagios_hostescalations.MYI
-rw-r-----. 1 mysql mysql       6678 Apr 20 14:15 nagios_hostgroup_378.sdi
-rw-r-----. 1 mysql mysql       4854 Apr 20 14:15 nagios_hostgroup_379.sdi
-rw-r-----  1 mysql mysql       5325 Aug 23 23:15 nagios_hostgroup_members.MYD
-rw-r-----  1 mysql mysql      13312 Aug 24 07:25 nagios_hostgroup_members.MYI
-rw-r-----  1 mysql mysql        504 Aug 23 23:15 nagios_hostgroups.MYD
-rw-r-----  1 mysql mysql       3072 Aug 24 07:25 nagios_hostgroups.MYI
-rw-r-----. 1 mysql mysql       4849 Apr 20 14:15 nagios_host_pare_384.sdi
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_host_parenthosts.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 24 07:25 nagios_host_parenthosts.MYI
-rw-r-----. 1 mysql mysql      63678 Apr 20 14:15 nagios_hosts_380.sdi
-rw-r-----  1 mysql mysql     151000 Aug 24 07:25 nagios_hosts.MYD
-rw-r-----  1 mysql mysql      18432 Aug 24 07:25 nagios_hosts.MYI
-rw-r-----. 1 mysql mysql      48408 Apr 20 14:15 nagios_hoststatu_381.sdi
-rw-r-----  1 mysql mysql     232960 Aug 24 08:17 nagios_hoststatus.MYD
-rw-r-----  1 mysql mysql      98304 Aug 24 08:17 nagios_hoststatus.MYI
-rw-r-----. 1 mysql mysql       4441 Apr 20 14:15 nagios_instances_385.sdi
-rw-r-----  1 mysql mysql         20 Jul 15 21:45 nagios_instances.MYD
-rw-r-----  1 mysql mysql       2048 Aug 24 07:25 nagios_instances.MYI
-rw-r-----. 1 mysql mysql       9820 Apr 20 14:18 nagios_logentrie_552.sdi
-rw-r-----  1 mysql mysql 1551289188 Aug 24 08:17 nagios_logentries.MYD
-rw-r-----  1 mysql mysql  339096576 Aug 24 08:17 nagios_logentries.MYI
-rw-r-----. 1 mysql mysql      13676 Apr 20 14:18 nagios_notificat_553.sdi
-rw-r-----  1 mysql mysql    9477280 Jul 15 21:31 nagios_notifications.MYD
-rw-r-----  1 mysql mysql     744448 Jul 15 21:33 nagios_notifications.TMM
-rw-r-----. 1 mysql mysql      16351 Apr 20 14:18 nagios_objects_549.sdi
-rw-r-----  1 mysql mysql     745820 Aug 24 07:26 nagios_objects.MYD
-rw-r-----  1 mysql mysql     443392 Aug 24 07:26 nagios_objects.MYI
-rw-r-----. 1 mysql mysql       8572 Apr 20 14:15 nagios_processev_389.sdi
-rw-r-----  1 mysql mysql     254200 Aug 24 07:26 nagios_processevents.MYD
-rw-r-----  1 mysql mysql      27648 Aug 24 07:26 nagios_processevents.MYI
-rw-r-----. 1 mysql mysql      23764 Apr 20 14:15 nagios_programst_390.sdi
-rw-r-----  1 mysql mysql        108 Aug 24 08:17 nagios_programstatus.MYD
-rw-r-----  1 mysql mysql       3072 Aug 24 08:17 nagios_programstatus.MYI
-rw-r-----. 1 mysql mysql       9340 Apr 20 14:15 nagios_runtimeva_391.sdi
-rw-r-----  1 mysql mysql        844 Aug 23 23:15 nagios_runtimevariables.MYD
-rw-r-----  1 mysql mysql       3072 Aug 24 07:26 nagios_runtimevariables.MYI
-rw-r-----. 1 mysql mysql      15912 Apr 20 14:15 nagios_scheduled_392.sdi
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_scheduleddowntime.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 24 07:26 nagios_scheduleddowntime.MYI
-rw-r-----. 1 mysql mysql       4873 Apr 20 14:15 nagios_service_c_402.sdi
-rw-r-----. 1 mysql mysql       4910 Apr 20 14:15 nagios_service_c_403.sdi
-rw-r-----. 1 mysql mysql      21621 Apr 20 14:15 nagios_servicech_393.sdi
-rw-r-----  1 mysql mysql    3019960 Aug 21 07:03 nagios_servicechecks.MYD
-rw-r-----  1 mysql mysql     408576 Aug 21 07:04 nagios_servicechecks.MYI
-rw-r-----  1 mysql mysql      67710 Aug 23 23:15 nagios_service_contactgroups.MYD
-rw-r-----  1 mysql mysql     130048 Aug 24 07:26 nagios_service_contactgroups.MYI
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_service_contacts.MYD
-rw-r-----  1 mysql mysql       1024 Aug 24 07:26 nagios_service_contacts.MYI
-rw-r-----. 1 mysql mysql      11804 Apr 20 14:15 nagios_servicede_394.sdi
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_servicedependencies.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 24 07:26 nagios_servicedependencies.MYI
-rw-r-----. 1 mysql mysql      11531 Apr 20 14:15 nagios_servicees_395.sdi
-rw-r-----. 1 mysql mysql       4914 Apr 20 14:15 nagios_servicees_396.sdi
-rw-r-----. 1 mysql mysql       4940 Apr 20 14:15 nagios_servicees_397.sdi
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_serviceescalation_contactgroups.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 24 07:26 nagios_serviceescalation_contactgroups.MYI
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_serviceescalation_contacts.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 24 07:26 nagios_serviceescalation_contacts.MYI
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_serviceescalations.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 24 07:26 nagios_serviceescalations.MYI
-rw-r-----. 1 mysql mysql       6764 Apr 20 14:15 nagios_servicegr_398.sdi
-rw-r-----. 1 mysql mysql       4869 Apr 20 14:15 nagios_servicegr_399.sdi
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_servicegroup_members.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 24 07:26 nagios_servicegroup_members.MYI
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_servicegroups.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 24 07:26 nagios_servicegroups.MYI
-rw-r-----. 1 mysql mysql       4870 Apr 20 14:15 nagios_service_p_404.sdi
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_service_parentservices.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 23 23:15 nagios_service_parentservices.MYI
-rw-r-----. 1 mysql mysql      55336 Apr 20 14:15 nagios_services_400.sdi
-rw-r-----  1 mysql mysql    2093120 Aug 24 07:26 nagios_services.MYD
-rw-r-----  1 mysql mysql     161792 Aug 24 07:26 nagios_services.MYI
-rw-r-----. 1 mysql mysql      49241 Apr 20 14:15 nagios_servicest_401.sdi
-rw-r-----  1 mysql mysql    3806556 Aug 24 08:17 nagios_servicestatus.MYD
-rw-r-----  1 mysql mysql     878592 Aug 24 08:17 nagios_servicestatus.MYI
-rw-r-----. 1 mysql mysql      13059 Apr 20 14:18 nagios_statehist_539.sdi
-rw-r-----  1 mysql mysql   28190756 Aug 24 08:17 nagios_statehistory.MYD
-rw-r-----  1 mysql mysql    2834432 Aug 24 08:17 nagios_statehistory.MYI
-rw-r-----. 1 mysql mysql      12317 Apr 20 14:15 nagios_systemcom_406.sdi
-rw-r-----  1 mysql mysql      37692 Aug 24 08:17 nagios_systemcommands.MYD
-rw-r-----  1 mysql mysql      12288 Aug 24 08:17 nagios_systemcommands.MYI
-rw-r-----. 1 mysql mysql       9433 Apr 20 14:18 nagios_timedeven_550.sdi
-rw-r-----. 1 mysql mysql      12646 Apr 20 14:18 nagios_timedeven_551.sdi
-rw-r-----  1 mysql mysql          0 Aug 23 23:15 nagios_timedeventqueue.MYD
-rw-r-----. 1 mysql mysql       1024 Aug 24 07:26 nagios_timedeventqueue.MYI
-rw-r-----  1 mysql mysql          0 Jul 15 21:45 nagios_timedevents.MYD
-rw-r-----  1 mysql mysql       1024 Aug 24 07:26 nagios_timedevents.MYI
-rw-r-----. 1 mysql mysql       6756 Apr 20 14:15 nagios_timeperio_409.sdi
-rw-r-----. 1 mysql mysql       6572 Apr 20 14:15 nagios_timeperio_410.sdi
-rw-r-----  1 mysql mysql       1108 Aug 23 23:15 nagios_timeperiods.MYD
-rw-r-----  1 mysql mysql       3072 Aug 24 07:26 nagios_timeperiods.MYI
-rw-r-----  1 mysql mysql       3192 Aug 23 23:15 nagios_timeperiod_timeranges.MYD
-rw-r-----  1 mysql mysql       8192 Aug 24 07:26 nagios_timeperiod_timeranges.MYI

dchurch · Post by **dchurch** » Tue Aug 24, 2021 11:44 am

Please try running the database repair script, and let me know if that is successful. Run the following as root from the terminal.

Code: Select all

/usr/local/nagiosxi/scripts/repair_databases.sh

See Repairing The Nagios XI Database for complete instructions

mcockram · Post by **mcockram** » Tue Aug 24, 2021 2:57 pm

Same error

Code: Select all

Backing up MySQL databases...
mysqldump: [Warning] Using a password on the command line interface can be insecure.
mysqldump: Got error: 1017: Can't find file: 'nagios_notifications' (errno: 2 - No such file or directory) when using LOCK TABLES
Error backing up MySQL database 'nagios' - check the password in this script!

mcockram · Post by **mcockram** » Tue Aug 24, 2021 3:21 pm

I tried running the below as well, but got the same error.

Code: Select all

systemctl stop crond
systemctl stop npcd
systemctl stop nagios
systemctl stop ndo2db
pkill -9 -u nagios
for i in $(ipcs -q | grep nagios |awk '{print $2}'); do ipcrm -q $i; done
rm -rf /usr/local/nagiosxi/var/dbmaint.lock
rm -rf /usr/local/nagiosxi/var/event_handler.lock
rm -rf /usr/local/nagiosxi/scripts/reconfigure_nagios.lock
systemctl restart mysqld || systemctl restart mariadb
cd /usr/local/nagiosxi/scripts
./repair_databases.sh
systemctl start npcd
systemctl start crond
systemctl start nagios
systemctl start ndo2db

dchurch · Post by **dchurch** » Tue Aug 24, 2021 4:04 pm

Please try running the following commands:

Code: Select all

cp -a /var/lib/mysql/nagios/nagios_notifications.* /tmp
mv /var/lib/mysql/nagios/nagios_notifications.TMM /var/lib/mysql/nagios/nagios_notifications.MYI
/usr/local/nagiosxi/scripts/repair_databases.sh

mcockram · Post by **mcockram** » Wed Aug 25, 2021 9:24 am

That seemed to fix it. One question, we went from 5.8.3 to 5.8.5 on RHEL 8. The upgrade finished successfully, but we lock down wget and curl commands to only root running them. Does this upgrade need to run the wget or curl commands from a non-root user? It looks like it downloaded nagios-dep from a dnf command.

ssax · Post by **ssax** » Wed Aug 25, 2021 5:22 pm

I don't think it runs any commands as a non-root user during the upgrade process, I didn't find anything searching the installer code for su. Since the installer is meant to be run with root/sudo it shouldn't run any of the subscripts as a regular user.

Nagios Support Forum

Error during upgrade - backup phase

Error during upgrade - backup phase

Re: Error during upgrade - backup phase

Re: Error during upgrade - backup phase

Re: Error during upgrade - backup phase

Re: Error during upgrade - backup phase

Re: Error during upgrade - backup phase

Re: Error during upgrade - backup phase

Re: Error during upgrade - backup phase

Re: Error during upgrade - backup phase