Page 1 of 2

Databse Error

Posted: Wed Oct 10, 2018 10:53 am
by mmontero
Hi guys, recently I made changes on my nagios XI and I restored a backup, but when I try to logg on WebPage appears the next error.

Databse Error
A database connection error has been detected, please follow the repair prompt below. If the issue persists, please contact Nagios support.

Run the following from the CLI as root to attempt to repair the DB:

/usr/local/nagiosxi/scripts/repair_databases.sh

I tried with those topics, but the problem persist:

https://support.nagios.com/forum/viewto ... 4&p=106240
https://support.nagios.com/forum/viewto ... 16&t=22287
https://support.nagios.com/kb/print.php?id=25

Re: Databse Error

Posted: Wed Oct 10, 2018 11:53 am
by scottwilkerson
do you get errors when running this from the command line?

Code: Select all

/usr/local/nagiosxi/scripts/repair_databases.sh
Can you also show the output of running the following?

Code: Select all

tail -50 /var/log/mariadb/mariadb.log
or

Code: Select all

tail -50 /var/log/mysqld.log

Re: Databse Error

Posted: Wed Oct 10, 2018 12:03 pm
by mmontero
No, in all cases everything is ok, I reviewed the mysql log but is ok.

The last lines y log are these:

181010 08:24:20 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
181010 08:27:03 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
181010 8:27:03 InnoDB: Initializing buffer pool, size = 8.0M
181010 8:27:03 InnoDB: Completed initialization of buffer pool
181010 8:27:03 InnoDB: Started; log sequence number 0 5427595
181010 8:27:03 [Note] Event Scheduler: Loaded 0 events
181010 8:27:03 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.73' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
181010 8:27:04 [Note] /usr/libexec/mysqld: Normal shutdown

181010 8:27:04 [Note] Event Scheduler: Purging the queue. 0 events
181010 8:27:06 InnoDB: Starting shutdown...
181010 8:27:08 InnoDB: Shutdown completed; log sequence number 0 5427595
181010 8:27:08 [Note] /usr/libexec/mysqld: Shutdown complete

181010 08:27:08 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
181010 08:27:11 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
181010 8:27:11 InnoDB: Initializing buffer pool, size = 8.0M
181010 8:27:11 InnoDB: Completed initialization of buffer pool
181010 8:27:11 InnoDB: Started; log sequence number 0 5427595
181010 8:27:11 [Note] Event Scheduler: Loaded 0 events
181010 8:27:11 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.73' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
You have mail in /var/spool/mail/root

On addition restore backup sendme this message on the proccess:

usr/local/nagiosxi/scripts/restore_xi.sh /store/backups/nagiosxi/autoupgrade_backup.1538587878.tar.gz
PHP Warning: require_once(/usr/local/nagiosxi/html/config.inc.php): failed to open stream: No such file or directory in /usr/local/nagiosxi/scripts/import_xiconfig.php on line 7
PHP Fatal error: require_once(): Failed opening required '/usr/local/nagiosxi/html/config.inc.php' (include_path='.:/usr/share/pear:/usr/share/php') in /usr/local/nagiosxi/scripts/import_xiconfig.php on line 7
TS=1539125997
Extracting backup to /store/backups/nagiosxi/1539125997-restore...
In /store/backups/nagiosxi/1539125997-restore/autoupgrade_backup.1538587878...
Backup files look okay. Preparing to restore...
Shutting down services...
Executable file /usr/local/nagios/bin/nagios not found. Exiting.
Executable file /usr/local/nagios/bin/ndo2db not found. Exiting.
rm: cannot remove `/usr/local/nagiosxi/var/subsys/npcd.pid': No such file or directory
NPCD was not running.
Restoring directories to /...
Restoring Nagios Core...
Restoring Nagios XI...
Restoring NRDP backups...
Restoring MRTG...
Restoring Nagvis backups...
Restoring MySQL databases...
Restoring Nagios XI PostgresQL database...
Starting postgresql service: [ OK ]
ERROR: role "nagiosxi" already exists
ERROR: database "nagiosxi" already exists
Sleeping for a few seconds (please wait)...
SET
SET
SET
SET
SET
SET
SET
ERROR: index "xi_usermeta_autoload_idx" does not exist
ERROR: index "xi_sysstat_metric_idx" does not exist
ERROR: index "xi_options_name_idx" does not exist
ERROR: index "xi_meta_metatype_id_idx" does not exist
ERROR: index "xi_meta_metaobj_id_idx" does not exist
ERROR: index "xi_meta_keyname_idx" does not exist
ERROR: index "xi_commands_event_time_idx" does not exist
ERROR: index "xi_auditlog_user" does not exist
ERROR: index "xi_auditlog_type" does not exist
ERROR: index "xi_auditlog_source" does not exist
ERROR: index "xi_auditlog_log_time" does not exist
ERROR: index "xi_auditlog_ip_address" does not exist
ERROR: index "event_time" does not exist
ERROR: relation "public.xi_users" does not exist
ERROR: relation "public.xi_users" does not exist
ERROR: relation "public.xi_usermeta" does not exist
ERROR: relation "public.xi_usermeta" does not exist
ERROR: relation "public.xi_sysstat" does not exist
ERROR: relation "public.xi_sessions" does not exist
ERROR: relation "public.xi_options" does not exist
ERROR: relation "public.xi_meta" does not exist
ERROR: relation "public.xi_events" does not exist
ERROR: relation "public.xi_commands" does not exist
ERROR: relation "public.xi_cmp_trapdata" does not exist
ERROR: relation "public.xi_cmp_trapdata" does not exist
ERROR: relation "public.xi_cmp_trapdata_log" does not exist
ERROR: relation "public.xi_auth_tokens" does not exist
ERROR: relation "public.xi_auditlog" does not exist
ERROR: relation "public.xi_auditlog" does not exist
ERROR: table "xi_users" does not exist
ERROR: sequence "xi_users_user_id_seq" does not exist
ERROR: table "xi_usermeta" does not exist
ERROR: sequence "xi_usermeta_usermeta_id_seq" does not exist
ERROR: table "xi_sysstat" does not exist
ERROR: sequence "xi_sysstat_sysstat_id_seq" does not exist
ERROR: table "xi_sessions" does not exist
ERROR: sequence "xi_sessions_id_seq" does not exist
ERROR: table "xi_options" does not exist
ERROR: sequence "xi_options_option_id_seq" does not exist
ERROR: table "xi_meta" does not exist
ERROR: sequence "xi_meta_meta_id_seq" does not exist
ERROR: table "xi_incidents" does not exist
ERROR: sequence "xi_incidents_id_seq" does not exist
ERROR: table "xi_events" does not exist
ERROR: sequence "xi_events_event_id_seq" does not exist
ERROR: table "xi_eventqueue" does not exist
ERROR: sequence "xi_eventqueue_eventqueue_id_seq" does not exist
ERROR: table "xi_commands" does not exist
ERROR: sequence "xi_commands_command_id_seq" does not exist
ERROR: table "xi_cmp_trapdata_log" does not exist
ERROR: sequence "xi_cmp_trapdata_trapdata_log_id_seq" does not exist
ERROR: table "xi_cmp_trapdata" does not exist
ERROR: sequence "xi_cmp_trapdata_trapdata_id_seq" does not exist
ERROR: table "xi_auth_tokens" does not exist
ERROR: sequence "xi_auth_tokens_id_seq" does not exist
ERROR: sequence "xi_auditlog_auditlog_id_seq" does not exist
ERROR: table "xi_auditlog" does not exist
ERROR: sequence "if_usermeta_id_seq" does not exist
ERROR: sequence "if_user_id_seq" does not exist
ERROR: sequence "if_sysstat_id_seq" does not exist
ERROR: sequence "if_option_id_seq" does not exist
ERROR: sequence "if_meta_id_seq" does not exist
ERROR: sequence "if_command_id_seq" does not exist
ERROR: must be owner of schema public
ERROR: schema "public" already exists
ALTER SCHEMA
ERROR: must be owner of schema public
SET
CREATE SEQUENCE
ALTER TABLE
setval
--------
20
(1 row)

CREATE SEQUENCE
ALTER TABLE
setval
--------
16
(1 row)

CREATE SEQUENCE
ALTER TABLE
setval
--------
25
(1 row)

CREATE SEQUENCE
ALTER TABLE
setval
--------
15
(1 row)

CREATE SEQUENCE
ALTER TABLE
setval
--------
14
(1 row)

CREATE SEQUENCE
ALTER TABLE
setval
--------
142
(1 row)

SET
SET
CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
ALTER SEQUENCE
setval
--------
83418
(1 row)

CREATE SEQUENCE
ALTER TABLE
setval
--------
170363
(1 row)

CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
setval
--------
1
(1 row)

CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
setval
--------
1
(1 row)

CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
setval
--------
8810
(1 row)

CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
setval
---------
5179863
(1 row)

CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
setval
----------
36457261
(1 row)

CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
setval
--------
1
(1 row)

CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
setval
----------
36457499
(1 row)

CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
setval
--------
255
(1 row)

CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
setval
--------
313
(1 row)

CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
setval
--------
103
(1 row)

CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
setval
--------
2970
(1 row)

CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
setval
--------
118
(1 row)

CREATE TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
WARNING: no privileges could be revoked for "public"
REVOKE
WARNING: no privileges could be revoked for "public"
REVOKE
WARNING: no privileges were granted for "public"
GRANT
WARNING: no privileges were granted for "public"
GRANT
Stopping postgresql service: [ OK ]
Starting postgresql service: [ OK ]
/usr/local/nagiosxi/scripts/restore_xi.sh: line 314: DROP TABLE IF EXISTS nagiosxi;: No such file or directory
Restarting database servers...
Stopping mysqld: [ OK ]
Starting mysqld: [ OK ]
Restoring Apache cronjobs...
Restoring logrotate config files...
Restoring Apache config files...
Stopping httpd: [ OK ]
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using borei01.invexgf.com for ServerName
[ OK ]
DEBUG: Config File = /usr/local/nagios/etc/pnp/npcd.cfg
CONFIG_OPT_LOGTYPE = file
CONFIG_OPT_LOGFILE = /usr/local/nagios/var/npcd.log
CONFIG_OPT_LOGFILESIZE = 10485760
CONFIG_OPT_LOGLEVEL = -1
CONFIG_OPT_SCANDIR = /usr/local/nagios/var/spool/perfdata/
CONFIG_OPT_RUNCMD = /usr/local/nagios/libexec/process_perfdata.pl
CONFIG_OPT_RUNCMD_ARG = -b
CONFIG_OPT_MAXTHREADS = 5
CONFIG_OPT_LOAD = 20.0
CONFIG_OPT_USER = nagios
CONFIG_OPT_GROUP = nagios
CONFIG_OPT_PIDFILE = /usr/local/nagiosxi/var/subsys/npcd.pid
CONFIG_OPT_SLEEPTIME = 15
CONFIG_OPT_IDENTMYSELF = (null)
---------------------------
DEBUG: load_threshold is enabled - ('20.000000')
NPCD started.
Starting ndo2db: done.
Starting nagios: done.

===============
RESTORE COMPLETE
===============

Re: Databse Error

Posted: Wed Oct 10, 2018 12:17 pm
by scottwilkerson
Your restore gave an alarming error

Code: Select all

PHP Warning: require_once(/usr/local/nagiosxi/html/config.inc.php): failed to open stream: No such file or directory in /usr/local/nagiosxi/scripts/import_xiconfig.php on line 7
PHP Fatal error: require_once(): Failed opening required '/usr/local/nagiosxi/html/config.inc.php' (include_path='.:/usr/share/pear:/usr/share/php') in /usr/local/nagiosxi/scripts/import_xiconfig.php on line 7
does this file now exist?

Code: Select all

ls -al /usr/local/nagiosxi/html/config.inc.php
Are you out of disk space?

Code: Select all

df -h

Re: Databse Error

Posted: Wed Oct 10, 2018 12:24 pm
by mmontero
Yes It exist now
ls -al /usr/local/nagiosxi/html/config.inc.php
-rwxr-x--- 1 nagios nagios 8032 Oct 10 12:07 /usr/local/nagiosxi/html/config.inc.php
You have mail in /var/spool/mail/root

df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_borei01-LogVol00
145G 66G 71G 49% /
tmpfs 16G 72K 16G 1% /dev/shm
/dev/sda1 477M 146M 307M 33% /boot
/dev/mapper/vg_borei01-LogVol01
96G 36G 56G 39% /home
/dev/mapper/vg_borei01-LogVol03
96G 33G 59G 37% /opt
/dev/mapper/vg_borei01-LogVol02
392G 44G 329G 12% /var

Re: Databse Error

Posted: Wed Oct 10, 2018 12:29 pm
by scottwilkerson
After the restore are you able to see the UI?

Re: Databse Error

Posted: Wed Oct 10, 2018 12:41 pm
by mmontero
only I can see Welcome, and when press access comes the error.

Re: Databse Error

Posted: Wed Oct 10, 2018 12:45 pm
by scottwilkerson
Can you attach your /usr/local/nagiosxi/html/config.inc.php

Also can you verify postgresql is running

Code: Select all

service postgresql status

Re: Databse Error

Posted: Wed Oct 10, 2018 12:47 pm
by scottwilkerson
Lets also run the following

Code: Select all

echo "vacuum;vacuum analyse;vacuum full;"|psql nagiosxi postgres
echo "vacuum;vacuum analyse;vacuum full;"|psql postgres postgres
echo "vacuum;vacuum analyse;vacuum full;"|psql template1 postgres

Re: Databse Error

Posted: Wed Oct 10, 2018 1:03 pm
by mmontero
service postgresql status
postmaster (pid 3418) is running...