Run the following from the CLI as root to attempt to repair

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
Locked
cwireless
Posts: 66
Joined: Sat Feb 05, 2011 11:37 pm

Run the following from the CLI as root to attempt to repair

Post by cwireless »

Hi,
We have this configuration:
Red Hat Enterprise Linux Server release 5.3; 32 bit; VMWare Image of NagiosXI; No special configurations.Using Internet Explorer.

We recently upgraded to version R2.4 and we received this error below.

Code: Select all

Message: A database connection error has been detected, we are attempting to repair the server, if the repair does not resolve the issue, please contact Nagios support.

Run the following from the CLI as root to attempt to repair the DB

/usr/local/nagiosxi/scripts/repair_databases.sh
Message: A database connection error has been detected, we are attempting to repair the server, if the repair does not resolve the issue, please contact Nagios support.

Run the following from the CLI as root to attempt to repair the DB

/usr/local/nagiosxi/scripts/repair_databases.sh
SQL: SQL Error [nagiosxi] : Database connection failed SQL: SQL Error [nagiosxi] : Database connection failed SQL: SQL Error [nagiosxi] : Database connection failedMessage: A database connection error has been detected, we are attempting to repair the server, if the repair does not resolve the issue, please contact Nagios support.

Run the following from the CLI as root to attempt to repair the DB

/usr/local/nagiosxi/scripts/repair_databases.sh
we tried to rollback to version R2.3 and then we got this message

Code: Select all

-bash-3.2$ psql nagiosxi nagiosxi
psql: FATAL:  database is not accepting commands to avoid wraparound data loss in database "nagiosxi"
HINT:  Stop the postmaster and use a standalone backend to vacuum database "nagiosxi".
-bash-3.2$

First we receive a message about 1 million transactions, when it finished, then the BD started to display 4,294,896,293 transactions. Now the error still remains
User avatar
lgroschen
Posts: 384
Joined: Wed Nov 27, 2013 1:17 pm

Re: Run the following from the CLI as root to attempt to rep

Post by lgroschen »

Did you attempt to run the repair script before rolling back?
/Luke
cwireless
Posts: 66
Joined: Sat Feb 05, 2011 11:37 pm

Re: Run the following from the CLI as root to attempt to rep

Post by cwireless »

Yes, in that moment did not work either, so we decided to rollback.
slansing
Posts: 7698
Joined: Mon Apr 23, 2012 4:28 pm
Location: Travelling through time and space...

Re: Run the following from the CLI as root to attempt to rep

Post by slansing »

Okay, so what exactly did you do to roll it back? That may have caused additional issues, if you can let us know the steps you took, that will help us get to the resolution faster. Thanks!
cwireless
Posts: 66
Joined: Sat Feb 05, 2011 11:37 pm

Re: Run the following from the CLI as root to attempt to rep

Post by cwireless »

First we executed the repair script.

For the rollback, we followed the bacuk and restore procedure(PDF) and we try to restore a backup from January 21st. And the error was displayed

/usr/local/nagiosxi/scripts/restore_xi.sh /store/backups/nagiosxi/1421895603.tar.gz

Code: Select all

-rw-r--r-- 1 nagios nagios 694744895 Jan 20 22:03 1421809202.tar.gz
-rw-r--r-- 1 nagios nagios 691407262 Jan 21 22:03 1421895603.tar.gz
cwireless
Posts: 66
Joined: Sat Feb 05, 2011 11:37 pm

Re: Run the following from the CLI as root to attempt to rep

Post by cwireless »

So far this is what we are doing.

Code: Select all

WARNING:  database "nagiosxi" must be vacuumed within 4289025932 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025931 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025930 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025929 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025928 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025927 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  some databases have not been vacuumed in over 1 billion transactions
HINT:  Better vacuum them soon, or you may have a wraparound failure.
WARNING:  database "nagiosxi" must be vacuumed within 4289025926 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
backend> WARNING:  database "nagiosxi" must be vacuumed within 4289025926 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025925 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025924 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025923 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025922 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025921 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025920 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025919 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025918 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025917 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025916 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025915 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025914 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025913 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025912 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025911 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025910 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025909 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025908 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025907 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025906 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025905 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025904 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025903 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025902 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "nagiosxi".
WARNING:  database "nagiosxi" must be vacuumed within 4289025901 transactions
HINT:  To avoid a database shutdown, execute a full-database VACUUM in "
User avatar
Box293
Too Basu
Posts: 5126
Joined: Sun Feb 07, 2010 10:55 pm
Location: Deniliquin, Australia
Contact:

Re: Run the following from the CLI as root to attempt to rep

Post by Box293 »

Try these commands:

Code: Select all

service postgresql stop
su postgres
echo "VACUUM FULL;" > /tmp/fix.sql
postgres -D /var/lib/pgsql/data nagiosxi < /tmp/fix.sql
postgres -D /var/lib/pgsql/data postgres < /tmp/fix.sql
postgres -D /var/lib/pgsql/data template1 < /tmp/fix.sql
exit
service postgresql start
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
cwireless
Posts: 66
Joined: Sat Feb 05, 2011 11:37 pm

Re: Run the following from the CLI as root to attempt to rep

Post by cwireless »

Box293 wrote:Try these commands:

Code: Select all

service postgresql stop
su postgres
echo "VACUUM FULL;" > /tmp/fix.sql
postgres -D /var/lib/pgsql/data nagiosxi < /tmp/fix.sql
postgres -D /var/lib/pgsql/data postgres < /tmp/fix.sql
postgres -D /var/lib/pgsql/data template1 < /tmp/fix.sql
exit
service postgresql start

Greattttt!!!!!!! It worked like a charm.........

Thanks guys for your support...

Saludos.
User avatar
tgriep
Madmin
Posts: 9190
Joined: Thu Oct 30, 2014 9:02 am

Re: Run the following from the CLI as root to attempt to rep

Post by tgriep »

Glad it is fixed. We'll close the post for you.
Be sure to check out our Knowledgebase for helpful articles and solutions!
Locked