Page 1 of 1
Nagios Crashed - DB Error - Root 100%
Posted: Mon Jan 27, 2014 8:06 am
by rredmond
Nagios XI crashed over the weekend due to what I was told was a database error. I was able to reboot the machine and putty in and quickly saw root full. Deleted logs, cleared messages but only got 99%. Noticed /var/log/messages.log was huge! Cleared it but it filled back up almost immediately. Any ideas???????
Randy J Redmond
[email protected]
Re: Nagios Crashed - DB Error - Root 100%
Posted: Mon Jan 27, 2014 8:52 am
by rredmond
Well, the more I dig it looks like DB corruption in MySql....and its fighting my attempts to repair....any help would be much appreciated
Re: Nagios Crashed - DB Error - Root 100%
Posted: Mon Jan 27, 2014 11:10 am
by abrist
XI provides a script for repairing mysql:
Code: Select all
/usr/local/nagiosxi/scripts/repairmysql.sh nagios
/usr/local/nagiosxi/scripts/repairmysql.sh nagiosql
Re: Nagios Crashed - DB Error - Root 100%
Posted: Tue Jan 28, 2014 7:52 am
by rredmond
Well, I do not have that script in that directory or anywhere else that I could find! I tried repairing mysql database nagios which worked until sometime last night at which time it must have corrupted again and once again filled the drive. Please advise...................Randy
Re: Nagios Crashed - DB Error - Root 100%
Posted: Tue Jan 28, 2014 8:08 am
by rredmond
t 1Im looking at some of the mail messages Im getting and Im now seeing:
SNMPGET Problem for ifInOctets.1 ifOutOctets.1 on
[email protected]
Received SNP response error code
error status: noSuchName
SNMPv1_Session (remote host: "172.19.1.2" [172.19.1.2].161)
over and over again for different ips
172.19.1.2 is our gateway router and Im told is accepting SNMP requests.......PLEASE advise!
Randy
Re: Nagios Crashed - DB Error - Root 100%
Posted: Tue Jan 28, 2014 8:31 am
by rredmond
PostGres logs also large.....told we have to Vacuum? logs are indeed huge....very confused
Re: Nagios Crashed - DB Error - Root 100%
Posted: Tue Jan 28, 2014 10:50 am
by lmiltchev
First, try to delete as many old logs as you can in "/var/log" directory and sub-directories. Check what you have in the "/tmp". You can probably delete some old backups that you don't need in "/store/backups/". After you clear some room, run the database repair script:
Code: Select all
cd /usr/local/nagiosxi/scripts
./repairmysql.sh nagios
./repairmysql.sh nagiosql
Then add some disk space. If this is a VM, you can follow the steps, outlined in this document:
http://assets.nagios.com/downloads/nagi ... M_Disk.pdf
Let me know if this helped.