Nagios Crashed - DB Error - Root 100%

[rredmond]

Nagios XI crashed over the weekend due to what I was told was a database error. I was able to reboot the machine and putty in and quickly saw root full. Deleted logs, cleared messages but only got 99%. Noticed /var/log/messages.log was huge! Cleared it but it filled back up almost immediately. Any ideas???????

Randy J Redmond
[email protected]

[rredmond]

Well, the more I dig it looks like DB corruption in MySql....and its fighting my attempts to repair....any help would be much appreciated

[abrist]

XI provides a script for repairing mysql:

Code: Select all

/usr/local/nagiosxi/scripts/repairmysql.sh nagios
/usr/local/nagiosxi/scripts/repairmysql.sh nagiosql

[rredmond]

Well, I do not have that script in that directory or anywhere else that I could find! I tried repairing mysql database nagios which worked until sometime last night at which time it must have corrupted again and once again filled the drive. Please advise...................Randy

[rredmond]

t 1Im looking at some of the mail messages Im getting and Im now seeing:

SNMPGET Problem for ifInOctets.1 ifOutOctets.1 on [email protected]

Received SNP response error code
error status: noSuchName
SNMPv1_Session (remote host: "172.19.1.2" [172.19.1.2].161)

over and over again for different ips

172.19.1.2 is our gateway router and Im told is accepting SNMP requests.......PLEASE advise!

Randy

[rredmond]

PostGres logs also large.....told we have to Vacuum? logs are indeed huge....very confused

[lmiltchev]

First, try to delete as many old logs as you can in "/var/log" directory and sub-directories. Check what you have in the "/tmp". You can probably delete some old backups that you don't need in "/store/backups/". After you clear some room, run the database repair script:

Code: Select all

cd /usr/local/nagiosxi/scripts
./repairmysql.sh nagios
./repairmysql.sh nagiosql

Then add some disk space. If this is a VM, you can follow the steps, outlined in this document:

http://assets.nagios.com/downloads/nagi ... M_Disk.pdf

Let me know if this helped.