Server not running, Disc Space full of old Indexes

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
Mossx
Posts: 3
Joined: Mon Dec 29, 2014 11:23 am

Server not running, Disc Space full of old Indexes

Post by Mossx »

Hello,

I install Nagios Log Server manually on CentOS.

Everything was going great, I left for the weekend only to find it not working.
It looks like the disk is full, and I was having trouble with it automatically deleting indexes (despite it being set to delete indexes older the 2 days in the maintenance and backup)

My first question: is there anyway to delete indexes manually from the command line so I can get the server back up?

Second question: what could be causing the issue where the server does not delete old indexes?

Thanks for any help you can provide.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Server not running, Disc Space full of old Indexes

Post by scottwilkerson »

You should be able to delete them through the UI, however you may need to free a little space to be able to get it the UI up and running.

Here's what I would suggest to free up some space

Code: Select all

service logstash stop
service elasticsearch stop
rm -rf /var/log/elasticsearch/*/*.log*
cat /dev/null > /var/log/logstash/logstash.log
service elasticsearch start
Wait a minute for it to load, then delete indexes manually through Admin -> Manage Indexes

Then

Code: Select all

service logstash start
Finally, lets verify what the settings are on the backup and maintenance page, can you post a screenshot?
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
Mossx
Posts: 3
Joined: Mon Dec 29, 2014 11:23 am

Re: Server not running, Disc Space full of old Indexes

Post by Mossx »

That worked out great, thank you.

Here's the screen shot of the backup and maintenance page settings.
You do not have the required permissions to view the files attached to this post.
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: Server not running, Disc Space full of old Indexes

Post by tgriep »

It looks like you have not setup a Backup Repository, below is the link on how to create one.
If you choose a drive with more free space, you shouldn't have any more problems.

http://assets.nagios.com/downloads/nagi ... enance.pdf
Be sure to check out our Knowledgebase for helpful articles and solutions!
Mossx
Posts: 3
Joined: Mon Dec 29, 2014 11:23 am

Re: Server not running, Disc Space full of old Indexes

Post by Mossx »

Thanks again, we'll see what happens over the next few days.
sreinhardt
-fno-stack-protector
Posts: 4366
Joined: Mon Nov 19, 2012 12:10 pm

Re: Server not running, Disc Space full of old Indexes

Post by sreinhardt »

Sounds great!
Nagios-Plugins maintainer exclusively, unless you have other C language bugs with open-source nagios projects, then I am happy to help! Please pm or use other communication to alert me to issues as I no longer track the forum.
Locked