Page 1 of 1

Deleting Test Data (but not ALL data)

Posted: Thu Oct 08, 2015 4:16 pm
by gehogan3
As we are testing Log Server, we have different folks sending in different test data into the system, each knows to tag their data with their name (typically using shipper.py we use something like: program:emmett-test) so it can be found easily.

Now, we want to be able to delete just the test data for certain folks during the cycle. For example, if we are doing manipulation of the data and adding fields during processing...we would want to purge the old data before resending it through shipper again.)

(Here's the part everyone will hate)...with other products, I can just: "program=emmett-test | delete" in the search bar and I am good to go. Only my matching data is gone.

Is it possible to do that with LS?

Thanks,
Emmett

Re: Deleting Test Data (but not ALL data)

Posted: Thu Oct 08, 2015 4:30 pm
by jolson
Is it possible to do that with LS?
Yup! This is no problem. First, let's form a query in the Web GUI for the data we're looking to erase. In my example, I'm going to look for anything with the 'syslog' type:
2015-10-08 16_25_40-Calculator.png
The resulting information in the 'All Events' panel is the information that will be deleted. Next, log into the command line interface of Nagios Log Server and execute the following command:

Code: Select all

curl -XDELETE 'http://localhost:9200/logstash-2015.10.08/_query?q=type:syslog'
The two important variables are

logstash-2015.10.08 - which represents the index that you're deleting data from (one index per day)
and
type:syslog - which matches the query that we entered in the GUI.

Please verify that the information you see in the GUI is the information that you'd like to delete. You will not be able to recover any of the deleted information unless you have backups in place.

After running the XDELETE command, you should see a result as follows:
2015-10-08 16_29_41-mRemoteNG - jesse.png
And in the GUI:
2015-10-08 16_30_32-Calculator.png
Best,


Jesse

Re: Deleting Test Data (but not ALL data)

Posted: Thu Oct 08, 2015 4:45 pm
by gehogan3
That worked GREAT! Thanks!

Re: Deleting Test Data (but not ALL data)

Posted: Thu Oct 08, 2015 4:57 pm
by jolson
No problem, would it be alright if I locked this thread and marked it as resolved?

Re: Deleting Test Data (but not ALL data)

Posted: Mon Oct 12, 2015 4:12 pm
by gehogan3
Sure