Deleting Test Data (but not ALL data)

[gehogan3]

As we are testing Log Server, we have different folks sending in different test data into the system, each knows to tag their data with their name (typically using shipper.py we use something like: program:emmett-test) so it can be found easily.

Now, we want to be able to delete just the test data for certain folks during the cycle. For example, if we are doing manipulation of the data and adding fields during processing...we would want to purge the old data before resending it through shipper again.)

(Here's the part everyone will hate)...with other products, I can just: "program=emmett-test | delete" in the search bar and I am good to go. Only my matching data is gone.

Is it possible to do that with LS?

Thanks,
Emmett

[jolson]

Is it possible to do that with LS?

Yup! This is no problem. First, let's form a query in the Web GUI for the data we're looking to erase. In my example, I'm going to look for anything with the 'syslog' type:

2015-10-08 16_25_40-Calculator.png

The resulting information in the 'All Events' panel is the information that will be deleted. Next, log into the command line interface of Nagios Log Server and execute the following command:

Code: Select all

curl -XDELETE 'http://localhost:9200/logstash-2015.10.08/_query?q=type:syslog'

The two important variables are

logstash-2015.10.08 - which represents the index that you're deleting data from (one index per day)
and
type:syslog - which matches the query that we entered in the GUI.

Please verify that the information you see in the GUI is the information that you'd like to delete. You will not be able to recover any of the deleted information unless you have backups in place.

After running the XDELETE command, you should see a result as follows:

2015-10-08 16_29_41-mRemoteNG - jesse.png

And in the GUI:

2015-10-08 16_30_32-Calculator.png

Best,


Jesse

[gehogan3]

That worked GREAT! Thanks!

[jolson]

No problem, would it be alright if I locked this thread and marked it as resolved?

[gehogan3]

Sure