Page 1 of 1

Different log storage duration based on type

Posted: Sat Oct 27, 2018 12:16 am
by technosol
Hi,

We have two types of logs coming into the system and we need to store one type of logs for a longer duration then the other type.

I do understand that NLS can only drop logs based on index and we cannot delete based on any other parameter of the log (please let me know if we have any other options)

To handle this we were thinking to handle based on index so that we can drop the index based on the duration.

If we to do this how can we go about it? Our main goal is to have different storage duration based on the log type.


There are other similar request to this.
https://support.nagios.com/forum/viewto ... 38&t=41857

Re: Different log storage duration based on type

Posted: Mon Oct 29, 2018 9:39 am
by cdienger
I think the best way to go about this would be to configure the retention period to save the data for the longer of the two periods, and then periodically run a command manually or via a cron job to delete the older log types. The command would look like:

curl -XDELETE 'http://localhost:9200/logstash-2018.02. ... ype:syslog'

Re: Different log storage duration based on type

Posted: Mon Oct 29, 2018 9:59 am
by technosol
Got it, we try out the method suggested. Thanks for the help

Re: Different log storage duration based on type

Posted: Mon Oct 29, 2018 1:14 pm
by cdienger
Sounds good! We'll be here :)