Compression

[WillemDH]

Some colleagues had questions about compression of the log data. is there compression configured on the log data in NLS?

In this article, the conclusion is

https://github.com/jordansissel/experim ... earch/disk

This test confirms what many logstash users have already reported: it is easy to achieve a 5-6x increase in storage from raw logs caused by common logstash filter uses, for example grok.

Summary of test results:

Enabling store compression uses 55% less storage
Removing the @message and @source fields save you 26% of storage.
Disabling the '_all' field saves you 13% in storage.
Using grok with 'singles => true' had no meaningful impact.
Compression ratios in LZF were the same as Snappy.
Final storage size was 25% the size of the common case (1358mb vs 344mb!)

I was wondering how Nagios configured compression?

Grtz

Willem

[jolson]

Please refer to the following URLs:
http://www.elasticsearch.org/guide/en/e ... store.html
http://www.elasticsearch.org/guide/en/e ... field.html

Selecting quotes from the above:

From version 0.90 onwards, store compression is always enabled.

From version 0.90 onwards, all stored fields (including _source) are always compressed.

In the most recent version of Nagios Log Server, Elasticsearch is at version 1.3.2, implying that compression is turned on.

Is there anything else we could help you with answering?

[WillemDH]

Thank you. This thread can be closed.