NLS stopped working

[WillemDH]

Scott,

Attached screenshot of the very nice concealed instance information..

Could you tell me how much data you think we should be able to handle daily with two nodes? Is there some way to see how much data each source uses? As we have several 'customers', it would be nice to be able to calculate the cost per source.

Grtz

Willem

[scottwilkerson]

Could you tell me how much data you think we should be able to handle daily with two nodes?

This is somewhat a loaded question as with many items, the answer is, it depends. Below are some of the factors it depends on:

• What type of filters (and quantity) are added to the logstash config
  Speed of Disks
  Amount of RAM
  Quantity of people querying the data
  How even the data comes in (bursty or steady stream)

One thing I will point out, performance wise, is that there is only a marginal benefit of 2 nodes over a single as all data is indexed on both instances, the real load reduction benefit comes with 3+ nodes as the indexing will always only happen on 2 instances.

Is there some way to see how much data each source uses?

Data usage by source is not available. The closest you can really get would be the quantity of docs per source

[WillemDH]

Ok, thanks. We will not have any budget left this year to buy other nodes I fear. My management wants to first see some results from it. I had my presentation today by the way and it all worked fine. So first impression for my 30 colleagues is ok.
How would I see the number of documents per source? With a document, do you mean a log entry? If so, I guess a dashboard with a list and count of number of alerts would be it right?

[scottwilkerson]

How would I see the number of documents per source? With a document, do you mean a log entry? If so, I guess a dashboard with a list and count of number of alerts would be it right?

Yes, I meant log entry. And yes, you can create a dashboard, taking a clue from the default "Top Sources and Types" dashboard, the "Top Hosts" panel in the upper gives exactly this information, however you would likely want to edit it so that it displays more than 10.

Configure -> Panel Tab -> Length = 99999

[WillemDH]

Ok, that would indeed give an idea of how much % of our storage is taken by each source. As the cost of storage tends to grow quickly we would need to be able to see who consumes what.

I have one more question, is there a way to configure data retention per source? We have some devices, like our firewall of which we need to save the logs fro at least one year (legally). Other sources such as our ESXi server which generate huge amount of logs are less critical and we would only need to save their logs for one month, (as it would just cost too much)

Is this possible somehow or is log retention a global setting?

[tmcdonald]

I don't believe this is possible, but I will defer to a developer for specifics. The reason I don't think it is possible is because we don't discriminate where a log comes from when it gets stored in an index - all indices are based on the date regardless of what goes in them. It should be possible in the ELK stack itself with inputs, but how that would work within Logserver would take some time to figure out.

[scottwilkerson]

I will mention it is on the roadmap to build a section for the admins to delete items based on query.

It will work something like this:

Create a query for items you will want to prune

Setup length of time to keep items matching query (e.g. X min, or X hours, or X days)

A job will be scheduled that will prune all data older than time specified that matches Query/filter specified.


Does this sound like it would fit the bill?

[WillemDH]

Scott,

The solution you propose would work for us. If we can schedule a job which purges the data from certain sources, this would be almost the same as setting retention per source. So that would fit the bill. Do you want me to make a feature request for this or does it already exists?

Grtz

Willem

[tmcdonald]

I could create it internally. Anything you wanna add or are all the details here?

[WillemDH]

Trevor,

If what Scott proposes

Create a query for items you will want to prune

Setup length of time to keep items matching query (e.g. X min, or X hours, or X days)

A job will be scheduled that will prune all data older than time specified that matches Query/filter specified.

Is done well, all the info can be found in this thread.

Is there a separate tracker for Nagios Log Server?

Grtz

Willem