Creating custom indexes

[krobertson71]

I know in elasticsearch you can create custom indices for specific events. NLS creates an index for each day.

I know I can go into the elasticsearch bin and create one that way but I have no idea how that would impact NLS.

I would like to get to a point where I can put security related events in one index, network events in another, etc...

Also I see ELK as released Shield as a commercial plug-in. I know AD integration was added, but what about role based user authorization so I can limit what a user can see based on an index.

Just curious where Nagios stands on all of this.

[jolson]

what about role based user authorization so I can limit what a user can see based on an index.

This is being actively worked on and a future release will allow role-based access to logs.

In terms of creating custom indices for certain 'types' of log data, I do not recommend doing that - instead, make separate dashboards using filters to filter out any data that isn't relevant to that particular dashboard. Making separate indices can cause problems because they won't rotate/be backed up among other things along with our default indices - they will be static and persist indefinitely.

I cover filter vs query creation here:
https://support.nagios.com/forum/viewto ... 38&t=36320

[krobertson71]

By default Logstash puts events into a daily index. So I am assuming that is what you built around?

[quote]Logstash, by default, creates a new index per day, which then contains all events received from its inputs. You can configure it to change the rotation period (if any) or make separate indices per user or other property (like the type of event). [/quote

It is also stated that you can put TTL's and other time based criteria on how long data is kept in a particular index and how often it is backed up.

This is why I was asking if something in NLS is not allowing for the creation of new indices or if you created NLS around the Logstash defaults which would cause system issues if attempted.

I do understand what you are saying about filters, but you can also do the same thing with kibana.

You said you are actively working on authorization.. will this include the ability (or is being discussed) to keep a person from viewing events based on a tag or field?

This is why I am going back to the indices question as that can be accomplished, albeit with a cost from ELK.

[jolson]

This is why I am going back to the indices question as that can be accomplished, albeit with a cost from ELK.

Understood. We don't support the shield component as we'll be adding in a similar component soon - to answer your question:

You said you are actively working on authorization.. will this include the ability (or is being discussed) to keep a person from viewing events based on a tag or field?

I have confirmed with the developers that our user-based control system will be based on filters rather than indices, though details are still being worked out.

[krobertson71]

Cool, glad to hear that. Already getting questions about sending some data to NLS that they don't want for public exposure.

Hopefully it will come soon.. Along with the ability to export reports or export events to be put into a report.

[jolson]

Those are actually the two most requested features for Nagios Log Server. We're aware of the importance of those features and are actively working towards a solution. Is there any particular report that would be useful for you?