Page 1 of 1
knowdge base article request .ym and mappings
Posted: Mon Jul 23, 2018 3:26 pm
by benhank
i just made a post hthat got me thinking:
Would you guys be kid enough to put up a knowledgebase article that explains mappings and how to use them with NLS 2.0?
My interest is how to use mappings to optimize NLS for storage and fast retrieval of data
And if possible an article that explains the .yml file and how to use it to make two servers with different HW Harddrive cpu ram um...work together?
and OOOH OOOH! and article that makes using grok and some of the more commonly used filters to transform and mutate data so no matter the datasource it all displays the same? (ok thats a tall order i know )
Thanks guys!
Re: knowdge base article request .ym and mappings
Posted: Mon Jul 23, 2018 3:56 pm
by scottwilkerson
We haven't done so because we believe that the mapping in ES is the best you can get by default and you risk a lot by getting the mappings wrong(e.g. no logs will save)
But, so you have it, elastic describes it here
https://www.elastic.co/guide/en/elastic ... pping.html
Re: knowdge base article request .ym and mappings
Posted: Tue Jul 24, 2018 8:37 am
by mcapra
I've been fortunate to have been under-the-hood of a few different ELK-based log collection platforms, and I would argue that a simple log collection platform (like NLS or any other competitor) is not the best launch-pad for a tricked-out custom ElasticSearch cluster. Most of those platforms are making some pretty sweeping assumptions about how ElasticSearch is structured and you're likely to trip over them at almost every turn.
There's strong arguments for a KB article that explains mappings as they relate to some of the default inputs (namely eventlog and syslog). I can think of several instances where logs weren't persisting because the default mappings made some incorrect assumptions.
Re: knowdge base article request .ym and mappings
Posted: Tue Jul 24, 2018 9:00 am
by scottwilkerson
mcapra wrote:I've been fortunate to have been under-the-hood of a few different ELK-based log collection platforms, and I would argue that a simple log collection platform (like NLS or any other competitor) is not the best launch-pad for a tricked-out custom ElasticSearch cluster. Most of those platforms are making some pretty sweeping assumptions about how ElasticSearch is structured and you're likely to trip over them at almost every turn.
There's strong arguments for a KB article that explains mappings as they relate to some of the default inputs (namely eventlog and syslog). I can think of several instances where logs weren't persisting because the default mappings made some incorrect assumptions.
Noted
Re: knowdge base article request .ym and mappings
Posted: Wed Jul 25, 2018 9:40 am
by benhank
Ok i get it, and you guys are right. I just didnt think it thru. Thanks for the replies!
Re: knowdge base article request .ym and mappings
Posted: Wed Jul 25, 2018 10:59 am
by scottwilkerson
benhank wrote:Ok i get it, and you guys are right. I just didnt think it thru. Thanks for the replies!
Locking