Page 1 of 1

Documentation on how NLS clustering works?

Posted: Thu Jan 12, 2017 3:46 pm
by gormank
Hi,
Do you have any documentation on how NLS clustering works?
Saying its redundant and fault tolerant isn't satisfying the people asking...

Thanks

Re: Documentation on how NLS clustering works?

Posted: Thu Jan 12, 2017 4:28 pm
by tmcdonald
Jesse Olson did a nice talk back in 2015 which serves as a Log Server intro but does go over some of the backend details:

https://www.youtube.com/watch?v=wORXARN0TA0

For documentation specifically, I have always liked this reference guide from the folks behind Elasticsearch:

https://www.elastic.co/guide/en/elastic ... -docs.html

Re: Documentation on how NLS clustering works?

Posted: Thu Jan 12, 2017 4:40 pm
by rkennedy
For the data portion, in a nut shell, NLS creates indices of your data, one per day. For example -

logstash-2017.01.10
logstash-2017.01.11
logstash-2017.01.12

Then, they are split into shards. With our settings, each index has a total of 5 shards. These shards have two pieces to them, a primary, and a replica.

- If your machine has one member, then only primary shards will exist.
- If your machine has more than two members, then the shards are distributed across all of your nodes, with one primary, and one replica specifically placed on different nodes.

The youtube video @tmcdonald posted above, and the Elastic information will be your best guidelines to follow, but I just wanted to bring this part up as an 'easy' explanation.

Re: Documentation on how NLS clustering works?

Posted: Thu Jan 12, 2017 6:26 pm
by gormank
Thanks for the links and quick overview. Now I have more reading to do... :)

I found the following info on the Elastic site as well. https://www.elastic.co/guide/en/elastic ... uster.html

Re: Documentation on how NLS clustering works?

Posted: Fri Jan 13, 2017 10:22 am
by rkennedy
Awesome - thanks for sharing @gormank! I'll leave this open should any other questions arise.