What is the best way to protect NLS from a split brain?

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
Jklre
Posts: 163
Joined: Wed May 28, 2014 1:56 pm

What is the best way to protect NLS from a split brain?

Post by Jklre »

What is the best way to protect NLS from a split brain scenario?
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: What is the best way to protect NLS from a split brain?

Post by jolson »

The setting that you're looking for is called 'discovery.zen.minimum_master_nodes' - this setting can be found in /usr/local/nagioslogserver/elasticsearch/config/elasticsearch.yml. Set this to a quorum of the total amount of instances in your cluster.

If you have 3 instances, minimum master nodes would be set to '2'.
If you have 4 instances, it would be set to '3'.
Five instances, set it to '3'.

Essentially you take the total amount of instances in your cluster, divide that number in half, and add one.

The only exception is if you have a two instance cluster, you will want to set minimum master nodes to 2. This is tricky, because setting it to 2 means that if one of your two instances goes down, the cluster will temporarily stop operating - but this is better than the split brain scenario that would occur if you have it set to '1'.

You will need to restart the elasticsearch service after you change the configuration file.

Does that makes sense?
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
Locked