Page 1 of 1

Node Maintenance

Posted: Mon Nov 17, 2014 8:09 pm
by stecino
Hi All,

I need to add more storage, CPU and memory to individual cluster nodes. How can I take a node out of rotation properly so I won't have to corrupt anything.
Also what would happen to the node where logstash is collecting locally?

Thanks in advance

Re: Node Maintenance

Posted: Tue Nov 18, 2014 10:49 am
by lgroschen
The shards have all been backed up on your other nodes so as long as you properly shut down your system the shards on the system that is going down will be recovered by the other nodes so you won't lose any data. Of course, when you turn the machine off to upgrade the hardware it won't report anything locally, but once you introduce it back into the cluster it will be allocated shards, and more than likely more since it's getting an upgrade.

Just do one at a time and give each some time to allocate shards. The best way to know if the cluster is ready to go is to navigate here:

Administration (navigation bar at top of page) > Cluster Status (left side menu) > Cluster Health > Status:

It will be either green, yellow or red. If the cluster shows a Green status then you know the cluster is finished allocating all the shards and you can move onto another server to upgrade.

Re: Node Maintenance

Posted: Tue Nov 18, 2014 2:11 pm
by stecino
lgroschen wrote:The shards have all been backed up on your other nodes so as long as you properly shut down your system the shards on the system that is going down will be recovered by the other nodes so you won't lose any data. Of course, when you turn the machine off to upgrade the hardware it won't report anything locally, but once you introduce it back into the cluster it will be allocated shards, and more than likely more since it's getting an upgrade.

Just do one at a time and give each some time to allocate shards. The best way to know if the cluster is ready to go is to navigate here:

Administration (navigation bar at top of page) > Cluster Status (left side menu) > Cluster Health > Status:

It will be either green, yellow or red. If the cluster shows a Green status then you know the cluster is finished allocating all the shards and you can move onto another server to upgrade.
Thanks, I actually had stopped elasticSearch and logstash on the node. Then I just shutdown the VM and did the resource allocation. Once I joined it back, I noticed that non of the stats for the node were present. It took a while for it to appear. The hosts that I have are testing only, so don't really care about the data.

Re: Node Maintenance

Posted: Tue Nov 18, 2014 4:05 pm
by lgroschen
Let us know how this goes for you or if you need any more assistance.

Re: Node Maintenance

Posted: Wed Nov 19, 2014 2:03 pm
by stecino
lgroschen wrote:Let us know how this goes for you or if you need any more assistance.
Will do thanks