upgrade to 2.2

[jvestrum]

Hello, I'm upgrading our two-node NLS cluster from R1.4 to R2.2. I ran the upgrade script on the first node, which seems to have gone well, but now the cluster state is Yellow and all the shards are Unassigned. The upgrade doc says to wait for the cluster state to go green before moving on to the next node, but nothing seems to be happening. Advice?

[jvestrum]

FYI:

[root@gtcs-nls01 nagioslogserver]# es cluster-health
{
"cluster_name" : "5e48d284-3c27-4184-b703-49e6a9830018",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 2,
"active_primary_shards" : 194,
"active_shards" : 194,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 194,
"number_of_pending_tasks" : -1,
"number_of_in_flight_fetch" : 0
}

[jvestrum]

I think this is what's going on:

https://www.elastic.co/guide/en/elastic ... g-upgrades

If it is not possible to assign the replica shards to another node with the higher version — e.g. if there is only one node with the higher version in the cluster — then the replica shards will remain unassigned, i.e. the cluster health will be status yellow. As soon as another node with the higher version joins the cluster, the replicas should be assigned and the cluster health will reach status green.

So I think I can just go ahead and upgrade the other node.

[jolson]

Quoted from the documentation:

During a rolling upgrade, primary shards assigned to a node with the higher version will never have their replicas assigned to a node with the lower version, because the newer version may have a different data format which is not understood by the older version.
If it is not possible to assign the replica shards to another node with the higher version — e.g. if there is only one node with the higher version in the cluster — then the replica shards will remain unassigned, i.e. the cluster health will be status yellow. As soon as another node with the higher version joins the cluster, the replicas should be assigned and the cluster health will reach status green.

Let's see where your shards are assigned:

Code: Select all

curl -s localhost:9200/_cat/shards

Are the primary shards (indicated by a 'p') assigned to your node with a higher version of elasticsearch?
You can check a nodes UUID by issuing the following command:

Code: Select all

cat /usr/local/nagioslogserver/var/node_uuid

If the primary shards are all on your recently upgraded node, you are good to upgrade your other 1.4 box. If not, let me know.

[jvestrum]

I went ahead and upgraded the other node. After that, it was still at Yellow. Then I re-enabled shard allocation (I had disabled it as per the docs) and now I'm back to Green. All looks good now, thanks.

[jolson]

No problem, glad to hear you're up and running.