about shard num

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
winsonlee
Posts: 42
Joined: Tue Mar 05, 2019 10:38 pm

about shard num

Post by winsonlee »

we extend cluster to 8 node, but index shards in "Cluster status" still show 5, it's correct?
You do not have the required permissions to view the files attached to this post.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: about shard num

Post by scottwilkerson »

Yes, each index always has 5 primary shards and 5 backup shards (a replica). Each of these shards are distributed between the 8 nodes in your cluster (making sure each primary and backup shard are always on different nodes), and all shards participate in returning search results.
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
winsonlee
Posts: 42
Joined: Tue Mar 05, 2019 10:38 pm

Re: about shard num

Post by winsonlee »

scottwilkerson wrote:Yes, each index always has 5 primary shards and 5 backup shards (a replica). Each of these shards are distributed between the 8 nodes in your cluster (making sure each primary and backup shard are always on different nodes), and all shards participate in returning search results.
our index has 300GB ~ 350GB per day, 5 shards it's mean each shard size about 60GB, is no issue about performance ?
User avatar
cdienger
Support Tech
Posts: 5045
Joined: Tue Feb 07, 2017 11:26 am

Re: about shard num

Post by cdienger »

No, this shouldn't be a problem.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
winsonlee
Posts: 42
Joined: Tue Mar 05, 2019 10:38 pm

Re: about shard num

Post by winsonlee »

cdienger wrote:No, this shouldn't be a problem.
if i want to increase shards number to 10, how to do that?
User avatar
cdienger
Support Tech
Posts: 5045
Joined: Tue Feb 07, 2017 11:26 am

Re: about shard num

Post by cdienger »

It can be set in /usr/local/nagioslogserver/elasticsearch/config/elasticsearch.yml but please note that this file can be overwritten during upgrades and we recommend the default value of 5 shards per index.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
winsonlee
Posts: 42
Joined: Tue Mar 05, 2019 10:38 pm

Re: about shard num

Post by winsonlee »

cdienger wrote:It can be set in /usr/local/nagioslogserver/elasticsearch/config/elasticsearch.yml but please note that this file can be overwritten during upgrades and we recommend the default value of 5 shards per index.
ok, thanks for you help and notice. let me test is different at 5 to 10 shards.
User avatar
cdienger
Support Tech
Posts: 5045
Joined: Tue Feb 07, 2017 11:26 am

Re: about shard num

Post by cdienger »

Sounds good. Let us know your results.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
winsonlee
Posts: 42
Joined: Tue Mar 05, 2019 10:38 pm

Re: about shard num

Post by winsonlee »

i change shards to 11,it's work, the query speed has been improved.
User avatar
cdienger
Support Tech
Posts: 5045
Joined: Tue Feb 07, 2017 11:26 am

Re: about shard num

Post by cdienger »

Thanks for the update!
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Locked