Hi
Suppose I have 90GB size of data input to NLS daily and the log retention period is 120 days. I am planinig to store the data in a fast SAN space.
Per NLS node/instance, recommended by Nagios; Load is 25GB per day. May I know how manay NLS instances required to cater my requirement?
As I understood, 4 no.of instances are enough to handle the daily load. Correct me if I am wrong and please share with me if you have a calculation method to identify no.of required NLS instances for such scenarios .
Thank you.
No.of required NLS instances
Re: No.of required NLS instances
It's difficult to say for as it depends on the type of data as well as how many indices you'll have open at a time. In most cases people only need immediate access to a fraction of what could be made available and should be closing/saving older data. It's best to keep open only recent indices and closing older ones as needed. That said, 4 instances is a good starting point and I would take advantage of NLS' free trial to install more if needed to get a good sizing that works for your environment.
For steps to setup archiving older data: https://assets.nagios.com/downloads/nag ... enance.pdf
For steps to setup archiving older data: https://assets.nagios.com/downloads/nag ... enance.pdf
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Re: No.of required NLS instances
Hi
Do we have access to all the features in the NLS free trial version?
Do we have access to all the features in the NLS free trial version?
Re: No.of required NLS instances
Yes, the trial enables all functionality for 60 days.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Re: No.of required NLS instances
Hi,
Thank you for the clarifications. Please help me clear the doubts as we need committed to the customer .
We have got a request from a client to store 120 days of logs each day generates 72GB of logs, based on the recommendation from Nagios for a node not to exceed 25GB per day.
We have proposed a 4 Node cluster of NLS to be deployed, we have couple of questions regarding this hope you can help to clarify.
1) Customer wants to know because this will be deployed on the VM and does not want to allocate too much storage, can we go ahead with a 4 node cluster but have a single data source for storage without replicating the data across the multiple nodes and let the VM handle the storage redundancy? is there any documentation which we can referrer as a guideline?
2) What is the recommended storage size and IOPS for a system this big?
3) Is there anything special we need to consider when it comes to storage size of 8TB +
4) Any other technical items or licensing we need to consider?
Thank you for the clarifications. Please help me clear the doubts as we need committed to the customer .
We have got a request from a client to store 120 days of logs each day generates 72GB of logs, based on the recommendation from Nagios for a node not to exceed 25GB per day.
We have proposed a 4 Node cluster of NLS to be deployed, we have couple of questions regarding this hope you can help to clarify.
1) Customer wants to know because this will be deployed on the VM and does not want to allocate too much storage, can we go ahead with a 4 node cluster but have a single data source for storage without replicating the data across the multiple nodes and let the VM handle the storage redundancy? is there any documentation which we can referrer as a guideline?
2) What is the recommended storage size and IOPS for a system this big?
3) Is there anything special we need to consider when it comes to storage size of 8TB +
4) Any other technical items or licensing we need to consider?
-
- DevOps Engineer
- Posts: 19396
- Joined: Tue Nov 15, 2011 3:11 pm
- Location: Nagios Enterprises
- Contact:
Re: No.of required NLS instances
I would highly discourage this as it removes the ability for the cluster to heal itself, however if you must you can run the procedure in this document on each of the Instances in the cluster and it will not make replica shards any longer.technosol wrote:1) Customer wants to know because this will be deployed on the VM and does not want to allocate too much storage, can we go ahead with a 4 node cluster but have a single data source for storage without replicating the data across the multiple nodes and let the VM handle the storage redundancy? is there any documentation which we can referrer as a guideline?
https://assets.nagios.com/downloads/nag ... oyment.pdf
Storage size depends on on the size you outlined which would be about 8.6TB + I would add a healthy amount of extra space to accommodate the indexes, and burst. I would say at least 15TB to be save, and because of the quantity of data being searched and indexes the amount of IOPS need to be extremely high, I would be looking for > 10,000 in an optimal setup.technosol wrote:2) What is the recommended storage size and IOPS for a system this big?
Just disk speed, the faster the better really.technosol wrote:3) Is there anything special we need to consider when it comes to storage size of 8TB +
It is important to reiterate, that there are many factors that we cannot consider like size of each log, number of filters configured on the inputs, burst (do the 72GB come evenly spread out over 24 hours or in bursts), and it may be possible that 4 Instances is under shooting, and the client may need to increase the number of nodes to accommodate the setup.technosol wrote: 4) Any other technical items or licensing we need to consider?