Page 1 of 2

Recommended specs?

Posted: Fri Oct 17, 2014 2:34 pm
by snapon_admin
Is there any particular recommended hardware specs for Log server? We're just going to turn up a pretty basic setup to test it out for now, but I know this will come up later if (realistically "when") we decide to go to production with this.

Re: Recommended specs?

Posted: Fri Oct 17, 2014 8:59 pm
by krobertson71
snapon_admin wrote:Is there any particular recommended hardware specs for Log server? We're just going to turn up a pretty basic setup to test it out for now, but I know this will come up later if (realistically "when") we decide to go to production with this.
It will ultimately be determined by how much log data you are going to be pulling in and how often you are going to index it.

For testing the prebuilt vm should do a fair amount.

Re: Recommended specs?

Posted: Mon Oct 20, 2014 11:13 am
by slansing
As Kris mentioned, one of the more important factors is going to be storage size, for more detailed information, you can take a look at the Elk Stack requirements as they are virtually identical to what we require:

http://www.elasticsearch.org/guide/en/e ... dware.html

Re: Recommended specs?

Posted: Tue Oct 21, 2014 2:00 pm
by snapon_admin
Ok, thanks for the tips. What are most people starting up with as far as RAM is concerned. I saw from the link your provided that 64GB is the sweet spot, but 32 and 16G machines are also common. We're likely going to be using the VMWare image for this, so expanding the memory will be as simple as a few clicks, but I was curious about what others that have this running already are doing.

Re: Recommended specs?

Posted: Tue Oct 21, 2014 2:29 pm
by vAJ

Re: Recommended specs?

Posted: Tue Oct 21, 2014 2:38 pm
by lmiltchev
@ snapon_admin
We're likely going to be using the VMWare image for this, so expanding the memory will be as simple as a few clicks, but I was curious about what others that have this running already are doing.
The number of people using Nagios Log Server is increasing fast, so I expect a lot of feedback from users in the near future. Let us know what were the "right" specs for your environment as it is difficult the do a "real world testing" in house.

Re: Recommended specs?

Posted: Thu Oct 23, 2014 6:56 pm
by eloyd
FYI: Do not attempt to run on a 1GB RAM, 2 core VM if you're going to send real-time virtual web server hosting log files at it. :-)

However, it ran fine just processing our multi-host VoIP logs on the same box. We're trying to tweak a minimum footprint VM to run our VoIP server logs as well as our VoIP server Apache logs, but not our web-hosting logs. So far, memory is (of course) the bottleneck.

It might be time to repurpose a previous box for our continued NLS testing...

Re: Recommended specs?

Posted: Fri Oct 24, 2014 7:53 am
by scottwilkerson
2GB RAM is the bare minimum... I may up the specs to 4 GB minimum as by default elasticsearch can use up to 2GB and it works best if you leave about the same for the system cache because it can take advantage of that.

64GB would really be the max recommended, only allowing elasticsearch the ability to use 32GB. If you allow elasticsearch the ability to use more than 32GB there is an affect where the performance starts to decrease and it would be more efficient in processing ability and cost to run an additional instance.

Re: Recommended specs?

Posted: Fri Oct 24, 2014 1:05 pm
by eloyd
Our NLS instance that's chewing through our VoIP server logs (Asterisk logs) is doing just great with 2GB and 2 cores on VM shared with some of the actual VoIP servers it's receiving logs from. In fact, we like it so much we're using this for our dashboard for getting an overview of our current call usage and patterns rather than our home-brew database extract/analysis tools!

Re: Recommended specs?

Posted: Fri Oct 24, 2014 1:07 pm
by sreinhardt
Awesome use case! Do you have any idea the amount of data your pushing presently?