Page 1 of 1

Front End Queue??

Posted: Wed Oct 22, 2014 10:17 am
by pbosley
We are looking to implement a 3 node Nagios Log Server cluster to support our environment. I am concerned that we may generate too much traffic for the logstash processes to handle during peak times. Do you have any suggestions for a receiving queue or the ability to "stack" multiple logstash instances on each node to handle the load should we start dropping data?

Re: Front End Queue??

Posted: Wed Oct 22, 2014 2:00 pm
by scottwilkerson
As logstash is multi-threaded starting additional instances on the same server won't increase throughput, however you could push the data to a queue such as Redis and then have logstash pull from the Redis queue

http://logstash.net/docs/1.4.2/inputs/redis