We've moved to a new support system!

The Nagios Answer Hub is a place where you can get help with technical questions from our experts. There, you can quickly open tickets and join discussion boards.

Request Nagios Answer Hub access here: https://info.nagios.com/answer-hub-access-new-users

After completing the access form, you will be given access to a portal where new tickets can be created. We will keep the old customer forum sections and ticket system available for current cases to be resolved.

Go to the Answer Hub

Small Cluster Design

Engage with the community of users including those using the open source solutions.
Includes Nagios Core, Plugins, and NCPA

Small Cluster Design

Postby JohnSonandrla » Tue Aug 16, 2022 5:06 am

We started off with Nagios Log Server (it's Elasticsearch underneath). However, I think its front-end is a little limited, and it's been choking on the data we feed it (OoM errors). Currently, it's getting ~15GB a day, but we have other logs that are not yet being sent. That's installed to a single VM with 4 CPUs and 8GB of RAM.

To be able to handle more logs, and avoid licensing costs, I thought I'd design a multi-node ELK cluster. However, I'm stuck on how to set this up. A lot of what I read use many huge machines. Small 3 node cluster - 16 CPUs and 32GB RAM each. Yeah we don't have the resources for that. I have a rough "budget" of 8 CPUs and 16 GB of RAM total to work with.

With such limited hardware available, is it even worth trying to do a multi-node cluster? Should I just install everything (E, L, & K) to one big VM? (Should I just bulk up the Nagios Log Server product and call it a day?)

My rough plan for a small cluster was:

3x data nodes running Elasticsearch (all masters), 2 CPUs, 4 GB RAM each

1x Kibana node, 2 CPUs, 4 GB RAM

1x dedicated Logstash, 2 CPUs, 4 GB RAM

I don't know if this is even a good idea. Does having multiple machines outweigh having so little RAM? I'm thinking no.

And then there's the whole issue of picking how many shards and replicas to use... (I was thinking 3 shards, 1 replica)

I'm completely overloaded with info, and I think I'm in over my head. Elasticsearch omegle shagle voojio is a huge topic and everything seems to depend on your specific data. Any guidance is super appreciated.
Last edited by JohnSonandrla on Wed Sep 21, 2022 7:24 am, edited 1 time in total.
JohnSonandrla
 
Posts: 4
Joined: Sun Jul 10, 2022 12:49 am

Re: Small Cluster Design

Postby gormank » Tue Aug 16, 2022 2:33 pm

Here's a doc with some info on sizing NLS, but I can't say much about best practices for setting up an ELK system, other than having a single logstash seems risky. I've been told that a minimum redundant NLS system is three hosts.

https://assets.nagios.com/downloads/nag ... hrough.pdf
https://assets.nagios.com/downloads/nag ... raluse.php
gormank
 
Posts: 1031
Joined: Tue Dec 02, 2014 12:00 pm


Return to Community Support

Who is online

Users browsing this forum: No registered users and 12 guests