Instance status rotates outages

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
jspink
Posts: 43
Joined: Wed Nov 25, 2015 3:27 pm

Instance status rotates outages

Post by jspink »

We currently have a 10 node cluster license. Recently, we've added the final servers to round it out and use all 10.
As soon as the 10th server gets added to the cluster, clicking Instance Status will randomly rotate through the list of server service status' and show that Elastic and LogStash are both down.
Refresh seconds later, and another server shows its down.

We've repeatedly checked each instance, and found that the services are NOT shutting down, yet each refresh of the instance status, indicates that this is happening.

If I drop a single server out of the cluster, every refresh shows each of my instances green.
See screenshot below.
Note that we are in the process of standardizing our drive size in the cluster, thus the differences in Total/Avail
Thoughts? Suggestions?
Nagios_Instance_status.png
You do not have the required permissions to view the files attached to this post.
Nagios Log Server: 10 Instances - 3,916,302,797 documents last check in 180 shards
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: Instance status rotates outages

Post by tmcdonald »

I'd like to get this tested on our end - what LS version are you running?
Former Nagios employee
jspink
Posts: 43
Joined: Wed Nov 25, 2015 3:27 pm

Re: Instance status rotates outages

Post by jspink »

1.4.1 Currently
Planning on upgrading to .4.2 early next week
Nagios Log Server: 10 Instances - 3,916,302,797 documents last check in 180 shards
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: Instance status rotates outages

Post by tmcdonald »

We'll make sure to get this tested, but we are currently doing some maintenance on our ESX infrastructure, so we might not get to this until tomorrow or early next week. Will definitely keep you posted!
Former Nagios employee
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: Instance status rotates outages

Post by mcapra »

Verified this internally. Filed an internal bug report (ID 9207).
Former Nagios employee
https://www.mcapra.com/
Locked