How to resync without rebooting

[vmesquita]

Hello,

We have 2 log server instances. Sometimes they lose sight of each other (i.e., one instance appears to be offline to the other althought both are still online). We found out that it's possible to fix this by rebooting the servers. Is there another way to fix this?

[rkennedy]

Generally if that's happening, it's an issue with elasticsearch. You could probably just restart that service to fix it. service elasticsearch restart

I'm curious to know what happens exactly, as this sounds like a bug of some sort. When it happens next time, could you post your /var/log/elasticsearch/<yourclusterid>.log file for us to review?

Lastly, what version of NLS and OS are you running?

[vmesquita]

Thanks, It solved the problem indeed. The logs on boths instances were empty (zero byte files). We are running Red Hat Enterprise Linux Server release 6.7 (Santiago). What does NLS stand for and how we can get its version number?

[jomann]

NLS = Nagios Log Server, the version number can be found at the very bottom lower left-hand corner of the interface.

[vmesquita]

Ah, right! Nagios Log Server 1.4.2

[jomann]

Is there anything that happens before they stop communicating with one another? Have you made any changes to your elasticsearch configuration? There is a "split brain" problem in elasticsearch were both nodes decide that they are master, which can cause issues. I believe that might have been what was happening. Although it's hard to say for sure, if it happens again, copy over the elasticsearch log for us and we can try to decipher what is causing the issue.