Page 1 of 1
Stopped Receiving Logs
Posted: Tue Sep 01, 2015 1:11 pm
by blariv
we have a 2 node cluster of nls. all has been working but logs stopped showing up from all servers pointing to it about a week ago. just wondering where I can look to see what is happening.
Re: Stopped Receiving Logs
Posted: Tue Sep 01, 2015 1:19 pm
by jolson
Is logstash running on your instances? Run the following on both nodes:
If logstash is stopped, you'll need to restart it:
If that's not what is causing the issue, I would like to see the output of the following logs:
Code: Select all
cat /var/log/logstash/logstash.log
tail -n200 /var/log/elasticsearch/*.log
Re: Stopped Receiving Logs
Posted: Tue Sep 01, 2015 1:38 pm
by blariv
thanks for the quick reply.
on my primary server I get no output from
cat /var/log/logstash/logstash.log
the rest are attached
Re: Stopped Receiving Logs
Posted: Tue Sep 01, 2015 1:43 pm
by jolson
Is elasticsearch running on the instance that you checked?
It's possible that elasticsearch was killed due to 'out of memory' problems.
Your logstash output also looks a little strange. Try running 'Apply Config' from the Web GUI and see if that helps. If it does not, I'd like to see the output of the following
from all of your instances:
Code: Select all
cat /usr/local/nagioslogserver/var/cluster_hosts
cat /usr/local/nagioslogserver/var/cluster_uuid
cat /usr/local/nagioslogserver/logstash/etc/conf.d/999_outputs.conf
Re: Stopped Receiving Logs
Posted: Tue Sep 01, 2015 1:52 pm
by blariv
that worked thanks!
Re: Stopped Receiving Logs
Posted: Tue Sep 01, 2015 1:57 pm
by jolson
No problem!
I assume the Apply Config worked for you? Is this thread good to close?
Re: Stopped Receiving Logs
Posted: Tue Sep 01, 2015 2:18 pm
by blariv
that is what solved it, you can close.