Stopped Receiving Logs

[blariv]

we have a 2 node cluster of nls. all has been working but logs stopped showing up from all servers pointing to it about a week ago. just wondering where I can look to see what is happening.

[jolson]

Is logstash running on your instances? Run the following on both nodes:

Code: Select all

service logstash status

If logstash is stopped, you'll need to restart it:

Code: Select all

service logstash start

If that's not what is causing the issue, I would like to see the output of the following logs:

Code: Select all

cat /var/log/logstash/logstash.log
tail -n200 /var/log/elasticsearch/*.log

[blariv]

thanks for the quick reply.

on my primary server I get no output from
cat /var/log/logstash/logstash.log

the rest are attached

[jolson]

Is elasticsearch running on the instance that you checked?

Code: Select all

service elasticsearch status

It's possible that elasticsearch was killed due to 'out of memory' problems.

Your logstash output also looks a little strange. Try running 'Apply Config' from the Web GUI and see if that helps. If it does not, I'd like to see the output of the following from all of your instances:

Code: Select all

cat /usr/local/nagioslogserver/var/cluster_hosts
cat /usr/local/nagioslogserver/var/cluster_uuid
cat /usr/local/nagioslogserver/logstash/etc/conf.d/999_outputs.conf

[blariv]

that worked thanks!

[jolson]

No problem! I assume the Apply Config worked for you? Is this thread good to close?

[blariv]

that is what solved it, you can close.