S3 Bucket Logs to Logstash not working? Advice?

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: S3 Bucket Logs to Logstash not working? Advice?

Post by scottwilkerson »

jslawrenc wrote:One update: Running logstash in debug, I observed the error "The shutdown process appears to be stalled due to busy or blocked plugins" which might suggest my S3 plugin is receiving data at a slow pace. This seems to be somewhat common with S3 bucket logs. I'm not sure how to remedy this yet, but thought I'd share in case it's useful for anyone else down the road.
You would likely have to add this to flag to the startup:
https://www.elastic.co/guide/en/logstas ... tdown.html

It could be added to LS_OPTS in /etc/sysconfig/logstash, like this
LS_OPTS=" -w 4 --allow-unsafe-shutdown"
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
Locked