logstash keep dying

[technosol]

Hi

I have created an output plugin to forward all the logs to external storage over UDP. But after sometime logstash dies. Is that because of the high load of data? of course, the load of data is quite high; also this is happening in the testbed environment.

udp {

host => 'remote ip'
port => 'port'

}

or is there a limitation in logstash how much data we can send in a UDP packet? Are you able to help me find a solution for it?

also please share with me, if you have elasticsearch, logstash fine tuning guide.


Thank you.

[mcapra]

Is that because of the high load of data?

I suspect not.

is there a limitation in logstash how much data we can send in a UDP packet?

Yes, via the buffer_size directive. 8192 bytes by default.

The Logstash logs held in /var/log/logstash likely have some clues.

[scottwilkerson]

The Logstash logs held in /var/log/logstash likely have some clues.

Correct, OP, can you post the output of

Code: Select all

tail -50 /var/log/logstash/logstash.log

[technosol]

Hi

Thank you for the reply.

I use following UDP output plugin to forwarding logs to an external storage.

https://www.elastic.co/guide/en/logstas ... s-udp.html

However, at the moment I have disabled the plugin as it causes crashing logstash continuously. So I am afraid I can find relevant logs.

[mcapra]

I use following UDP output plugin to forwarding logs to an external storage.

I overlooked that this is an output rule. Slightly more relevant documentation:
https://www.elastic.co/guide/en/logstas ... s-udp.html

Which doesn't appear to have any particular restrictions other than the number of worker threads. If the destination system *does* have such restrictions, that could be causing issues within the logstash-udp-output plugin which will eventually choke the message processing pipeline; This kills the Logstash.

No real way of knowing for sure without seeing the specific exceptions being thrown around in the logs, though.

[scottwilkerson]

I completely agree with @mcapra

I will add that you likely want to add the workers directive and a higher number than 1 which will allow more threads to be opened, however this could still backlog if the receiving side cannot accept the messages fast enough

Code: Select all

workers => 8