json formatted logs

[mike4vr]

We have a specific application that writes logs in json format. I configured the server as a linux server and sent the logs as 'file.' Unfortunately, the log server picks up the json entries one-by-one for each line, which is understandable. I'm not sure if there's any way to get NLS to understand the json format. At any rate, we changed the log output to "trunkate" each json entry into one line. That worked, but it doesn't appear to be picking up the whole entry. The last part of each entry gets cut off. Is there a character limit in each value of the "message" field?

[jolson]

mike4vr,

When you say that you're using the log output to 'truncate' the json entry into one line, do you mean that you used rsyslog's ReadMode 2? I ask because that's the way that I've accomplished this task - and it cuts off the log file as you noted.

There is a 'json' input codec, which will take json formatted logs and concatenate them, have you tried the json codec out? http://logstash.net/docs/1.4.2/codecs/json

If you can post an example of the log files you're receiving, I will do some tests on my end to see if I can get this functioning.

[mike4vr]

No, we changed the way the application outputs the log and made it a one liner. Problem with that is that the log server appears to only accept a certain number of characters. I wanted to clarify that.

As for the codec, it isn't exactly clear as to how to utilize it. Can you elaborate on that, if possible?

[jolson]

Could you send an example log my way?

Problem with that is that the log server appears to only accept a certain number of characters.

Nagios Log Server does not limit the amount of characters that you can input - or at least the threshold is not very high. I tested this by sending a 19,000 character one-liner to logstash, and it was indexed properly:

2015-05-01 14_09_35-Dashboard • Nagios Log Server.png

These logs were sent to the generic 'syslog' input, as well as a 'tcp' input.
My guess is that your log files are being cut off before they are sent to NLS - rsyslog is the most notable offender that I've run into.

To determine whether or not the JSON codec would work for you, an example log would be helpful - but the codec is quite simple, you implement it at the 'input' level:
http://logstash.net/docs/1.4.2/codecs/json
An input example:

Code: Select all

input { 
  tcp {
    codec => json {
    }
  }

Let me know if you have further questions. Thanks!

[mike4vr]

Thanks for the reply.

I used the same exact block as you posted for testing and this is the result I get:

[jolson]

mike4vr,

You will need to implement the json codec on the Logstash side of things - I have included a screenshot and an example config for clarity.

2015-05-04 09_12_41-Instance Configuration • Nagios Log Server.png

This input tells Logstash to recognize json formatted logs and to parse them appropriately. You will also note that I am using the 'tcp' input - you are free to use 'udp' here as well if your logs are exported using udp. You can read further about inputs here: http://logstash.net/docs/1.4.2/

Please note that whatever port you use, you will need to open in your Nagios Log Server firewall. Thanks!

[mike4vr]

Fantastic. Thank you for your help. I have it working. I did, however, need to modify rsyslog.conf to increase the message size. Without this, it will truncate the message and deem the json invalid. The entry is:

Code: Select all

$MaxMessageSize 40k

Size can be adjusted to needs.

[jolson]

mike4vr,

That is fantastic to know - I've been looking for that entry for some time. Much appreciate you getting back to us.

I'll lock this thread. Feel free to open a new one if you have additional problems or questions. Thanks again!