Page 1 of 2
Nagios Log Server - dissect logstash plugin
Posted: Thu Aug 31, 2017 3:18 pm
by bpizzutiWHI
Looks like it doesn't come with it installed. When I try to install it manually, I get this:
[root@nagioslog01-cv2 bin]# ./plugin install logstash-filter-dissect
Validating logstash-filter-dissect
Installing logstash-filter-dissect
Plugin version conflict, aborting
ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core":
In snapshot (Gemfile.lock):
logstash-core (= 1.5.1)
In Gemfile:
Followed by a ton of these repeating:
logstash-input-s3 (>= 0) java depends on
logstash-mixin-aws (>= 0) java depends on
logstash-core (< 2.0.0, >= 1.4.0) java
And finally ending with this:
logstash-filter-dissect (>= 0) java depends on
logstash-core-plugin-api (<= 2.99, >= 1.60) java depends on
logstash-core (<= 2.4.99, >= 2.4.0.snapshot1) java
logstash-core (= 1.5.1) java
Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.
This is not the appliance, this is a custom Cent7 box.
Any ideas on how to get past this?
Re: Nagios Log Server - dissect logstash plugin
Posted: Thu Aug 31, 2017 4:24 pm
by cdienger
The plugin requires a newer version of logstash. I can only find it in documentation starting in version 5. While the new NLS release this year(
https://www.nagios.com/roadmaps/) will include an updated version of logstash, it still doesn't look like this plugin will be available for it. What are you trying to do? Perhaps there's another plugin available that can help.
Re: Nagios Log Server - dissect logstash plugin
Posted: Fri Sep 01, 2017 8:34 am
by bpizzutiWHI
I'm trying to take the information in an Schannel (sourcename) event log entry and convert it into fields. I'm not sure I can do this with grok, dissect seemed to be ideal. Here's the format of the message field I'm trying to dissect:
Code: Select all
An SSL server handshake completed successfully. The negotiated cryptographic parameters are as follows.
Protocol: TLS 1.1
CipherSuite: 0xA
Exchange strength: 2048
Looking to grab Protocol, CipherSuite, and Exchange Strength, and put them each into a field, and also "An SSL <blank>" would be either server or client, make that another field.
I wrote a dissect filter like this to handle it, no idea if I got it right since I can't test it.
Code: Select all
if [SourceName] == "Schannel" {
dissect {
mapping => {
"message" => "%{an} %{ssl} %{&ssl_handshake} %{completed} %{successfully.} %{The} %{negotiated} %{cryptographic} %{parameters} %{are} %{as} %{follows.} %{?ssl_protocol}: %{&ssl_protocol} %{?ssl_cipersuite}: %{&ssl_cipersuite} %{ssl_exch}: %{&ssl_exch}"
}
}
}
Re: Nagios Log Server - dissect logstash plugin
Posted: Fri Sep 01, 2017 9:03 am
by mcapra
bpizzutiWHI wrote:I'm not sure I can do this with grok
Grok is the Swiss army knife of Logstash parsing. It's inefficient and unwieldy at times, but it can get pretty much any job done.
Give this a try against your sample message using the
Grok Debugger:
Code: Select all
Protocol: %{DATA:Protocol}\n.*CipherSuite: %{WORD:CipherSuite}\n.*Exchange strength: %{NUMBER:ExchangeStrength}
I get the following fields parsed out:
Code: Select all
{
"Protocol": [
[
"TLS 1.1"
]
],
"CipherSuite": [
[
"0xA"
]
],
"ExchangeStrength": [
[
"2048"
]
],
"BASE10NUM": [
[
"2048"
]
]
}
If "Exchange strength" is ever anything other than a numeric value, you might want to type it as a string to avoid problems in the future.
Also, when using that pattern in a Logstash configuration, you
might need to escape the backslashes. I can't remember and don't have a lab machine handy.
Re: Nagios Log Server - dissect logstash plugin
Posted: Fri Sep 01, 2017 9:07 am
by bpizzutiWHI
That looks like it should work, but what about parsing out client versus server handshakes?
Grok is the Swiss army knife of Logstash parsing. It's inefficient and unwieldy at times, but it can get pretty much any job done.
Yeah, I wasn't talking about whether Grok could do it, I was talking about whether I wasn cabable of doing it with Grok. It's...intimidating.
Re: Nagios Log Server - dissect logstash plugin
Posted: Fri Sep 01, 2017 9:16 am
by mcapra
(?m) is a special flag for handling multi-line input a bit more effectively.
How about this:
Code: Select all
(?m)An SSL %{WORD:HandshakeType} handshake.*Protocol: %{DATA:Protocol}\n.*CipherSuite: %{WORD:CipherSuite}\n.*Exchange strength: %{NUMBER:ExchangeStrength}
Which produces:
Code: Select all
{
"HandshakeType": [
[
"server"
]
],
"Protocol": [
[
"TLS 1.1"
]
],
"CipherSuite": [
[
"0xA"
]
],
"ExchangeStrength": [
[
"2048"
]
],
"BASE10NUM": [
[
"2048"
]
]
}
Re: Nagios Log Server - dissect logstash plugin
Posted: Fri Sep 01, 2017 9:23 am
by bpizzutiWHI
Ok, so that's exactly what I want. So if I'm understanding this correctly (big "if") I should be adding a filter as follows:
Code: Select all
if [SourceName] == "Schannel" {
grok {
match => {"message" => [ (?m)An SSL %{WORD:HandshakeType} handshake.*Protocol: %{DATA:Protocol}\n.*CipherSuite: %{WORD:CipherSuite}\n.*Exchange strength: %{NUMBER:ExchangeStrength} ]
}
}
}
}
Except that keeps coming back unable to verify:
Error: Expected one of #, => at line 187, column 52 (byte 4113) after
filter {
.
. (snip)
.
if [SourceName] == "Schannel" {
grok {
match => {"message" => [ {Protocol
If you want to see what's snipped out, let me know, but without the new block all of that verifies correctly.
Re: Nagios Log Server - dissect logstash plugin
Posted: Fri Sep 01, 2017 9:27 am
by mcapra
If you were storing it as a single filter "configuration" in Nagios Log Server, it would look like this:
Code: Select all
if [SourceName] == "Schannel" {
grok {
match => { "message" => "(?m)An SSL %{WORD:HandshakeType} handshake.*Protocol: %{DATA:Protocol}\n.*CipherSuite: %{WORD:CipherSuite}\n.*Exchange strength: %{NUMBER:ExchangeStrength}" }
}
}
Re: Nagios Log Server - dissect logstash plugin
Posted: Fri Sep 01, 2017 9:32 am
by bpizzutiWHI
AHHH...I forgot a set of quotation marks. Doh! Thanks!
Re: Nagios Log Server - dissect logstash plugin
Posted: Fri Sep 01, 2017 9:36 am
by bpizzutiWHI
Ok, i speaketh too soon, got a _grokparsefailure in my logs now.