Page 1 of 2
Regex to filter out last column of a log
Posted: Mon Feb 24, 2020 12:51 pm
by tvoll
I am wanting to filter out the last column of a log (in this case, the one with the number 3.20) to only show results 75.00 and higher.
Here is what the field i'm trying to parse out looks like:
<133>Feb 24 11:44:03 HostDBP01 HostDBP: sda 0.00 40.00 0.00 170.00 0.00 7.60 91.58 0.09 0.52 0.00 0.52 0.19 3.20
Attached is a photo of my current filter/regex that gets close sometimes, but still checks every single column (instead of just the last one).
Is this possible with Nagios Log Server?
Re: Regex to filter out last column of a log
Posted: Mon Feb 24, 2020 1:52 pm
by scottwilkerson
Are they all 13 number sets long?
maybe something line this
Code: Select all
(([0-9]+.[0-9]{2} ){12}[8-9]+.[0-9]{2})
This would match anything for the first 12 sets and then 80+ for the last set
Re: Regex to filter out last column of a log
Posted: Mon Feb 24, 2020 2:09 pm
by tvoll
scottwilkerson wrote:Are they all 13 number sets long?
maybe something line this
Code: Select all
(([0-9]+.[0-9]{2} ){12}[8-9]+.[0-9]{2})
This would match anything for the first 12 sets and then 80+ for the last set
Yes, they are all 13 number sets long.
Is that a new regex that you think I should add on to the existing one that I have? Or just replace it? If I replace it in Nagios Log Server, I get 0 results, just a constant loading. I know for fact that there are results in the range that should appear.
Re: Regex to filter out last column of a log
Posted: Mon Feb 24, 2020 3:25 pm
by scottwilkerson
This was just a guess, but I do see that I made a mistake
try this
Code: Select all
(([0-9]+.[0-9]{2} ){12}[8-9]+[0-9].[0-9]{2})
I don't have any data like this so I'm really just throwing something out there based on this doc
https://www.elastic.co/guide/en/elastic ... query.html
Re: Regex to filter out last column of a log
Posted: Mon Feb 24, 2020 3:37 pm
by tvoll
Still no go.

Re: Regex to filter out last column of a log
Posted: Tue Feb 25, 2020 2:17 pm
by scottwilkerson
I have tried everything I could think of and the more I try, the more I believe it isn't possible without breaking the messages up when they come in with a grok filter and placing each of the values in their own fields.
https://assets.nagios.com/downloads/nag ... ilters.pdf
Re: Regex to filter out last column of a log
Posted: Wed Feb 26, 2020 2:20 pm
by tvoll
Alright, that might be a better option.
Could you give me an idea of what grok pattern/filter input would be best to use to accomplish this issue?
Re: Regex to filter out last column of a log
Posted: Wed Feb 26, 2020 3:24 pm
by tvoll
Code: Select all
if [host] == '0.0.0.0' {
grok {
match => [ 'message', '<%{INT}>%{MONTH} %{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND} %{WORD} %{WORD}: %{WORD} %{BASE10NUM} %{BASE10NUM} %{BASE10NUM} %{BASE10NUM} %{BASE10NUM} %{BASE10NUM} %{BASE10NUM} %{BASE10NUM} %{BASE10NUM} %{BASE10NUM} %{BASE10NUM} %{BASE10NUM} %{BASE10NUM:IOUTIL}' ]
overwrite => [ 'message' ]
}
}
Getting a Grok Parse Failure after using this grok pattern within the filter.
The IP 0.0.0.0 is replaced with the actual valid ip.
Re: Regex to filter out last column of a log
Posted: Wed Feb 26, 2020 5:57 pm
by scottwilkerson
Using the
http://grokdebug.herokuapp.com/
and this line for reference
Code: Select all
<133>Feb 24 11:44:03 HostDBP01 HostDBP: sda 0.00 40.00 0.00 170.00 0.00 7.60 91.58 0.09 0.52 0.00 0.52 0.19 3.20
I came up with this grok filter
Code: Select all
%{SYSLOG5424PRI}%{SYSLOGBASE} %{WORD:sda} %{BASE16FLOAT:val1} %{BASE16FLOAT:val2} %{BASE16FLOAT:val3} %{BASE16FLOAT:val4} %{BASE16FLOAT:val5} %{BASE16FLOAT:val6} %{BASE16FLOAT:val7} %{BASE16FLOAT:val8} %{BASE16FLOAT:val9} %{BASE16FLOAT:val10} %{BASE16FLOAT:val11} %{BASE16FLOAT:val12} %{BASE16FLOAT:val13}
This would split the fields into val1 to val13
then you can just look at val13
Re: Regex to filter out last column of a log
Posted: Thu Feb 27, 2020 9:57 am
by tvoll
scottwilkerson wrote:Using the
http://grokdebug.herokuapp.com/
and this line for reference
Code: Select all
<133>Feb 24 11:44:03 HostDBP01 HostDBP: sda 0.00 40.00 0.00 170.00 0.00 7.60 91.58 0.09 0.52 0.00 0.52 0.19 3.20
I came up with this grok filter
Code: Select all
%{SYSLOG5424PRI}%{SYSLOGBASE} %{WORD:sda} %{BASE16FLOAT:val1} %{BASE16FLOAT:val2} %{BASE16FLOAT:val3} %{BASE16FLOAT:val4} %{BASE16FLOAT:val5} %{BASE16FLOAT:val6} %{BASE16FLOAT:val7} %{BASE16FLOAT:val8} %{BASE16FLOAT:val9} %{BASE16FLOAT:val10} %{BASE16FLOAT:val11} %{BASE16FLOAT:val12} %{BASE16FLOAT:val13}
This would split the fields into val1 to val13
then you can just look at val13
That's great, but I can't just throw that Grok Filter into Nagios Log Server (or can I?), I have to script it out like I did above. How would that be done? If it were that simple, I feel like my example above wouldn't have had issues with it.