Page 1 of 1
Filtering out white noise
Posted: Wed Jul 26, 2017 10:25 am
by bricehutch
I have a lot of white noise going on in my events, to the tune of 500k a day from this one source that I would liketo filter out. I'm trying the following to filter it out but it doesn't seem to be working. Not sure if TargetUserNane is the correct way to call it in the filter, but that's what it shows up as in the parsed out event.
eventlog being the name of the input for my windows event logs;
tcp {
type => 'eventlog'
[...]
}
then for the filter;
if [type] == 'eventlog' {
filter {
if [TargetUserName] == "username1" {
if [TargetUserName] == "unsername2" {
drop {}
}
}
}
}
Re: Filtering out white noise
Posted: Wed Jul 26, 2017 11:27 am
by cdienger
You can remove the "filter" action if you're adding this via the web UI under Administration > Global Configuration > Filters(recommend) and simplify/correct the username check by using the OR operator. Try:
Code: Select all
if [type] == 'eventlog' {
if [TargetUserName] == "username1" or [TargetUserName] == "unsername2" {
drop {}
}
}
Re: Filtering out white noise
Posted: Wed Jul 26, 2017 1:00 pm
by bricehutch
So that almost worked. I can use that format to filter other fields
Code: Select all
if [type] == 'eventlog' {
if [Severity] == 'INFO' and [Workstation] == 'xxxxxxxx' {
drop {}
}
}
This drops anything with severity INFO from one machine that I don't care about seeing. I confirmed that I can filter on items like severity, workstation, Event ID, ProcessID, and other
but when I replace
with
it just doesn't want to filter. I even tried TargetUserName.raw and nothing. Can't get it to filter away usernames.
Re: Filtering out white noise
Posted: Wed Jul 26, 2017 3:59 pm
by cdienger
Interesting. I believe the filters are case sensitive unless you use regex. Double check for this. Also, does the username contain any special characters like a \ ? You may need to escape those. Try \\.
Failing any of that, I'd like to see a screenshot showing the parsed field as well as the filters found in /usr/local/nagioslogserver/logstash/etc/conf.d/.
Re: Filtering out white noise
Posted: Thu Jul 27, 2017 10:06 am
by bricehutch
filters conf below, screen shot of parsed field attached. I've also tried this config with TargetUserName.raw to the same result.
Code: Select all
[root@PC5-NXLOG conf.d]# cat 500_filters.conf
#
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Thu, 27 Jul 2017 10:54:18 -0400
#
#
# Global filters
#
filter {
if [program] == 'apache_access' {
grok {
match => [ 'message', '%{COMBINEDAPACHELOG}']
}
date {
match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z', 'MMM dd HH:mm:ss', 'ISO8601' ]
}
mutate {
replace => [ 'type', 'apache_access' ]
convert => [ 'bytes', 'integer' ]
convert => [ 'response', 'integer' ]
}
}
if [program] == 'apache_error' {
grok {
match => [ 'message', '\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\] \[%{WORD:class}\] \[%{WORD:originator} %{IP:clientip}\] %{GREEDYDATA:errmsg}']
}
mutate {
replace => [ 'type', 'apache_error' ]
}
}
if [type] == 'eventlog' {
if [TargetUserName] == 'ngxiadmin' {
drop {}
}
}
}
#
# Local filters
#
But I can run this filter in place of the one I added, and it works fine.
Code: Select all
if [type] == 'eventlog' {
if [Severity] == 'INFO' and [Workstation] == 'PC5-NGXI' {
drop {}
}
Re: Filtering out white noise
Posted: Thu Jul 27, 2017 12:31 pm
by cdienger
Click the little magnifying glass seen in the screenshot next to TargetUserName. This will create a new filter you can see at the top of the dashboard and sometimes we see hidden characters(like \n) with this. The filter looks good so I wonder if there's something we're not seeing in the entry that makes it not match. I can try setting this up and reproduce - which eventlog is this field seen in and how is the event generated?
Re: Filtering out white noise
Posted: Thu Jul 27, 2017 2:06 pm
by bricehutch
Does not appear to be any special characters when i query in the UI.
The event is from the security logs on Windows domain controllers, all of the DCs i'm using as inputs are generating these. event ID 4634 - an account was logged off.
ngxiadmin is the account that my NGXI instance uses to authenticate to machines that get monitored over WMI. So with how frequently, and with intervals that make sense, it looks like these events are caused by Nagios trying to poll the server, or authenticate LDAP. The other side of the event, the logon side, I was able to filter out by dropping events by the machine name, but these ones only have the username in them.
Re: Filtering out white noise
Posted: Thu Jul 27, 2017 4:14 pm
by cdienger
I've been able to set this up but cannot reproduce it. Go ahead and gather a profile(Administration > System > System Status), send it to
[email protected], and we'll get a ticket started to have a closer look.