Page 1 of 1
Event Log and Grok Filtering
Posted: Wed Aug 01, 2018 12:38 pm
by CameronWP
Hello:
I am kind of pulling my hair out trying to get this to work. The message appears as follows:
Remote Desktop Services: Session logon succeeded:
User: domain.com\username
Session ID: 142
Source Network Address: 10.10.10.1
I have tried a few different grok filters and KV filters but nothing I try is working. Can I get some advice on how to extract that message into fields removing the first line and the space as well?
Thanks!
Re: Event Log and Grok Filtering
Posted: Wed Aug 01, 2018 3:05 pm
by cdienger
Is the event going to the default event log input on 3515 in NLS? As long as the event is in JSON fomat it should be parsing fields automatically. I'd like to get a copy of the raw event to look into this further - you can do this by editing the nslog.conf file, uncomment the line:
#Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
and then restart the nxlog service. The raw events will then be written to nxlog_output.log.
Re: Event Log and Grok Filtering
Posted: Wed Aug 01, 2018 3:14 pm
by jjeremydiaz
Hi CameronWP,
Groks are essentially in this format:
grok {
match => {
"field_to_parse" => [
"%{new_field_name:built_in_parser_value}"
]
}
}
Custom Patterns can be created using this syntax:
(?<field_name>pattern)
In your particular case we may want something like this:
grok {
match => {
"message" => [
"User: (?<User>.*)\nSession ID: (?<Session_ID>[0-9]+)\nSource Network Address: %{IP:ip_address}"
]
}
}
Make sure there are no additional spaces or tabs, otherwise you will need to add them after the newlines.
Re: Event Log and Grok Filtering
Posted: Thu Aug 02, 2018 7:43 am
by CameronWP
cdienger wrote:Is the event going to the default event log input on 3515 in NLS? As long as the event is in JSON fomat it should be parsing fields automatically. I'd like to get a copy of the raw event to look into this further - you can do this by editing the nslog.conf file, uncomment the line:
#Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
and then restart the nxlog service. The raw events will then be written to nxlog_output.log.
Hi, thanks for the reply! Here is the raw event:
{"EventTime":"2018-08-02 08:40:12","Hostname":"WPREMOTE02.mhcp.on.ca","Keywords":1152921504606846976,"EventType":"INFO","SeverityValue":2,"Severity":"INFO","EventID":21,"SourceName":"Microsoft-Windows-TerminalServices-LocalSessionManager","ProviderGuid":"{5D896912-022D-40AA-A3A8-4FA5515C76D7}","Version":0,"Task":0,"OpcodeValue":0,"RecordNumber":40791,"ActivityID":"{F420AFBB-805E-462F-9CDF-F4677DDA0000}","ProcessID":908,"ThreadID":13300,"Channel":"Microsoft-Windows-TerminalServices-LocalSessionManager/Operational","Domain":"NT AUTHORITY","AccountName":"SYSTEM","UserID":"S-1-5-18","AccountType":"User","Opcode":"Info","EventReceivedTime":"2018-08-02 08:40:13","SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog","message":"Remote Desktop Services: Session logon succeeded:\r\n\r\nUser: MHCP.ON.CA\\cmrprime\r\nSession ID: 152\r\nSource Network Address: 10.1.23.69"}
Thanks!
Re: Event Log and Grok Filtering
Posted: Thu Aug 02, 2018 9:52 am
by cdienger
I haven't had a chance to feed this through NLS yet, but the first step I always take is finding a proper grok filter with
http://grokdebug.herokuapp.com/. Using that I came up with the following to split the message field:
^%{GREEDYDATA:firstlines}:\\r\\n\\r\\nUser: %{GREEDYDATA:user}\\r\\nSession ID: %{NUMBER:session_id}\\r\\nSource Network Address: %{IP:source_network_address}
Re: Event Log and Grok Filtering
Posted: Thu Aug 02, 2018 10:06 am
by CameronWP
cdienger wrote:I haven't had a chance to feed this through NLS yet, but the first step I always take is finding a proper grok filter with
http://grokdebug.herokuapp.com/. Using that I came up with the following to split the message field:
^%{GREEDYDATA:firstlines}:\\r\\n\\r\\nUser: %{GREEDYDATA:user}\\r\\nSession ID: %{NUMBER:session_id}\\r\\nSource Network Address: %{IP:source_network_address}
Thanks for this! I now have the following as a filter:
if [type] == 'eventlog' {
if [EventID] == 21 {
grok {
match => {
"message" => [ "^%{GREEDYDATA:firstlines}:\\r\\n\\r\\nUser: %{GREEDYDATA:user}\\r\\nSession ID: %{NUMBER:session_id}\\r\\nSource Network Address: %{IP:source_network_address}"
]
}
}
}
}
But I am receiving a _grokparse error. I have been looking but haven't been able to find logging for the logstash to determine what it is choking on. I will keep testing but if you happen to have any suggestions it would be helpful, I have learned quite a bit from this exercise as it is. Thanks!
Re: Event Log and Grok Filtering
Posted: Thu Aug 02, 2018 11:04 am
by CameronWP
Thanks for all of your help! Here is the working filter I ended up with:
if [type] == 'eventlog' {
if [EventID] == 21 {
grok {
match => {
'message' => '(?m) *User: %{GREEDYDATA:user} *Session ID: %{DATA:session_id} *Source Network Address: %{IP:source_network_address}'
}
}
}
}
Thanks again!
Re: Event Log and Grok Filtering
Posted: Thu Aug 02, 2018 4:21 pm
by scottwilkerson
Thanks for sharing!
Locking