Event Log and Grok Filtering

[CameronWP]

Hello:

I am kind of pulling my hair out trying to get this to work. The message appears as follows:

Remote Desktop Services: Session logon succeeded:

User: domain.com\username
Session ID: 142
Source Network Address: 10.10.10.1

I have tried a few different grok filters and KV filters but nothing I try is working. Can I get some advice on how to extract that message into fields removing the first line and the space as well?

Thanks!

[cdienger]

Is the event going to the default event log input on 3515 in NLS? As long as the event is in JSON fomat it should be parsing fields automatically. I'd like to get a copy of the raw event to look into this further - you can do this by editing the nslog.conf file, uncomment the line:

#Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");

and then restart the nxlog service. The raw events will then be written to nxlog_output.log.

[jjeremydiaz]

Hi CameronWP,

Groks are essentially in this format:

grok {
match => {
"field_to_parse" => [
"%{new_field_name:built_in_parser_value}"
]
}
}

Custom Patterns can be created using this syntax:

(?<field_name>pattern)
In your particular case we may want something like this:

grok {
match => {
"message" => [
"User: (?<User>.*)\nSession ID: (?<Session_ID>[0-9]+)\nSource Network Address: %{IP:ip_address}"
]
}
}

Make sure there are no additional spaces or tabs, otherwise you will need to add them after the newlines.

[CameronWP]

Is the event going to the default event log input on 3515 in NLS? As long as the event is in JSON fomat it should be parsing fields automatically. I'd like to get a copy of the raw event to look into this further - you can do this by editing the nslog.conf file, uncomment the line:

#Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");

and then restart the nxlog service. The raw events will then be written to nxlog_output.log.

Hi, thanks for the reply! Here is the raw event:

{"EventTime":"2018-08-02 08:40:12","Hostname":"WPREMOTE02.mhcp.on.ca","Keywords":1152921504606846976,"EventType":"INFO","SeverityValue":2,"Severity":"INFO","EventID":21,"SourceName":"Microsoft-Windows-TerminalServices-LocalSessionManager","ProviderGuid":"{5D896912-022D-40AA-A3A8-4FA5515C76D7}","Version":0,"Task":0,"OpcodeValue":0,"RecordNumber":40791,"ActivityID":"{F420AFBB-805E-462F-9CDF-F4677DDA0000}","ProcessID":908,"ThreadID":13300,"Channel":"Microsoft-Windows-TerminalServices-LocalSessionManager/Operational","Domain":"NT AUTHORITY","AccountName":"SYSTEM","UserID":"S-1-5-18","AccountType":"User","Opcode":"Info","EventReceivedTime":"2018-08-02 08:40:13","SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog","message":"Remote Desktop Services: Session logon succeeded:\r\n\r\nUser: MHCP.ON.CA\\cmrprime\r\nSession ID: 152\r\nSource Network Address: 10.1.23.69"}

Thanks!

[cdienger]

I haven't had a chance to feed this through NLS yet, but the first step I always take is finding a proper grok filter with http://grokdebug.herokuapp.com/. Using that I came up with the following to split the message field:

^%{GREEDYDATA:firstlines}:\\r\\n\\r\\nUser: %{GREEDYDATA:user}\\r\\nSession ID: %{NUMBER:session_id}\\r\\nSource Network Address: %{IP:source_network_address}

[CameronWP]

I haven't had a chance to feed this through NLS yet, but the first step I always take is finding a proper grok filter with http://grokdebug.herokuapp.com/. Using that I came up with the following to split the message field:

^%{GREEDYDATA:firstlines}:\\r\\n\\r\\nUser: %{GREEDYDATA:user}\\r\\nSession ID: %{NUMBER:session_id}\\r\\nSource Network Address: %{IP:source_network_address}

Thanks for this! I now have the following as a filter:

if [type] == 'eventlog' {
if [EventID] == 21 {
grok {
match => {
"message" => [ "^%{GREEDYDATA:firstlines}:\\r\\n\\r\\nUser: %{GREEDYDATA:user}\\r\\nSession ID: %{NUMBER:session_id}\\r\\nSource Network Address: %{IP:source_network_address}"
]
}
}
}
}

But I am receiving a _grokparse error. I have been looking but haven't been able to find logging for the logstash to determine what it is choking on. I will keep testing but if you happen to have any suggestions it would be helpful, I have learned quite a bit from this exercise as it is. Thanks!

[CameronWP]

Thanks for all of your help! Here is the working filter I ended up with:

if [type] == 'eventlog' {
if [EventID] == 21 {
grok {
match => {
'message' => '(?m) *User: %{GREEDYDATA:user} *Session ID: %{DATA:session_id} *Source Network Address: %{IP:source_network_address}'
}
}
}
}

Thanks again!

[scottwilkerson]

Thanks for sharing!

Locking