Optional Grok Fields

[aer9480]

Hello, I am having an issue when I try to parse a logstash log that has an optional field. The optional field is called installer_name, but when I try

Code: Select all

(installer_name=%{QS:Bit9_installer_exe})? 

it doesn't match. I am using http://grokconstructor.appspot.com/do/match#result to test my pattern, and it doesn't get past the installer_name. I have other fields after this that need to be parsed, but it's not letting me get past the installer_name field.

My log looks similar to this:

Code: Select all

<13>May 04 12:40:31 Bit9.xxxx.com Bit9 event:  text="File 'c:\windows\temp\low\skypeclicktocall\download\skypetoolbars.msi' [8fb885c633a92a5391b9493afff529b12bdd90cbcbe15045612711280ad4b027] was blocked because it was unapproved." type="Policy Enforcement" subtype="Execution block (unapproved file)" hostname="523-AAPALMER-LT" username="NT AUTHORITY\SYSTEM" date="5/4/2016 12:40:21 PM" ip_address="192.168.1.1" process="c:\windows\system32\msiexec.exe" file_path="c:\windows\temp\low\skypeclicktocall\download\skypetoolbars.msi" file_name="skypetoolbars.msi" file_hash="8fb885c633a92a5391b9493afff529b12bdd90cbcbe15045612711280ad4b027" installer_name="skypec2cautoupdatesvc.exe" policy="MPA-High" rule_name="Block unapproved scripts" process_key="00000178-0000-1954-01d1-a6021ef1126c" server_version="7.2.1.1562" file_trust="-1" file_threat="-1" process_trust="10" process_threat="0"

Not all of them have installer_name.

Any ideas? Thanks!

[hsmith]

Can you post the full grok filter here? I have a feeling the issue might be coming from a spacing issue.

[aer9480]

Code: Select all

<%{POSINT:SysLogPri}>%{CISCOTIMESTAMP} %{GREEDYDATA:Bit9_Server} Bit9 event: +text=%{QS:Bit9_text} type=%{QS:Bit9_Type} subtype=%{QS:Bit9_subtype} hostname=%{QS:Bit9_Hostname} username=%{QS:Bit9_User} date=%{QS:Bit9_Date} ip_address=%{QS:Bit9_client_ip} process=%{QS:Bit9_Process} file_path=%{QS:Bit9_Path} file_name=%{QS:Bit9_File} file_hash=%{QS:Bit9_hash} (installer_name=%{QS:Bit9_installer_exe})? policy=%{QS:Bit9_Policy} rule_name=%{QS:Bit9_rule_name} process_key=%{QS:Bit9_proc_key} server_version=%{QS:Bit9_server_ver} file_trust=%{QS:Bit9_file_trust} file_threat=%{QS:Bit9_file_threat} process_trust=%{QS:Bit9_process_trust} process_threat=%{QS:Bit9_process_threat}

I changed the GREEDYDATA to Quoted String like you advised last time. I also changed a few things around because of our log formats. I remembered you said to use ()? for optional fields, but I cant seem to get it working. Thanks!

[hsmith]

Try removing the space between {QS:Bit9_hash} and (installer_name=%{QS:Bit9_installer_exe})?

Reason for this it's looking for two spaces now if installer_name is not there. One after Bit9_hash and another before policy. Do you see where I'm coming from?

[aer9480]

Worked perfectly. I didn't even think of that. Thanks buddy!

[hsmith]

Glad to hear it! Should I go ahead and close the thread?

[aer9480]

Sure. Should be all set.