Nagios Support Forum

Hello:

I have looked around at the forum and Google in general but can't wrap my head around creating an input filter for a Mcafee EPO syslog message. It is currently arriving with the following as the message field (some fields altered for public):

<29>1 2017-08-14T02:58:27.0Z EPOSVR1 EPOEvents - EventFwd [agentInfo@3401 tenantId="1"] <?xml version="1.0" encoding="UTF-8"?><EE_Event><MachineInfo><MachineName>Computer_Name</MachineName><AgentGUID>{942a970f-c4ed-424a-ae94-b573676a0d6d}</AgentGUID><IPAddress>10.10.10.12</IPAddress><OSName>Windows 7</OSName><UserName>SYSTEM</UserName><TimeZoneBias>240</TimeZoneBias><RawMACAddress>94659cb1e8fb</RawMACAddress></MachineInfo><EventData ProductName="Drive Encryption" ProductVersion="7.2.1.24" ProductFamily="MCAFEE_EED"><EventNode><EventID>30030</EventID><Severity>0</Severity><GMTTime>2017-08-14T14:57:45</GMTTime><Data><?xml version="1.0" encoding="UTF-8"?> <ESAuditLogItemList xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:ns1="ns1" xsi:type="ns1:ESAuditLogItemList"> <maxEntries>1000</maxEntries> <audits xsi:type="ns1:ESAuditLogItem"> <id>0</id> <type>0</type> <event>30113</event> <timestamp xsi:type="ns1:MfeEpeTimestamp"> <milliseconds>13147192837657</milliseconds> </timestamp> <parameters xsi:type="ns1:ESAuditLogItemParameter"> <name>userUuid</name> <value>mrobinson</value> </parameters> </audits> </ESAuditLogItemList></Data><DataType>ESAuditLogItemList</DataType></EventNode></EventData></EE_Event>

Can I get some advice on converting that message into something helpful? Thanks!

Use a mutate filter to strip this part out:
And you can store it in it's own field or just drop it entirely, then pass the resulting message through an xml filter. Example:
This filter rule would first replace everything in message that matches the <.*]\s regular expression, then parse the rest of the message as XML. This seems to work based on my tests on regexr (I am working with a small sample set; The regex may need to be refined):
http://regexr.com/3gi64

So as part of the mutate step, everything preceding the XML object in the message gets dropped. Then we can just apply a regular old xml filter against the resulting message field and get nice, neat individual fields (hopefully).

Thanks for the assist, mcapra!

CameronWP, let us know once you've had a chance to test this suggestion.

Thanks for that response! I am having a bit of a problem implementing it though as it seems when the filter is applied, I don't get any messages in the log at all. If I remove the XML portion of the filter the messages appear with the front of the message removed as per the GSUB command but stiull in that raw XML format. If I disable the filter they show up as before of course. Sorry if this is a basic question, I am new to the filtering aspect of this application. I currently have things set as:

Input:

tcp {
type => 'mcafee'
port => 6514
ssl_cacert => "/etc/pki/tls/certs/rootCA.pem"
ssl_cert => "/etc/pki/tls/certs/device-nls.crt"
ssl_key => "/etc/pki/tls/private/device-nls.key"
ssl_enable => true
ssl_verify => false
}

Filter:

if [type] == 'mcafee' {
mutate {
gsub => [
'message', '<.*]\s', ''
]
}
}

xml {
source => 'message'
}

Your xml step is outside the mcafee logic. This means the xml step is applied to all messages; no good. Make sure the xml step is included in your mcafee logic: I also modified the regex a bit to match the head of the message a bit better. See if those changes get the logs flowing, and if they don't share a copy of your Logstash log. It can typically be found here:

Thanks for the response! Things are flowing again. I will need to do some experimentation around the parsing of the XML but I think I am on the right track. Thanks!

You might also need to strip out the <?xml version="1.0" encoding="UTF-8"?> section from the message. logstash-filter-xml is a bit brittle from what I remember. The Logstash log may have some hints if it's encountering parsing errors, at any rate.

Thanks for the update. Keep us in the loop with any questions or issues that may pop up.

Hi:

I am finally getting back to this and working towards a solution. I have the XML parsing from the message as expected into new fields that contain information in a JSON string. I am trying to pass that resultant field into a JSON filter but it isn't working as expected. Here is my filter:
Here is the result in Logserver with that filter in place:

We seem to have enough data here to try and reproduce it. I'll be able to look more into it tomorrow and will get back to you.