Nagios Support Forum

I'm not having much luck with the json filter either, but the xml filter is behaving much better:

Hi:

So it appears that it is by design that logstash doesn't parse any fields with a period in it. There is a new filter called de_dot that removes the period from field names that might make things work.

De_Dot announcement: https://www.elastic.co/guide/en/logstas ... e_dot.html
Period Issue: https://www.elastic.co/guide/en/elastic ... names.html
Another Article: https://discuss.elastic.co/t/field-name ... tain/33251

I have an insanely long filter parsing with XML and it kind of works but it would be nice to make it simpler. I am playing with a bit of Ruby code and will post what I get working. Thanks!

Thanks for the update! Look forward to anything you come up with!

Hi all:

Here is the finished version I came up with that is parsing things reasonably well:

if [type] == 'mcafee' {
mutate {
gsub => [
'message', '^<.*\?>', ''
]
}

if [message] =~ /^<EE_Event>/ {
xml {
source => 'message'
store_xml => false
xpath => [ "EE_Event/MachineInfo", "HostDetails" ]
xpath => [ "EE_Event/EventData", "EventDetails" ]
}

xml {
source => HostDetails
target => MachineDetails
}
xml {
source => EventDetails
target => Event
}
}

if [message] =~ /^<DLPAGENT9400>/ {
xml {
source => 'message'
store_xml => false
xpath => [ "DLPAGENT9400/MachineInfo", "HostDetails" ]
xpath => [ "DLPAGENT9400/EventList", "EventDetails" ]
}
xml {
source => HostDetails
target => MachineDetails
}
xml {
source => EventDetails
target => Event
}
}

if [message] =~ /^<BehaviourBlockEvent>/ {
xml {
source => 'message'
store_xml => false
xpath => [ "BehaviourBlockEvent/MachineInfo", "HostDetails" ]
xpath => [ "BehaviourBlockEvent/ScannerSoftware", "EventDetails" ]
}
xml {
source => HostDetails
target => MachineDetails
}
xml {
source => EventDetails
target => Event
}
}

if [message] =~ /^<UpdateEvents>/ {
xml {
source => 'message'
store_xml => false
xpath => [ "UpdateEvents/MachineInfo", "HostDetails" ]
xpath => [ "UpdateEvents/McAfeeCommonUpdater", "EventDetails" ]
}
xml {
source => HostDetails
target => MachineDetails
}
xml {
source => EventDetails
target => Event
}
}

if [message] =~ /^<DLPAGENT9300>/ {
xml {
source => 'message'
store_xml => false
xpath => [ "DLPAGENT9300/MachineInfo", "HostDetails" ]
xpath => [ "DLPAGENT9300/EventList", "EventDetails" ]
}
xml {
source => HostDetails
target => MachineDetails
}
xml {
source => EventDetails
target => Event
}

}

if [message] =~ /^<EPO[Ee]vent>/ {
xml {
source => 'message'
store_xml => false
xpath => [ "EPOevent/MachineInfo", "HostDetails" ]
xpath => [ "EPOevent/SoftwareInfo", "EventDetails" ]
xpath => [ "EPOevent/SoftwareInfo/Event/CommonFields", "CommonDescription" ]
xpath => [ "EPOevent/SoftwareInfo/CommonFields", "CommonDescription" ]
}
xml {
source => HostDetails
target => MachineDetails
}
xml {
source => EventDetails
target => Event
}
xml {
source => CommonDescription
target => Common
}
if [message] =~ /<CustomFields/ {
xml {
source => 'message'
store_xml => false
xpath => [ "EPOevent/SoftwareInfo/Event/CustomFields", "CustomDescription" ]
}
}
xml {
source => CustomDescription
target => Custom
}
}
mutate {
remove_field => [ "message" ]
remove_field => [ "HostDetails" ]
remove_field => [ "EventDetails" ]
}
}

Thanks for sharing!

Did you have any more questions or are we okay to lock this up?

I am good, the parser seems to be doing what it is supposed to. Thanks!

Sounds good! I'll be closing this thread!

If you have any more questions, feel free to create another thread.

Thanks for using the Nagios Support Forum!