Nagios Support Forum

Posted: **Fri Sep 01, 2017 9:04 am**

mcapra, scottwilkerson, and cdienger,

First, thank you very much for your assistance and patience. I apologize for the delay in posting this message. I have been out of town for some time.

Second, I discovered a couple of things while troubleshooting this issue and implementing the suggestions you provided me.

I missed two sets of \" \" in my GROK statement I posted last. These generated the GROK error.
After fixing these issues I had at least six variables with duplicate values in their tags. Some were parsed by GROK. Some were parsed by KV.
1. ap AP221U,AP221U
2. logtime 1502908275,1502908275
3. signal -95,-95
4. radioband 802.11n,802.11n
After implementing scottwilkerson's suggestion and reviewing more information and examples on the Internet, I discovered the reuse of the variable "message" between GROK and KV caused the duplication of values. One suggestion was to use a different variable name on each out subsequent function. Therefore, I used the new variable "message_remainder" in the GREEDYDATA function output. I had noticed duplicate values appear in the GROKDEBUGGER output when testing,
but didn't make the correlation between what was happening in memory (please correct me if I am wrong as I am often. ).

Here is my latest, working filter. This includes a working GEOIP filter.

Code: Select all

if [type] == 'FortiLog' {
              grok {
                    match => [ 'message' , "\<%{DATA:grokpri}\>date=%{DATA:grokdate} time=%{TIME:groktime} %{GREEDYDATA:message_remainder}" ]
                    tag_on_failure => [ 'failure_grok_fortiOS' ]
                    }
   mutate {
          gsub => [ 'message_remainder','^<[0-9]*>','' ]
          }
        kv {
            source => 'message_remainder'
            }
            geoip {
                   source => 'dstip'
                   }
                      mutate {
                             remove_field => [ 'message_remainder' ]
                             }
                         }

Here is a couple screenshots of my main dashboard and a shot of a successfully parsed Fortinet message for you to review.
https://mega.nz/#!HsRDxQrS!NtxTwT_hLUkI ... v-K8zhYA3g
https://mega.nz/#!6xZTVRBb!rWIXXhCCOmYU ... JhqbIAQY7o

Again, I apologize for the delay in responding to your assistance, but I hope this information helps someone who is going through the same issues I have gone through.

Thank you for all your help.

Rodney.

Posted: **Fri Sep 01, 2017 1:27 pm**

Thanks for the update and I think I can speak for the others when I say we don't mind the delay - especially since you came back with good news : ) Are you okay with us locking the thread at this point?

Posted: **Fri Sep 01, 2017 9:01 pm**

Nagios Support Forum

FortiOS 5.6 GrokParseFailure

Re: FortiOS 5.6 GrokParseFailure

Re: FortiOS 5.6 GrokParseFailure

Re: FortiOS 5.6 GrokParseFailure