grok whitespace strip not working

[gsl_ops_practice]

Hello,

We have input data that is space-delimited and I created a filter to parse it. All fields get populated as I need, except if there I have a 6-character field that contains a numeric value that takes up less than 6 characters. I need to graph these values over time but the leading white spaces are causing NagiosLog to throw an error about an incorrect data type.

Example- the APTIME and the BKTIME fields have only 53 and 389 respectively, but the actual field width as specified is 6, per the filter:

Code: Select all

(?<APTIME>.{6}) (?<BKTIME>.{6})

After looking at LogStash documentation there was a strip feature that should remove the leading whitespaces, but this did not happen, this is the filter:

Code: Select all

if [type] == 'appv2' {

	mutate {
		gsub => ["message", "[\u0000]", ""]
	}

grok {
      match => [ "message", "[\u0000]", "%{TIMESTAMP_ISO8601:logdate} (?<MDNIT>.{8}) (?<V>.{1}) (?<TRANCODE>.{12}) (?<FIELD1>.{2})  (?<FIELD2>.{2})  (?<FIELD3>.{2}) (?<FIELD4>.{6}) (?<PROCESS>.{6}) (?<APTIME>.{6}) (?<BKTIME>.{6}) (?<FIELD5>.{6}) (?<CODES>.{8}) (?<FIELD6>.{2}) (?<MC>.{2}) (?<PV>.{2}) (?<PC>.{2}) (?<PX>.{2}) (?<SYS>.{5}) (?<CT>.{2}) (?<RESULTS>.{9}) (?<FIELD7>.{13})"]
    }

mutate {
        strip => [
            "MDNIT",
            "APTIME",
            "PROCESS",
            "BKTIME",
            "FIELD5",
            "CODES"
            "FIELD7"
            "RESULTS"
                 ]
        }

date {
      match => [ "logdate", "YYYY-MM-dd HH:mm:ss.SSS" ]
  }
}

And here is a sample message:

Code: Select all

{"message": "2016-09-05 15:23:44.617 55424617 1 TRS123456789 XX  YY  21 345678    919    128    118    544          ZZ  2  N  P  1 APPLC 21 NNNNNNNNN KKKKKKKKKKKKk\n"}

As per the attachment, the APTIME and BKTIME numbers show leading white spaces after parsing but they should be fully left-justified in this example

[eloyd]

I suggest you play with the Grok Debugger at https://grokdebug.herokuapp.com to fine tune your grok match. I'm guessing that you're matching the space as part of the pattern somewhere but I didn't look closely at your filter.

[gsl_ops_practice]

I have seen this website before and have never been able to get any output except "compile error" regardless of the inputs I provided. Your continued assistance in resolving this would be appreciated.

[eloyd]

In the grok debugger, I used your sample message (minus the \n at the end) as the message:

Code: Select all

"2016-09-05 15:23:44.617 55424617 1 TRS123456789 XX  YY  21 345678    919    128    118    544          ZZ  2  N  P  1 APPLC 21 NNNNNNNNN KKKKKKKKKKKKk"

And your grok pattern as the pattern:

Code: Select all

"%{TIMESTAMP_ISO8601:logdate} (?<MDNIT>.{8}) (?<V>.{1}) (?<TRANCODE>.{12}) (?<FIELD1>.{2})  (?<FIELD2>.{2})  (?<FIELD3>.{2}) (?<FIELD4>.{6}) (?<PROCESS>.{6}) (?<APTIME>.{6}) (?<BKTIME>.{6}) (?<FIELD5>.{6}) (?<CODES>.{8}) (?<FIELD6>.{2}) (?<MC>.{2}) (?<PV>.{2}) (?<PC>.{2}) (?<PX>.{2}) (?<SYS>.{5}) (?<CT>.{2}) (?<RESULTS>.{9}) (?<FIELD7>.{13})"

Here are the results I got back:

Code: Select all

{
  "logdate": [
    [
      "2016-09-05 15:23:44.617"
    ]
  ],
  "YEAR": [
    [
      "2016"
    ]
  ],
  "MONTHNUM": [
    [
      "09"
    ]
  ],
  "MONTHDAY": [
    [
      "05"
    ]
  ],
  "HOUR": [
    [
      "15",
      null
    ]
  ],
  "MINUTE": [
    [
      "23",
      null
    ]
  ],
  "SECOND": [
    [
      "44.617"
    ]
  ],
  "ISO8601_TIMEZONE": [
    [
      null
    ]
  ],
  "MDNIT": [
    [
      "55424617"
    ]
  ],
  "V": [
    [
      "1"
    ]
  ],
  "TRANCODE": [
    [
      "TRS123456789"
    ]
  ],
  "FIELD1": [
    [
      "XX"
    ]
  ],
  "FIELD2": [
    [
      "YY"
    ]
  ],
  "FIELD3": [
    [
      "21"
    ]
  ],
  "FIELD4": [
    [
      "345678"
    ]
  ],
  "PROCESS": [
    [
      "   919"
    ]
  ],
  "APTIME": [
    [
      "   128"
    ]
  ],
  "BKTIME": [
    [
      "   118"
    ]
  ],
  "FIELD5": [
    [
      "   544"
    ]
  ],
  "CODES": [
    [
      "        "
    ]
  ],
  "FIELD6": [
    [
      "ZZ"
    ]
  ],
  "MC": [
    [
      " 2"
    ]
  ],
  "PV": [
    [
      " N"
    ]
  ],
  "PC": [
    [
      " P"
    ]
  ],
  "PX": [
    [
      " 1"
    ]
  ],
  "SYS": [
    [
      "APPLC"
    ]
  ],
  "CT": [
    [
      "21"
    ]
  ],
  "RESULTS": [
    [
      "NNNNNNNNN"
    ]
  ],
  "FIELD7": [
    [
      "KKKKKKKKKKKKk"
    ]
  ]
}

Since you're only putting a single space between fields, the extra spaces are considered part of the field. If you change single spaces to \s+ and change everything from:

Code: Select all

(?<PROCESS>.{6})

to

Code: Select all

%{INT:PROCESS}

Then it will work fine. I've included a sample output below, having changed just the PROCESS field. The rest is left as an exercise for you.

Code: Select all

"%{TIMESTAMP_ISO8601:logdate} (?<MDNIT>.{8}) (?<V>.{1}) (?<TRANCODE>.{12}) (?<FIELD1>.{2})  (?<FIELD2>.{2})  (?<FIELD3>.{2}) (?<FIELD4>.{6})\s+%{INT:PROCESS} (?<APTIME>.{6}) (?<BKTIME>.{6}) (?<FIELD5>.{6}) (?<CODES>.{8}) (?<FIELD6>.{2}) (?<MC>.{2}) (?<PV>.{2}) (?<PC>.{2}) (?<PX>.{2}) (?<SYS>.{5}) (?<CT>.{2}) (?<RESULTS>.{9}) (?<FIELD7>.{13})"

Results in:

{
  "logdate": [
    [
      "2016-09-05 15:23:44.617"
    ]
  ],
  "YEAR": [
    [
      "2016"
    ]
  ],
  "MONTHNUM": [
    [
      "09"
    ]
  ],
  "MONTHDAY": [
    [
      "05"
    ]
  ],
  "HOUR": [
    [
      "15",
      null
    ]
  ],
  "MINUTE": [
    [
      "23",
      null
    ]
  ],
  "SECOND": [
    [
      "44.617"
    ]
  ],
  "ISO8601_TIMEZONE": [
    [
      null
    ]
  ],
  "MDNIT": [
    [
      "55424617"
    ]
  ],
  "V": [
    [
      "1"
    ]
  ],
  "TRANCODE": [
    [
      "TRS123456789"
    ]
  ],
  "FIELD1": [
    [
      "XX"
    ]
  ],
  "FIELD2": [
    [
      "YY"
    ]
  ],
  "FIELD3": [
    [
      "21"
    ]
  ],
  "FIELD4": [
    [
      "345678"
    ]
  ],
  "PROCESS": [
    [
      "919"
    ]
  ],
  "APTIME": [
    [
      "   128"
    ]
  ],
  "BKTIME": [
    [
      "   118"
    ]
  ],
  "FIELD5": [
    [
      "   544"
    ]
  ],
  "CODES": [
    [
      "        "
    ]
  ],
  "FIELD6": [
    [
      "ZZ"
    ]
  ],
  "MC": [
    [
      " 2"
    ]
  ],
  "PV": [
    [
      " N"
    ]
  ],
  "PC": [
    [
      " P"
    ]
  ],
  "PX": [
    [
      " 1"
    ]
  ],
  "SYS": [
    [
      "APPLC"
    ]
  ],
  "CT": [
    [
      "21"
    ]
  ],
  "RESULTS": [
    [
      "NNNNNNNNN"
    ]
  ],
  "FIELD7": [
    [
      "KKKKKKKKKKKKk"
    ]
  ]
}

[gsl_ops_practice]

I finally got the debugger to work, but it's not telling me anything I didn't know before, there are extra white spaces in the fields and the Mutate Strip function doesn't do what it is supposed to according to LogStash documentation:

Code: Select all

  "APTIME": [
    [
      "   128"
    ]
  ],
  "BKTIME": [
    [
      "   118"
    ]

[gsl_ops_practice]

Brilliant, that does appear to work better, I will change the filter and see if I am now able to graph those values.

[mcapra]

Can you share the mutate operation? I can't recall any specific weirdness happening with strip.

[eloyd]

It was posted in the original note. I ignored it, since I knew I could fix it with a better pattern match

Code: Select all

mutate {
        strip => [
            "MDNIT",
            "APTIME",
            "PROCESS",
            "BKTIME",
            "FIELD5",
            "CODES"
            "FIELD7"
            "RESULTS"
                 ]
        }

[mcapra]

Ah, my mistake. Thanks @eloyd!

[gsl_ops_practice]

Hello,

The proposed solution does work in the Grok debugger; however, when adding this to NagiosLog as per below, I still get the behavior shown in the attached screenshot. Please advise what else we can do to resolve this. I did try to remove one or both mutate statements to no effect.

Code: Select all

if [type] == 'appv2' {

   mutate {
      gsub => ["message", "[\u0000]", ""]
   }

grok {
      match => [ "message", "[\u0000]", "%{TIMESTAMP_ISO8601:logdate} (?<MDNIT>.{8}) (?<V>.{1}) (?<TRANCODE>.{12}) (?<FIELD1>.{2})  (?<FIELD2>.{2})  (?<FIELD3>.{2}) (?<FIELD4>.{6}) (?<PROCESS>.{6})\s+%{INT:APTIME} (?<BKTIME>.{6}) (?<FIELD5>.{6}) (?<CODES>.{8}) (?<FIELD6>.{2}) (?<MC>.{2}) (?<PV>.{2}) (?<PC>.{2}) (?<PX>.{2}) (?<SYS>.{5}) (?<CT>.{2}) (?<RESULTS>.{9}) (?<FIELD7>.{13})"]
    }

mutate {
        strip => [
            "MDNIT",
            "APTIME",
            "PROCESS",
            "BKTIME",
            "FIELD5",
            "CODES"
            "FIELD7"
            "RESULTS"
                 ]
        }

date {
      match => [ "logdate", "YYYY-MM-dd HH:mm:ss.SSS" ]
  }
}