Page 2 of 2

Re: Logstash: filter not filtering(?)

Posted: Thu Nov 07, 2019 3:11 pm
by mbeebe
scottwilkerson wrote:You can but be aware that the config you pass cannot contain the same ports that you are using while running as a service.
Crud, that's not going to work, then.

I'm not sure why we're seeing the results we are. Our updated filter works correctly in a grok emulator, but appears to have no impact in NLS. Is the "message" field immutable in NLS?

-- Mike Beebe

Re: Logstash: filter not filtering(?)

Posted: Thu Nov 07, 2019 3:26 pm
by scottwilkerson
mbeebe wrote:Is the "message" field immutable in NLS?
no, it should be able to be removed or changed

Re: Logstash: filter not filtering(?)

Posted: Thu Nov 07, 2019 3:33 pm
by scottwilkerson
You literally should be able to this in a filter to delete it

Code: Select all

mutate { 
    remove_field => [ "message" ] 
}
and additionally this to add something to it

Code: Select all

mutate {
    add_field => { "message" => "some text" }
}
or to add the contents of a different field

Code: Select all

mutate {
    add_field => { "message" => "%{message_body}" }
}
putting it all together, i you have a field called message_body that contained the contents you want

Code: Select all

mutate { 
    remove_field => [ "message" ] 
}
mutate {
    add_field => { "message" => "%{message_body}" }
}

Re: Logstash: filter not filtering(?)

Posted: Fri Nov 08, 2019 11:37 am
by mbeebe
Hi Scott,

We finally figured out why the filter I sent you wasn't working. The issue is the way we were doing the initial trigger for the filter.

Original, non-functional:

Code: Select all

if [program] == 'program_multiline' { (…)

Functional filter trigger:

Code: Select all

if [type] == 'program_multiline' { (…)

Unfortunately, due to the way we're structuring our messages, I will not be able to accomplish the original goal of this issue. Back to the drawing board.

Issue is ready for lock and thanks for all your help,

-- Mike Beebe

Re: Logstash: filter not filtering(?)

Posted: Fri Nov 08, 2019 11:42 am
by scottwilkerson
mbeebe wrote:Hi Scott,

We finally figured out why the filter I sent you wasn't working. The issue is the way we were doing the initial trigger for the filter.

Original, non-functional:

Code: Select all

if [program] == 'program_multiline' { (…)

Functional filter trigger:

Code: Select all

if [type] == 'program_multiline' { (…)

Unfortunately, due to the way we're structuring our messages, I will not be able to accomplish the original goal of this issue. Back to the drawing board.

Issue is ready for lock and thanks for all your help,

-- Mike Beebe
Ahhh... Best of luck!

Locking thread