Page 1 of 1

Losing CR/LF's in emailed alerts

Posted: Tue May 14, 2019 3:50 pm
by mbeebe
Hello,

I've got an odd issue that I'm hoping someone smarter than me can shed light on.

I'm collecting logs from a Linux server. The logs in question look like this:

Code: Select all

[2019/05/14-11:51:40.097] [FT-ScenarioThread-Xm6NvwMU-411624] [INFO] [dip.scenarios.mail.abstract] scenario PUBLICAPITESTING.SHOULDFAIL#2019-05-14-11-51-39-307 - Expanded message body text to
****Operations Support Alert****
line1^Mline2^Mline3
****End Operations Support Alert****
Note the ^M's separating the lines line1, line2 and line 3. These should be the equivalent of CR/LF in Windows.

I'm ingesting these logs into NLS via a multiline input with the following filter:

Code: Select all

tcp {
    type => dataiku_multiline
    codec => multiline {
        pattern => '^<133>%{GREEDYDATA}\[[0-9]{4}[/][0-9]{2}[/][0-9]{2}[-][0-9]{2}[:][0-9]{2}[:][0-9]{2}.[0-9]{3}\]'
        negate => true
        what => previous
    }
    port => 6688
    }
So far, so good.

The logs appear in NLS and look like this:

Code: Select all

<133>May 14 11:51:41 XXXXXXXX dataiku_multiline: [2019/05/14-11:51:40.097] [FT-ScenarioThread-Xm6NvwMU-411624] [INFO] [dip.scenarios.mail.abstract] scenario PUBLICAPITESTING.SHOULDFAIL#2019-05-14-11-51-39-307 - Expanded message body text to
<133>May 14 11:51:41 XXXXXXXXXX dataiku_multiline: ****Operations Support Alert****
<133>May 14 11:51:41 XXXXXXXXXX dataiku_multiline: [b]line1
line2
line3
[/b]<133>May 14 11:51:41 XXXXXXXXXX dataiku_multiline: ****End Operations Support Alert****
Note that line1, line 2 and line3 all have appear to have the correct linefeeds.

Now things start to go sideways. When my alert fires and emails me, here's what I get:

Code: Select all

<133>May 14 11:51:41 XXXXXXX dataiku_multiline: [2019/05/14-11:51:40.097] [FT-ScenarioThread-Xm6NvwMU-411624] [INFO] [dip.scenarios.mail.abstract] scenario PUBLICAPITESTING.SHOULDFAIL#2019-05-14-11-51-39-307 - Expanded message body text to <133>May 14 11:51:41 XXXXXXXX dataiku_multiline: ****Operations Support Alert**** <133>May 14 11:51:41 XXXXXXXXX dataiku_multiline: line1line2line3 <133>May 14 11:51:41 XXXXXXXX dataiku_multiline: ****End Operations Support Alert****
Note that line1, line2 and line3 are all run together now.

Anyone have any idea of why this is happening and how to make the messages come across with the correct LF/CR encodings?

Thanks,

-- Mike Beebe

Re: Losing CR/LF's in emailed alerts

Posted: Wed May 15, 2019 12:40 pm
by cdienger
I'm seeing the same behavior and at this point don't know if it's expected or not. I'll ping dev regarding it and follow up.

Re: Losing CR/LF's in emailed alerts

Posted: Wed May 15, 2019 1:42 pm
by mbeebe
Much appreciated -- thank you.

Re: Losing CR/LF's in emailed alerts

Posted: Thu May 16, 2019 9:51 am
by cdienger
A bug has been filed and this will be fixed in 2.1.

Re: Losing CR/LF's in emailed alerts

Posted: Thu May 16, 2019 10:33 am
by mbeebe
Thank you -- that's excellent news.

Please feel free to close this ticket.

-- Mike

Re: Losing CR/LF's in emailed alerts

Posted: Thu May 16, 2019 10:35 am
by cdienger
Locking thread.