Page 1 of 2
imap input
Posted: Fri Jun 19, 2015 11:40 am
by Envera IT
Anyone get the imap input working for logstash? Seems like no matter what I do it crashes with
Code: Select all
{:timestamp=>"2015-06-19T12:26:29.290000-0400", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::IMAP type=>\"mail\", host=>\"imap.gmail.com\", user=>\"[email protected]\", content_type=>\"text/plain\">\n Error: undefined method `encode' for nil:NilClass", :level=>:error}
Here's my input config
Code: Select all
imap {
type => mail
host => "imap.gmail.com"
user => "[email protected]"
password => "mypasswordhere"
port => 993
secure => true
check_interval => 10
}
I've also tried changing the content option but that hasn't helped. I've confirmed the mails are coming in as Content-Type: text/plain.
After spending around two hours trying to get it setup I've run out of google results. Need to be able to accept emails as an input in logstash as we have a few hundred devices that only support email for alert generation.
Re: imap input
Posted: Fri Jun 19, 2015 12:10 pm
by jolson
Typically this error is a result of improper syntax on the input. Could you try the following?
Code: Select all
imap {
type => mail
host => "imap.gmail.com"
port => 993
user => [email protected]
password => mypasswordhere
secure => "true"
check_interval => 10
}
Re: imap input
Posted: Fri Jun 19, 2015 12:26 pm
by Envera IT
jolson wrote:Typically this error is a result of improper syntax on the input. Could you try the following?
Code: Select all
imap {
type => mail
host => "imap.gmail.com"
port => 993
user => [email protected]
password => mypasswordhere
secure => "true"
check_interval => 10
}
THANK YOU
good to go on my end now.
Re: imap input
Posted: Fri Jun 19, 2015 12:41 pm
by jolson
No problem. I'll lock this thread up. Let us know if you have any further questions!
Jesse
Re: imap input
Posted: Fri Jun 26, 2015 4:21 pm
by Envera IT
Still having issues with the imap plugin. I've traced it to an issue with the content_type variable. Basically I'm not sure how to work with multipart messages. Want to add that I'm testing this on a fresh NLS install with nothing else running on it. Symptom is the same, after successfully pulling emails in from the mail box for awhile, the plugin crashes and logstash with it. Logstash needs to be manually restarted after this happens.
My input looks like this.
imap {
type => mail
host => "imap.gmail.com"
port => 993
user =>
[email protected]
password => mypasswordhere
secure => "true"
check_interval => 10
}
Logstash documentation just says -
https://www.elastic.co/guide/en/logstas ... ntent_type
content_type
Value type is string
Default value is "text/plain"
For multipart messages, use the first part that has this content-type as the event message.
These are the logstash.log errors I'm getting.
{:timestamp=>"2015-06-19T12:26:29.290000-0400", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::IMAP type=>\"mail\", host=>\"imap.gmail.com\", user=>\"
[email protected]\", content_type=>\"text/plain\">\n Error: undefined method `encode' for nil:NilClass", :level=>:error}
{:timestamp=>"2015-06-19T11:50:16.689000-0400", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::IMAP type=>\"mail\", host=>\"imap.gmail.com\", user=>\"
[email protected]\", content_type=>\"text/plain\">\n Error: Can not decode an entire message, try calling #decoded on the various fields and body or parts if it is a multipart message.", :level=>:error}
The raw email I'm receiving from the device is this,
Return-Path: <
[email protected]>
Received: from localhost (redacted. [redacted])
by mx.google.com with ESMTPSA id z194sm7851754ywz.56.2015.06.24.05.39.17
for <
[email protected]>
(version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128);
Wed, 24 Jun 2015 05:39:18 -0700 (PDT)
Message-ID: <372554.386357905-sendEmail@localhost>
From: "
[email protected]" <
[email protected]>
To: "
[email protected]" <
[email protected]>
Subject: 0001 redactedTEST: 24-06-15 08:39:38 - Z000 1055 REAL 001 SYSTEM |CAM01 LIVE VIEW
Date: Wed, 24 Jun 2015 12:39:39 +0000
X-Mailer: sendEmail-1.56
MIME-Version: 1.0
Content-Type: multipart/related; boundary="----MIME delimiter for sendEmail-167244.87494594"
This is a multi-part message in MIME format. To properly display this message you need a MIME-Version 1.0 compliant Email program.
------MIME delimiter for sendEmail-167244.87494594
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
0001 redactedTEST: 24-06-15 08:39:38 - Z000 1055 REAL 001 SYSTEM |CAM01 LIVE VIEW
------MIME delimiter for sendEmail-167244.87494594--
I really just need this part of the message,
0001 redactedTEST: 24-06-15 08:39:38 - Z000 1055 REAL 001 SYSTEM |CAM01 LIVE VIEW
Seems to be related to this -
https://logstash.jira.com/browse/LOGSTASH-2080 which given the lack of updates is disheartening
Any advice?
Re: imap input
Posted: Mon Jun 29, 2015 9:44 am
by jolson
Please give the following a try:
Code: Select all
imap {
type => mail
host => "imap.gmail.com"
port => 993
user => [email protected]
password => mypasswordhere
secure => "true"
check_interval => 10
content_type => "text/html"
}
I agree that this looks like a possibly longstanding bug in logstash - I am hoping that the above will work for you.
Re: imap input
Posted: Mon Jun 29, 2015 9:45 pm
by Envera IT
I've made the change, will report back on what I find in a day or two. Thanks for the help so far!
Re: imap input
Posted: Tue Jun 30, 2015 9:14 am
by tmcdonald
Sure thing. Keep us posted!
Re: imap input
Posted: Tue Jun 30, 2015 9:15 am
by jolson
No problem! Let us know what you find out.
Re: imap input
Posted: Tue Jun 30, 2015 9:19 am
by eloyd
Sure thing. No problem! Keep us posted one what you find out.
