Search found 73 matches

by polarbear1
Fri Mar 24, 2017 2:55 pm
Forum: Nagios Log Server
Topic: Logstash process exited, but running?
Replies: 9
Views: 2163

Re: Logstash process exited, but running?

avandemore wrote: However this appears to be different than what you originally posted unless I'm missing something?
I think we're on the same page. Give it a crappy pattern, restart logstash (can't say for feeding it matching logs though), it crashes, but doesn't report as dead on the dashboard.
by polarbear1
Fri Mar 24, 2017 1:22 pm
Forum: Nagios Log Server
Topic: Logstash process exited, but running?
Replies: 9
Views: 2163

Re: Logstash process exited, but running?

Pretty sure there's nothing else. Just verified by doing the above steps on an entirely separate cluster (2 nodes) and the results where the same. [root@schpnag11 ~]# cat /var/log/logstash/logstash.log {:timestamp=>"2017-03-24T13:21:26.543000-0500", :message=>"SIGTERM received. Shutti...
by polarbear1
Fri Mar 24, 2017 10:24 am
Forum: Nagios Log Server
Topic: Logstash process exited, but running?
Replies: 9
Views: 2163

Re: Logstash process exited, but running?

Do you know how to induce this behavior? I'm trying to replicate it. Yes sir! Just was able to recreate it on the spot. 1. In NLS -- Administration > Global Configuration > +Add Filter 2. Make a filter with an undefined pattern. (I just grabbed a part of the default Apache filter, changed MONTHDAY ...
by polarbear1
Thu Mar 23, 2017 2:03 pm
Forum: Nagios Log Server
Topic: Logstash process exited, but running?
Replies: 9
Views: 2163

Re: Logstash process exited, but running?

Ahh, I remembered what I forgot. I had a custom grok pattern defined before the update that was lost in the updates, so grok failed on that note because in my global config it did preserve the custom filter. Well then. I cleared that up and now everything is as expected. This is what was different f...
by polarbear1
Thu Mar 23, 2017 10:17 am
Forum: Nagios Log Server
Topic: Logstash process exited, but running?
Replies: 9
Views: 2163

Logstash process exited, but running?

Greetings, I recently migrated my NLS clusters from RHEL 6.8 to 7.3, updated to NLS 1.4.4 while I was at it, and I'm seeing a weird behavior on a couple of them. Long story short, the dashboard shows Logstash as running and NLS seems to be behaving correctly, however on the server itself there is no...
by polarbear1
Fri Aug 12, 2016 2:21 pm
Forum: Nagios Log Server
Topic: Logstash logs - growing too big, too fast.
Replies: 17
Views: 3117

Re: Logstash logs - growing too big, too fast.

Looking at your fields, you could add another if statement to check what the Program is equal to, since I'm guessing each log file will be unique this is one way to separate them. Yes, and sadly no. The reason for me wildcarding the folder is that it's a pretty dynamic environment. New log files wi...
by polarbear1
Thu Aug 11, 2016 11:58 am
Forum: Nagios Log Server
Topic: Logstash logs - growing too big, too fast.
Replies: 17
Views: 3117

Re: Logstash logs - growing too big, too fast.

when you apply the grok filter to the field SourceModuleName = iso, if the log doesn't have anything to match it's going to failure. When NLS does grok filtering, it needs a common way to differentiate the logs. Can you clarify that. I was under the impression (and this is my objective - if there i...
by polarbear1
Wed Aug 10, 2016 4:15 pm
Forum: Nagios Log Server
Topic: Logstash logs - growing too big, too fast.
Replies: 17
Views: 3117

Re: Logstash logs - growing too big, too fast.

As you can see from the previous screenshot, the Grok config in question is: if [SourceModuleName] == 'iso' { grok { match => ['message', '%{LOGLEVEL1:severity_label}'] } } Of course, it would help to know what "LOGLEVEL1" is as that's a custom pattern. And yes, I have it defined on all se...
by polarbear1
Wed Aug 10, 2016 8:59 am
Forum: Nagios Log Server
Topic: Logstash logs - growing too big, too fast.
Replies: 17
Views: 3117

Re: Logstash logs - growing too big, too fast.

The logs are coming from a Windows box, using nxlog. Also - since the screenshot of the globalconfig, I removed the line about parsing DATESTAMP - that was something else I was playing around with that I don't need anymore. So now we're just trying to parse SEVERITY out of the message. Attached are ...
by polarbear1
Tue Aug 09, 2016 2:23 pm
Forum: Nagios Log Server
Topic: Logstash logs - growing too big, too fast.
Replies: 17
Views: 3117

Re: Logstash logs - growing too big, too fast.

Yes you can. Attached. I only expected the configs for the non-default groks I added. Everything else is default. As you can probably tell from teh context - the "ISO Parser" config is the only relevant one to the 400 Errors I posted above. The middletier config is not applicable for those...