logstash daemon stopping

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
User avatar
BanditBBS
Posts: 2474
Joined: Tue May 31, 2011 12:57 pm
Location: Scio, OH
Contact:

Re: logstash daemon stopping

Post by BanditBBS »

abrist wrote:No word yet from Ben. Bandit, any plans to test the patch? Or are we waiting on a response from the test subject?
I put the files on my two servers. Its been stopping on them quite regularly, so I'll provide feedback within 2 days.

p.s. I also am pushing to get my license purchased finally as well :) YAY me.
2 of XI5.6.14 Prod/DR/DEV - Nagios LogServer 2 Nodes
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
User avatar
eloyd
Cool Title Here
Posts: 2190
Joined: Thu Sep 27, 2012 9:14 am
Location: Rochester, NY
Contact:

Re: logstash daemon stopping

Post by eloyd »

Bitnetix would be happy to provide you discounts on license purchases.
Image
Eric Loyd • http://everwatch.global • 844.240.EVER • @EricLoyd
I'm a Nagios Fanatic! • Join our public Nagios Discord Server!
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: logstash daemon stopping

Post by abrist »

BanditBBS wrote: I put the files on my two servers. Its been stopping on them quite regularly, so I'll provide feedback within 2 days.
Great. Let us know how it goes . . .
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
User avatar
benhank
Posts: 1264
Joined: Tue Apr 12, 2011 12:29 pm

Re: logstash daemon stopping

Post by benhank »

maybe you are on to something.

I a, processing"

Code: Select all

25,535,679Documents
10.4GBPrimary Size
10.4GBTotal Size
1Data Instances
932Total Shards
94Indices
But after trying the patch, it has crashed twice but not once over the long weekend.
Proudly running:
NagiosXI 5.4.12 2 node Prod Env 2500 hosts, 13,000 services
Nagiosxi 5.5.7(test env) 2500 hosts, 13,000 services
Nagios Logserver 2 node Prod Env 500 objects sending
Nagios Network Analyser
Nagios Fusion
User avatar
lgroschen
Posts: 384
Joined: Wed Nov 27, 2013 1:17 pm

Re: logstash daemon stopping

Post by lgroschen »

Hey Ben,

Let us know tomorrow if anything changes.
/Luke
User avatar
benhank
Posts: 1264
Joined: Tue Apr 12, 2011 12:29 pm

Re: logstash daemon stopping

Post by benhank »

The logstash and elasticsearch daemons were still running when I logged in.
However there are no logs displayed in the default dashboards from 0840 to now.
I have included my setup as well ad the logs for today.
i'd send you the actual log file, but its almost 25 gigs :(
You do not have the required permissions to view the files attached to this post.
Proudly running:
NagiosXI 5.4.12 2 node Prod Env 2500 hosts, 13,000 services
Nagiosxi 5.5.7(test env) 2500 hosts, 13,000 services
Nagios Logserver 2 node Prod Env 500 objects sending
Nagios Network Analyser
Nagios Fusion
User avatar
BanditBBS
Posts: 2474
Joined: Tue May 31, 2011 12:57 pm
Location: Scio, OH
Contact:

Re: logstash daemon stopping

Post by BanditBBS »

My daemons seem to be stable after applying the patch. WIll update if that changes. Have another issue now though, but that's for a new thread.
2 of XI5.6.14 Prod/DR/DEV - Nagios LogServer 2 Nodes
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: logstash daemon stopping

Post by tmcdonald »

benhank, anything in your logstash or elasticsearch logs indicating any errors?

Code: Select all

/var/log/logstash/logstash.log
/var/log/elasticsearch/<really long id>.log
Former Nagios employee
User avatar
benhank
Posts: 1264
Joined: Tue Apr 12, 2011 12:29 pm

Re: logstash daemon stopping

Post by benhank »

Yeah. it says "error something aint working" ...
After I set up the logging server to pull the info ill post the results
Proudly running:
NagiosXI 5.4.12 2 node Prod Env 2500 hosts, 13,000 services
Nagiosxi 5.5.7(test env) 2500 hosts, 13,000 services
Nagios Logserver 2 node Prod Env 500 objects sending
Nagios Network Analyser
Nagios Fusion
User avatar
lgroschen
Posts: 384
Joined: Wed Nov 27, 2013 1:17 pm

Re: logstash daemon stopping

Post by lgroschen »

Get back to us tomorrow, we'll fix you up good sir
/Luke
Locked