NLS Frozen

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
User avatar
WillemDH
Posts: 2320
Joined: Wed Mar 20, 2013 5:49 am
Location: Ghent
Contact:

NLS Frozen

Post by WillemDH »

HMm,

I think I might have made a dashboard with a too complex query (something with topn of messgae field)

and it seems My NLS servers are both frozen atm. WHat do I best do to solve this? reboot? Kill Java process? Restart services?

Grtz

Willem
You do not have the required permissions to view the files attached to this post.
Nagios XI 5.8.1
https://outsideit.net
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: NLS Frozen

Post by tmcdonald »

That query would be handled by elasticsearch, so one at a time restart elasticsearch, waiting a good 15 minutes between restarts.
Former Nagios employee
User avatar
WillemDH
Posts: 2320
Joined: Wed Mar 20, 2013 5:49 am
Location: Ghent
Contact:

Re: NLS Frozen

Post by WillemDH »

Restarting elasticsearch service does seem to solve the issue, untill I go to that particular dashboard again lol. So, will I Have to delete that dashboard? Or can I edit it without starting / refreshing it?
Nagios XI 5.8.1
https://outsideit.net
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: NLS Frozen

Post by tmcdonald »

Viewing that dashboard will kick it off, unfortunately. You could save it quickly to disk, edit the JSON to remove the offending panel, then reload it. Of course this relies on being able to save it quickly. A dev might be able to help more with doing this from the backend.
Former Nagios employee
User avatar
WillemDH
Posts: 2320
Joined: Wed Mar 20, 2013 5:49 am
Location: Ghent
Contact:

Re: NLS Frozen

Post by WillemDH »

I managed to quickly delete the query causing java procs to spike. Isn't there some mechanism preventing cpu choking like this from happening?
Nagios XI 5.8.1
https://outsideit.net
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: NLS Frozen

Post by tmcdonald »

Within Logserver and the associated ELK stack? I can't really say. I would like to imagine that Java itself would have some watchdog capabilities but even those can be faulty. Did you have something in mind when you asked this?
Former Nagios employee
User avatar
WillemDH
Posts: 2320
Joined: Wed Mar 20, 2013 5:49 am
Location: Ghent
Contact:

Re: NLS Frozen

Post by WillemDH »

Lol Trevor, yeah some vague feature request thingy. Not really urgent, but it would be useful imho if for example a timeout could be configured that would stop the loading of the dashlets on the dashboard after x time (eg 120 seconds or so) . I'm not sure what is possible and what not. You can close this thread if you want Trevor.
Nagios XI 5.8.1
https://outsideit.net
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: NLS Frozen

Post by tmcdonald »

WillemDH wrote:I'm not sure what is possible and what not.
I think we are in the same boat here. I started learning Logserver from the backend up, so I am not too familiar with what would be possible in the interface. My gut tells me this is still a backend issue, but AJAX is a hell of a thing and there might be a solution here. I'll put in that request.
Former Nagios employee
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: NLS Frozen

Post by jolson »

Feature request 5110 has been filed regarding this issue.
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
Locked