Page 1 of 2
Upgrade and status staying yellow
Posted: Mon Sep 12, 2016 8:55 am
by BanditBBS
I just upgraded from 1.4.1 to 1.4.2 and my cluster status won't leave yellow! Looks the same on both instances:
Capture.PNG
Re: Upgrade and status staying yellow
Posted: Mon Sep 12, 2016 9:36 am
by BanditBBS
It may have been yellow already.....followed link on
https://support.nagios.com/kb/article.php?id=469 to increase low disk watermark and things are changing....
Having an issue with the clustered URL now though, getting "Could not authenticate, invalid token given" when hovering the red icons on top. Or, it looks like it is working then goes back to login page.....waiting for shards to finish allocating....will report back in a few.
Re: Upgrade and status staying yellow
Posted: Mon Sep 12, 2016 10:05 am
by mcapra
Let us know when there's an update! I'd be interested to see if the Javascript console contains anything meaningful for the "Could not authenticate" issue.
The elasticsearch logs might provide some insights on the previous issue (/var/log/elasticsearch/*.log)
Re: Upgrade and status staying yellow
Posted: Mon Sep 12, 2016 12:55 pm
by BanditBBS
Ok, cluster status is back to green! Still having issue with the cluster URL though
Now when I try and login it just bounces right back to the login screen.
I checked http error and access logs, no errors in either.
Re: Upgrade and status staying yellow
Posted: Mon Sep 12, 2016 1:31 pm
by mcapra
Can you share a screenshot of what you're looking at? The previously mentioned error of Could not authenticate. Invalid token given. pretty exclusively pertains to incorrect user permissions or an invalid API key. If the login process is now just booting you back to the main login screen without any sort of error message, I would like to see the content of the elasticsearch logs at /var/log/elasticsearch/*.log
Re: Upgrade and status staying yellow
Posted: Mon Sep 12, 2016 1:41 pm
by BanditBBS
I can login to either node directly and it works great. It is only the cluster(F5 Load balancer) URL that I keep getting looped back to the login screen. Every so often it does pretend to login but then gives the "Could not authenticate. Invalid token given." error and in the iframe of the number of events the login screen appears and if I click anything it reloads the login page. This happens maybe 1 out of 100 tries to login. The elasticsearch log is basically empty except for the issue when I was crossing the watermark levels. Nothing else written since the cluster has been turned green.
Re: Upgrade and status staying yellow
Posted: Mon Sep 12, 2016 1:45 pm
by mcapra
We have seen some intermittent
Same-origin policy violations when accessing a cluster behind a load balancer. I would check the Javascript console from your preferred browser when the problem happens to see if any useful debug information is in there.
Re: Upgrade and status staying yellow
Posted: Mon Sep 12, 2016 2:13 pm
by BanditBBS
I messages my networking team to assist here, in case any changes made recently. That being said, only change on my part was upgrading from 1.4.1 to 1.4.2. I'm very certain that the cluster domain name was working for browsing just last week. Also, no ports are changed so it *shouldn't* be a same origin policy issue, F5 and nodes' ports for browsing are the same.
Re: Upgrade and status staying yellow
Posted: Mon Sep 12, 2016 3:17 pm
by mcapra
It could be other Javascript related issues. If the apache/elasticsearch logs aren't shedding any light, the Javascript console would be the next place I would check.
Re: Upgrade and status staying yellow
Posted: Mon Sep 12, 2016 3:22 pm
by BanditBBS
Error in java console:
Code: Select all
Navigated to http://naglog.itciss.com/nagioslogserver/login?redirect=
Navigated to http://naglog.itciss.com/nagioslogserver/
jquery-1.10.1.min.js:5 Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/.
/nagioslogserver/ZeroClipboard.swf?noCache=1473708621106:1 GET http://naglog.itciss.com/nagioslogserver/ZeroClipboard.swf?noCache=1473708621106
(program):73 ZeroClipboard error of type "flash-deactivated": Flash is too outdated for your browser and/or is configured as click-to-activate
Navigated to http://naglog.itciss.com/nagioslogserver/login?redirect=dashboard/fetch_rss/marketing
That happened when I disabled catch in the java console. I also saw another issue related to the fetch-rss for marketing stuff. Could that be the cause(and possible same-origin policy issue?).