Issue on one node after 1.4.2 upgrade

[WillemDH]

One of my 2 NLS nodes is no longer showing any data in the paginated table with the record overview since upgrading to 1.4.2. Check the screenshot. Tried rebooting the node and compared some config files, but everything seems identical.

[hsmith]

If you click one of the messages to expand it, is there still data?

[WillemDH]

No, it seems to be all empty..

[hsmith]

Nothing was changed should have caused this..

Code: Select all

1.4.2 - 07/22/2016
==================
- Fixed German and French language options causing javascript errors in Dashboards tab [TPS#8943] -JO
- Fixed issue where using non-default port to connect would cause Dashboards to not load properly [TPS#8861] -JO
- Fixed audit log 'created_by' field being populated with user ID instead of username [TSP#8609] -JO
- Fixed AD/LDAP import to allow parenthesis [TPS#8920] -SS, JO
- Fixed XSS vulnerability in logs that are viewed through the table in Dashboards section [TPS#8694] -JO
- Fixed various security vulnerabilities -JO
- Fixed issue with bettermap no longer working [TPS#8694] -JO

I'm wondering if - Fixed XSS vulnerability in logs that are viewed through the table in Dashboards section [TPS#8694] -JO could be causing this. Any considerations such as proxy/load balancer?

[WillemDH]

No load balancers here. I did also yum update today. Could be related. But everything I've done today I did on both nodes. So I don't have a clue what went wrong on the one node..

[hsmith]

This is CentOS 7.2? I can yum update and try to replicate. Either way, pointing this thread out to the developers to try to get some idea generation here, since it's working for all of us AFAIK.

[WillemDH]

These both are from the standard vm image supplied by Nagios..

Node 1:

Code: Select all

       Release = CentOS release 6.8 (Final)
        Kernel = Linux 2.6.32-279.11.1.el6.x86_64
      Platform = VMware Virtual Platform/440BX Desktop Reference Platform
        Uptime = 0 day(s). 1:25:2
     CPU Usage = 14.6% average CPU usage over 6 core(s)
      CPU Load = 1m: 0.72, 5m: 0.65, 15m: 0.49
        Memory = Free: 0.17GB (1%), Used: 15.52GB (99%), Total: 15.69GB
          Swap = Free: 1.95GB (100%), Used: 0.00GB (0%), Total: 1.95GB
          Root = Free: 109.21GB (65%), Used: 56.44GB (35%), Total: 167.35GB
       Updates = 0 yum updates available
      Sessions = 1 sessions
     Processes = 186 running processes of 128457 maximum processes
           PHP = Version: 5.3.3
        Apache = Version: 2.2.15

Node 2:

Code: Select all

       Release = CentOS release 6.8 (Final)
        Kernel = Linux 2.6.32-279.11.1.el6.x86_64
      Platform = VMware Virtual Platform/440BX Desktop Reference Platform
        Uptime = 0 day(s). 2:33:43
     CPU Usage = 16.72% average CPU usage over 6 core(s)
      CPU Load = 1m: 1.32, 5m: 1.14, 15m: 1.10
        Memory = Free: 0.17GB (1%), Used: 15.52GB (99%), Total: 15.69GB
          Swap = Free: 1.95GB (100%), Used: 0.00GB (0%), Total: 1.95GB
          Root = Free: 100.00GB (60%), Used: 65.65GB (40%), Total: 167.35GB
       Updates = 0 yum updates available
      Sessions = 2 sessions
     Processes = 189 running processes of 128457 maximum processes
           PHP = Version: 5.3.3
        Apache = Version: 2.2.15

Seems like the nodes are still replicating, the last reboot of the problem node was 1;25 hours ago. It usually doesn't take so long.. Let me know if you need any more info, I have plenty of metrics. I'm just not sure which ones would interesting in this case. I though maybe eit was a visudo issues, as I noticed the visudo file was changed after the upgrade. But the visudo file is identical to the working node.

[hsmith]

Any javascript errors when you load the page(Chrome / FF dev console)?

[WillemDH]

Good idea. Seem there is a difference between the two nodes. On the working node I'm NOT getting this error:

Code: Select all

app.js?ver=2015R1.4-3.1.1-nagios3:11 Error: Circular dependency: 
    at Error (native)
    at Object.c [as get] (https://mydomain/nagioslogserver/app/app.js?ver=2015R1.4-3.1.1-nagios3:11:5873)
    at https://mydomain/nagioslogserver/app/app.js?ver=2015R1.4-3.1.1-nagios3:12:15764
    at s (https://mydomain/nagioslogserver/app/app.js?ver=2015R1.4-3.1.1-nagios3:11:27695)
    at r (https://mydomain/nagioslogserver/app/app.js?ver=2015R1.4-3.1.1-nagios3:11:27660)
    at p (https://mydomain/nagioslogserver/app/app.js?ver=2015R1.4-3.1.1-nagios3:11:27477)
    at $a (https://mydomain/nagioslogserver/app/app.js?ver=2015R1.4-3.1.1-nagios3:11:30238)
    at https://mydomain/nagioslogserver/app/app.js?ver=2015R1.4-3.1.1-nagios3:11:31969
    at Object.$eval (https://mydomain/nagioslogserver/app/app.js?ver=2015R1.4-3.1.1-nagios3:12:7203)
    at Object.runBinders (https://mydomain/nagioslogserver/app/app.js?ver=2015R1.4-3.1.1-nagios3:22:23882) <span ng-if="!panel.localTime || panel.timeField != field" bo-html="(event.kibana.highlight[field]||event.kibana._source[field]) | tableHighlightLinkNoXML | tableTruncate:panel.trimFactor:panel.fields.length" class="table-field-value ng-scope">(anonymous function) @ app.js?ver=2015R1.4-3.1.1-nagios3:11(anonymous function) @ app.js?ver=2015R1.4-3.1.1-nagios3:11n @ app.js?ver=2015R1.4-3.1.1-nagios3:11f @ app.js?ver=2015R1.4-3.1.1-nagios3:11(anonymous function) @ app.js?ver=2015R1.4-3.1.1-nagios3:11(anonymous function) @ app.js?ver=2015R1.4-3.1.1-nagios3:13$digest @ app.js?ver=2015R1.4-3.1.1-nagios3:12$apply @ app.js?ver=2015R1.4-3.1.1-nagios3:12f @ app.js?ver=2015R1.4-3.1.1-nagios3:12r @ app.js?ver=2015R1.4-3.1.1-nagios3:12v.onreadystatechange @ app.js?ver=2015R1.4-3.1.1-nagios3:12

[hsmith]

Can you try an incognito/private window or clearing your browser cache?