post upgradeNagios Log Server - Waiting For Database Startup

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
P201420
Posts: 28
Joined: Fri Feb 15, 2019 9:22 am

post upgradeNagios Log Server - Waiting For Database Startup

Post by P201420 »

2 cluster NLS
both have latest applied
post upgrade got waiting for DB message
have rebooted both per kb site

getting
Nagios Log Server - Waiting For Database Startup
Last edited by P201420 on Thu Sep 26, 2019 11:52 am, edited 1 time in total.
benjaminsmith
Posts: 5324
Joined: Wed Aug 22, 2018 4:39 pm
Location: saint paul

Re: post upgradeNagios Log Server - Waiting For Database Sta

Post by benjaminsmith »

Hello,

Have you checked the status of elasticsearch, is it running on the server?

Code: Select all

systemctl status elasticsearch.service
Please post the output to the following commands from the terminal. Thanks.

Code: Select all

curl -XGET 'http://localhost:9200'
tail -n 50 /var/log/elasticsearch/*.log
ps aux | grep java
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.

Be sure to check out our Knowledgebase for helpful articles and solutions!
P201420
Posts: 28
Joined: Fri Feb 15, 2019 9:22 am

Re: post upgradeNagios Log Server - Waiting For Database Sta

Post by P201420 »

0
Last edited by P201420 on Thu Sep 26, 2019 11:53 am, edited 1 time in total.
P201420
Posts: 28
Joined: Fri Feb 15, 2019 9:22 am

Re: post upgradeNagios Log Server - Waiting For Database Sta

Post by P201420 »

o
Last edited by P201420 on Thu Sep 26, 2019 11:53 am, edited 1 time in total.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: post upgradeNagios Log Server - Waiting For Database Sta

Post by scottwilkerson »

On nl85u1016 can you show the output of the following

Code: Select all

cat /usr/local/nagioslogserver/var/cluster_hosts
I thing there is an errant blank line in there
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
P201420
Posts: 28
Joined: Fri Feb 15, 2019 9:22 am

Re: post upgradeNagios Log Server - Waiting For Database Sta

Post by P201420 »

-
Last edited by P201420 on Thu Sep 26, 2019 11:53 am, edited 1 time in total.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: post upgradeNagios Log Server - Waiting For Database Sta

Post by scottwilkerson »

On nl85u1016 can you edit /usr/local/nagioslogserver/var/cluster_hosts and remove the blank line

then restart elasticsearch

Code: Select all

service elasticsearch restart
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
P201420
Posts: 28
Joined: Fri Feb 15, 2019 9:22 am

Re: post upgradeNagios Log Server - Waiting For Database Sta

Post by P201420 »

-
Last edited by P201420 on Thu Sep 26, 2019 11:54 am, edited 1 time in total.
P201420
Posts: 28
Joined: Fri Feb 15, 2019 9:22 am

Re: post upgradeNagios Log Server - Waiting For Database Sta

Post by P201420 »

-
Last edited by P201420 on Thu Sep 26, 2019 11:54 am, edited 1 time in total.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: post upgradeNagios Log Server - Waiting For Database Sta

Post by scottwilkerson »

So, at the bottom of each log I see one node says new_master

Code: Select all

[2019-09-25 12:05:52,407][INFO ][cluster.service ] [0358e3fa-37dc-45fc-afe0-65853775d580] new_master [0358e3fa-37dc-45fc-afe0-65853775d580][nwLWtBlqRZSSpYF9jpzJKQ][nl85u1015.resideo.com][inet[/10.86.2.81:9300]]
and the other says detected_master

Code: Select all

[2019-09-25 12:05:58,822][INFO ][cluster.service ] [a7805855-0b0a-4d41-bf96-ba7d5940f677] detected_master [0358e3fa-37dc-45fc-afe0-65853775d580][nwLWtBlqRZSSpYF9jpzJKQ][nl85u1015.resideo.com][inet[/10.86.2.81:9300]]
This looks like they should be connected
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
Locked