Advice for switching Nagios Log Server to different machines

[2evanowen]

Hey guys,

What I am trying to do is this:
We recently got two new servers, (let's call them A+B), which we would like to purpose them for Nagios Log Sever. We currently have Nagios Log Server running on two other machines (let's call them 1+2) and would like to re-purpose these machines for something other than NLS. Right now we are using a server alias so I can switch where the logs go without changing the Rsyslog configuration files on each of our machines.

My Confusion:
How should I switch machines for Nagios Log Sever? It seems like there could be a couple different ways of doing it.
1) Would I add the new machines A+B to the existing cluster then remove the old machines 1+2 from the cluster? If so, how do you remove old machines and make it show them not being there anymore?
2) Do I have to start over from fresh install on A+B and make a new cluster? If so, Can I reuse my license when 1+2 are retired? Is there anyway to keep the NLS data intacted and moved over to A+B.

[jolson]

1) Would I add the new machines A+B to the existing cluster then remove the old machines 1+2 from the cluster? If so, how do you remove old machines and make it show them not being there anymore?

This is exactly the right procedure. Follow these steps:

1. Let's start with nodes A and B that already exist in a cluster. Add nodes C and D into the cluster as usual (making your cluster 4 nodes total).

2. Navigate to 'Administration -> Cluster Status' and ensure that your status is green. The cluster may take some time to get into a green state.

3. Turn node A off.

4. Ensure that your cluster status is green.

5. Turn off Node B.

6. Ensure that the cluster status is green.

7. Navigate to "Administration -> Per Instance" and remove the old instances from the cluster entirely.


Your configurations will remain intact. If you need to change the IP addresses at this point, feel free to do so using this procedure:

Code: Select all

service elasticsearch stop
]service logstash stop

(change ip address)

Code: Select all

service elasticsearch start
service logstash start

[2evanowen]

Perfect!
Last question, I only have a license for 2 servers, will adding 4 temporarily flag something bad? That flag should go away right?

[jolson]

Having more than 2 instances in your cluster will temporarily disable your ability to search in NLS - but you should still be able to complete your migration. When you're back to 2 instances, you'll be good to go again.

[2evanowen]

Looks like that worked perfectly!
The cluster is not turning back to green from yellow after I removed the second sever though.

[jolson]

You'll have to wait for some time while the cluster re-balances your information. Let me know if a few hours/days (depending on the amount of log data you have) pass and it's still not green - thanks!

[2evanowen]

All of this worked perfectly, Thanks for your help! Feel free to close this thread

[jolson]

Glad to hear it! I'll close it up.