Page 1 of 1
How to move data?
Posted: Wed Jan 20, 2016 12:43 pm
by 2evanowen
Hey guys,
Our /usr directory is becoming very large. We are looking to move only the data from elasticsearch over and keeping all of the configuration files in place.
What is the best way to do this? Which folders will I have to move over? Which configuration files will I need to modify? Which services do I need to stop before doing this?
I was thinking of copying all the data from /usr/local/nagioslogserver/elasticsearch/data to /data1/data, then just making a soft link from /usr/local/nagioslogserver/elasticsearch/data to /data1/data. Would that work? Is that all I have to do?
Re: How to move data?
Posted: Wed Jan 20, 2016 3:13 pm
by jolson
was thinking of copying all the data from /usr/local/nagioslogserver/elasticsearch/data to /data1/data, then just making a soft link from /usr/local/nagioslogserver/elasticsearch/data to /data1/data. Would that work? Is that all I have to do?
I tested the above in a lab, and it worked like a charm. Full procedure:
Code: Select all
service elasticsearch stop
cd /usr/local/nagioslogserver/elasticsearch
mkdir /data1 ; chown nagios.nagios /data1/
mv data/ /data1/
ln -s /data/data/ data
service elasticsearch start
If elasticsearch is running and notices that it has lost data, it will try rebuilding your database with _nothing_ inside of it. You should be able to perform this procedure in a rolling fashion - one instance at a time until complete.
Let us know if you have any problems, thanks!
Re: How to move data?
Posted: Wed Jan 20, 2016 4:18 pm
by 2evanowen
It looks like it worked, until I log in. Then clicking around I have an error and it looks like most of my logs aren't there:
Screen Shot 2016-01-20 at 2.13.56 PM.png
Re: How to move data?
Posted: Wed Jan 20, 2016 4:32 pm
by 2evanowen
I have only updated one of the nodes so far, should I do the other now?
Re: How to move data?
Posted: Wed Jan 20, 2016 6:29 pm
by jolson
Give the other one a try as well - and be sure that the 'nagios' user is allowed to write to your new directory properly.
Re: How to move data?
Posted: Wed Jan 20, 2016 6:38 pm
by 2evanowen
Okay I did the other one as well. Still Doesn't seem to be helping...
Both servers have the nagios:nagios for the owner and group....
Last one:
Code: Select all
[owen@barium data1]$ ls -al
total 4
drwxr-xr-x 3 nagios nagios 17 Jan 20 13:55 .
dr-xr-xr-x. 19 root root 4096 Jan 20 09:55 ..
drwxr-xr-x. 4 nagios nagios 92 Nov 23 15:04 data
New server:
Code: Select all
[owen@calcium data1]$ ls -al
total 4
drwxr-xr-x. 3 nagios nagios 17 Jan 20 15:39 .
dr-xr-xr-x. 19 root root 4096 Jan 20 09:40 ..
drwxr-xr-x. 4 nagios nagios 92 Nov 23 15:06 data
It looks like calcium is getting logs from barium, but that's the only server:
Screen Shot 2016-01-20 at 4.35.40 PM.png
Re: How to move data?
Posted: Wed Jan 20, 2016 6:54 pm
by 2evanowen
Oh yes,
I am getting this FacetPhaseExecutionException message every once in while on the dashboard:
Screen Shot 2016-01-20 at 4.52.51 PM.png
Re: How to move data?
Posted: Wed Jan 20, 2016 7:04 pm
by jolson
Is your cluster status green?
Code: Select all
curl 'localhost:9200/_cluster/health?level=indices&pretty'
If not, it's possible that you need to give the cluster some time to distribute data appropriately.
Alternatively, you could try changing the DATA_DIR directive in
/etc/sysconfig/elasticsearch:
DATA_DIR="/data1/data"
Give elasticsearch a restart after that and let me know if it helps - this directive will not be overwritten on upgrade.
Jesse
Re: How to move data?
Posted: Thu Jan 21, 2016 1:09 pm
by 2evanowen
I got it working!
My global settings were messed up for some reason and didn't have my cluster hostname in there.
Both ways work for me now, the link and the DATA_DIR.
Thanks for your help!
Re: How to move data?
Posted: Thu Jan 21, 2016 1:21 pm
by bwallace
Thanks for that update and glad to hear it is working -- we'll lock this thread now and feel free to open another should you require assistance with anything else.