Page 1 of 2
Nagios Log Server on RHEL 6
Posted: Mon Dec 01, 2014 10:40 am
by krobertson71
Our Linux admins break out our RHEL6 installs into separate partitions like so:
/
/boot
/var
/opt
/usr
/tmp
etc
What I need to know is where Log Server keeps most of it's data so I can tell them which partitions need to be what size. I understand the size depends on what I am collecting, I am just asking which partition(s) should I be putting the "bulk" of my capacity in. Or, if you have a recommendation on each of these then that would be helpful as well.
Re: Nagios Log Server on RHEL 6
Posted: Mon Dec 01, 2014 10:46 am
by slansing
If I am reading what you are asking correctly, it should be "/usr", more specifically, your indicies, and cluster data are stored in the elasticsearch directory tree at:
Code: Select all
/usr/local/nagioslogserver/elasticsearch/data/<CLUSTER_ID>/nodes/<NODE_#>/
In addition, /var will grow quite a bit with both the current logstash.log, and it's tarred archives in:
Re: Nagios Log Server on RHEL 6
Posted: Mon Dec 01, 2014 10:59 am
by krobertson71
Yeah that is what I was looking for. So Logstash doesn't need a big store as well? And log files... are they going to var or within /usr as well?
Re: Nagios Log Server on RHEL 6
Posted: Mon Dec 01, 2014 2:39 pm
by slansing
That would be another good one to provide some additional space on, more specifically:
As you had mentioned, that contains your current logstash.log, as well as the tarred archives. Apologies for not mentioning that in my original post, I'm hunting around with:
Code: Select all
du -m --max-depth 1 | sort -rn | head -11
There is also an elasticsearch directory in /var which houses cluster snippets, I need to do more looking into the elasticsearch mechanics we run on logserver to say what is actually being moved or copied out, and what is not.
Re: Nagios Log Server on RHEL 6
Posted: Mon Dec 01, 2014 9:55 pm
by krobertson71
Cool keep my posted. I will do my research on this end.
Re: Nagios Log Server on RHEL 6
Posted: Tue Dec 02, 2014 11:39 am
by cmerchant
Good info to know regarding file locations and space allocation. I suspect that this will be a good FAQ or doc to be added to the standard NLS documentation. Thanks.
Re: Nagios Log Server on RHEL 6
Posted: Tue Dec 02, 2014 4:15 pm
by krobertson71
Yeah I need more info I believe..
So lets say I will be collecting 100 gigs of log data...
How big should /var be considering logstash...
How big should /usr be considering elasticsearch and cluster/indexes etc are located..
Re: Nagios Log Server on RHEL 6
Posted: Tue Dec 02, 2014 4:27 pm
by krobertson71
actually I think most everything is in /usr
Selection_031.png
This is from my laptop running logserver in VirtualBox.
Here is /var compared to /usr. To me it looks like most of my space allocation needs to go to /usr. Tell me if I am wrong about this.
Selection_032.png
Re: Nagios Log Server on RHEL 6
Posted: Tue Dec 02, 2014 4:50 pm
by sreinhardt
I completely agree with krobertson. /var may raise somewhat in size, but the vast vast majority of files should be in /usr/local/nagioslogserver/elasticsearch/data as slansing correctly pointed out originally. Especially if logrotate is used with the logstash log, I would not expect it or it's rotations to grow too out of control.
Re: Nagios Log Server on RHEL 6
Posted: Tue Dec 02, 2014 5:09 pm
by bdgoecke
For point of reference, on my test log server I have 3 lightly used linux machines loging to the server for a month or so..
_Cluster_Statistics_
Code: Select all
1,357,738 Documents
287MB Primary Size
287MB Total Size
1 Data Instances
352 Total Shards
36 Indices
_Disk_Usage_
[@LogServer-centos65-64bit /]$ sudo du -sh *
Code: Select all
5.8M bin
88M boot
220K dev
29M etc
488M home
448M lib
22M lib64
16K lost+found
4.0K media
4.0K mnt
4.0K opt
179M root
14M sbin
4.0K selinux
4.0K srv
12K store
0 sys
420K tmp
1.4G usr
346M var
[@LogServer-centos65-64bit /]$ cd /var
[@LogServer-centos65-64bit var]$ sudo du -sh *
Code: Select all
53M cache
24K db
8.0K empty
4.0K games
51M lib
4.0K local
16K lock
226M log
0 mail
4.0K nis
4.0K opt
4.0K preserve
104K run
112K spool
4.0K tmp
18M www
4.0K yp
[@LogServer-centos65-64bit var]$ cd /usr
[@LogServer-centos65-64bit usr]$ sudo du -sh *
Code: Select all
30M bin
4.0K etc
4.0K games
17M include
232M lib
109M lib64
17M libexec
695M local
29M sbin
292M share
12K src
0 tmp