Page 1 of 1
Nagios Logging - archiving and compression
Posted: Wed Jun 24, 2020 9:29 am
by songviet
Hi support team,
We need your help to support us on how Nagios archive and store logs more efficiently (via compression, archiving, whatever).
- Log retention (keep last XX days of logs)
- Log archiving interval: daily, or weekly..
Our logs are growing pretty fast and we need to figure out how to reduce the amount of disk space they are using.
Currently, I can see default location is under path:/usr/local/nagioslogserver/elasticsearch/data/
Thanks in advance.
Re: Nagios Logging - archiving and compression
Posted: Wed Jun 24, 2020 4:24 pm
by ssax
Re: Nagios Logging - archiving and compression
Posted: Wed Jun 24, 2020 11:22 pm
by songviet
Thank you, ssax for your support.
I am configuring New Repositoty and unfortunately error message said :
The snapshot repository in directory /mnt/snapshot_repository could not be created. Possibly due to permissions. Make sure the nagios user can write to the location.
In meantime, directory permission already 775. Out Nagios log server has already joined to primary cluster, does it main reason may cause?
Re: Nagios Logging - archiving and compression
Posted: Thu Jun 25, 2020 3:50 pm
by cdienger
Each machine in the cluster would need to have the same mount. Is this the case? If so, are you able to do a directory listing:
Code: Select all
su - nagios
ll /mnt/snapshot_repository
?
I would also like to point out that the location of the database can be changed -
https://assets.nagios.com/downloads/nag ... Server.pdf
Re: Nagios Logging - archiving and compression
Posted: Fri Jun 26, 2020 3:40 am
by songviet
Hi,
I am not sure we are navigating right way. With your official guidelines, there looks messy for me.
We just 2 concerns:
-Retention: maybe Nagios UI can help?
-Archiving: how? to reduce disk space efficiently?
With Node-Cluster mode, how do we do for both side? Is there mounting (/mnt/) primary cluster for all nodes?
Re: Nagios Logging - archiving and compression
Posted: Fri Jun 26, 2020 4:11 pm
by cdienger
Retention settings for data are found under Admin > System > Snapshots & Maintenance. "Delete indexes older than" controls how many days worth of data will be saved on the NLS instance.
You can create a repository on the same screen. If you have a cluster of NLS nodes then you will need to make sure that all nodes in the cluster mount to the same location.
Once a repo is configured you will have options to store snapshots(archives) to the repo. You will also be given the "Delete snapshots older than" option which controls how old a snapshot can be before it is removed from the repo. Repos are meant to hold data that you'd like to save but don't necessary need to search on a day to day basis. You can use them to hold old data and then restore that data if needed.
Elasticsearch takes care of compressing the data. There is no need to compress it any further then what is done automatically.
Re: Nagios Logging - archiving and compression
Posted: Sun Jun 28, 2020 9:21 pm
by songviet
Thank you.
So just one thing I need to do is: create repository, mount all nodes to primary cluster and configure storing snapshots in Admin UI.
One more question, can I use 3rd app (like 7zip to zip these previous logs folder on each local NLS node?), does it affect to NLS system?
Re: Nagios Logging - archiving and compression
Posted: Mon Jun 29, 2020 9:39 am
by cdienger
The system isn't expecting a zip file so it wouldn't be able to do anything with them until they were unzipped.
Re: Nagios Logging - archiving and compression
Posted: Tue Jun 30, 2020 4:11 am
by songviet
Thanks.
Please help to close this topic -- any further concerns, I will open new thread.
Have a nice day all!
Re: Nagios Logging - archiving and compression
Posted: Tue Jun 30, 2020 8:08 am
by scottwilkerson
songviet wrote:Thanks.
Please help to close this topic -- any further concerns, I will open new thread.
Have a nice day all!
Great!
Locking thread