Snapshot storage maintenance

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
rferebee
Posts: 733
Joined: Wed Jul 11, 2018 11:37 am

Snapshot storage maintenance

Post by rferebee »

Hello,

Is there any maintenance that can be ran on the storage location where my Log Server snapshots are stored?

This might have more to do with my storage than with Log Server, but I've noticed in the last couple weeks that when I delete an old snapshot that it won't free up any space in the repository where they are stored. It's really weird because I know the snapshot is like 300-400GB+ in size, but the amount of free space will remain the same after it's deleted.

It happened this morning after I deleted yesterdays snapshot. Not sure what that's about.

It looks like the metadata-curator and snapshot-curator files are up to date.

Thank you.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Snapshot storage maintenance

Post by scottwilkerson »

rferebee wrote:Is there any maintenance that can be ran on the storage location where my Log Server snapshots are stored?
Not really.

It is worth pointing out that each snapshot only contains the difference from the previous snapshotted indices and any new indexes.

So, each snapshot isn't the whole of all of the indices, and deleting one snapshot may not make much difference if any, it would only remove the amount of data that is in any index that are not also contained in any other snapshots.
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
rferebee
Posts: 733
Joined: Wed Jul 11, 2018 11:37 am

Re: Snapshot storage maintenance

Post by rferebee »

I understand that, I guess why I'm confused is because on average we get around 250GB+ worth of logs everyday. I would think that deleting the snapshot would remove at least that much data.

For example, before the snapshot was taken last night we had 1345.92GB of free space and after it finished we had 1069.40GB of free space. So, deleting the snapshot from the prior day should free up the space taken by the index the day before it was taken since that data is now contained in today's snapshot, right?
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Snapshot storage maintenance

Post by scottwilkerson »

rferebee wrote:So, deleting the snapshot from the prior day should free up the space taken by the index the day before it was taken since that data is now contained in today's snapshot, right?
No. Because today's snapshot is told that it needs a copy of all of the indices that were active at the time the snapshot was created.

Now when you delete yesterdays snapshot the system knows that it was the only other snapshot that contained yesterdays index, and as such it transfers that data to today snapshot because you will still need it if you restore todays snapshot.

Now, if you removed the OLDEST snapshot on the system, it most certainly should contain indices that no other snapshots contain and as such they would be completely removed from the drive, and it would free up the space they were taking, and they would be gone forever.
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
rferebee
Posts: 733
Joined: Wed Jul 11, 2018 11:37 am

Re: Snapshot storage maintenance

Post by rferebee »

Ok, I understand.

Thank you for clarifying. You can lock this.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Snapshot storage maintenance

Post by scottwilkerson »

rferebee wrote:Ok, I understand.

Thank you for clarifying. You can lock this.
Great!

Locking
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
Locked