Page 1 of 2

Cluster backup repository?

Posted: Tue Aug 09, 2016 8:36 am
by gsl_ops_practice
Hello,

Apologies if this has been covered elsewhere but I was not able to find a conclusive answer in this forum so far.

We have a 2-node cluster, everything is working perfectly. Since the configuration is identical on both nodes, the backups are currently done to an individual local file system on both nodes - this means we need double the storage space for all the backups.

Can you please help me understand how we can have a multi-node cluster but only use 1x the amount of space for the backups, and not Nx (where N is the number of nodes in the cluster).

Thanks,
Alex

Re: Cluster backup repository?

Posted: Tue Aug 09, 2016 9:20 am
by rkennedy
You could use any sort of shared storage for this. Just need to setup a mount, and point your backups to that shared repository. The one important thing is make sure it's writable by the nagios user on both systems.

Re: Cluster backup repository?

Posted: Tue Aug 09, 2016 9:44 am
by gsl_ops_practice
I would like to clarify - that means both nodes will be executing the backups to the same shared directory and will use only 1x the storage and not 2x? This part is not explicitly spelled out anywhere. Will each node create its own directory in the backup location?

Thanks,
Alex

Re: Cluster backup repository?

Posted: Tue Aug 09, 2016 10:49 am
by rkennedy
I just tried to lab up your environment, and was unable to get curator to work the way your system does. Since NLS1 can't write to /nlsbackup on NLS2 I expected it would fail, and it did. I'm unsure how your system is currently setup.

Code: Select all

2016-08-09 11:38:15,903 ERROR     Failed to verify all nodes have repository access.
Are you sure that /nagioslog_backup isn't a mounted folder already? Could you please run mount -ls | grep nagios on both machines and post the output?

Re: Cluster backup repository?

Posted: Tue Aug 09, 2016 12:09 pm
by gsl_ops_practice
Hi,

Each node of the cluster has a purely local filesystem called /nagioslog_backup. Each node therefore has their own independent copy of all the backups. I hope this clarifies the issue.

If we have a shared NFS volume to which both log servers perform their backups, will we have only 1 copy of the backup or one copy per cluster node?

Thanks,
Alex

Re: Cluster backup repository?

Posted: Tue Aug 09, 2016 12:18 pm
by rkennedy
gsl_ops_practice wrote: If we have a shared NFS volume to which both log servers perform their backups, will we have only 1 copy of the backup or one copy per cluster node?
This wording helps, it would be a total of one backup.

Re: Cluster backup repository?

Posted: Tue Aug 09, 2016 12:23 pm
by gsl_ops_practice
Great, thank you.

To migrate from the current local backups, I would presume the following needs to happen:

1. back up current /nagioslog_backup on both nodes while maintaining correct permissions
2. remove /nagioslog_backup from both nodes
3. create NFS mount /nagioslog_backups with appropriate nagios:nagios permissions
4. restore from backup while maintaining correct permissions

Can you please confirm?

Thanks,
Alex

Re: Cluster backup repository?

Posted: Tue Aug 09, 2016 4:39 pm
by rkennedy
I'm a bit confused by your wording, because you won't need to do a restore at all. I would just create a new backup repository, after you have created the NFS share. Let it be a new path '/nls_backups' for example. Then, once you've determined that the backup process is indeed working and has everything once again (give it a week or two), remove the /nagioslog_backups folders.

You are correct about the permissions though.

Re: Cluster backup repository?

Posted: Wed Aug 10, 2016 7:51 am
by gsl_ops_practice
Hello,

I do need to keep the backups that we currently have, to be accessible to both nodes. This is the reason for the backup and restore.

Re: Cluster backup repository?

Posted: Wed Aug 10, 2016 12:57 pm
by rkennedy
Got it - then moving the backup files from nagioslog_backups to nls_backups (using my example previously) should work once they are setup.