Cluster backup repository?

[gsl_ops_practice]

Hello,

Apologies if this has been covered elsewhere but I was not able to find a conclusive answer in this forum so far.

We have a 2-node cluster, everything is working perfectly. Since the configuration is identical on both nodes, the backups are currently done to an individual local file system on both nodes - this means we need double the storage space for all the backups.

Can you please help me understand how we can have a multi-node cluster but only use 1x the amount of space for the backups, and not Nx (where N is the number of nodes in the cluster).

Thanks,
Alex

[rkennedy]

You could use any sort of shared storage for this. Just need to setup a mount, and point your backups to that shared repository. The one important thing is make sure it's writable by the nagios user on both systems.

[gsl_ops_practice]

I would like to clarify - that means both nodes will be executing the backups to the same shared directory and will use only 1x the storage and not 2x? This part is not explicitly spelled out anywhere. Will each node create its own directory in the backup location?

Thanks,
Alex

[rkennedy]

I just tried to lab up your environment, and was unable to get curator to work the way your system does. Since NLS1 can't write to /nlsbackup on NLS2 I expected it would fail, and it did. I'm unsure how your system is currently setup.

Code: Select all

2016-08-09 11:38:15,903 ERROR     Failed to verify all nodes have repository access.

Are you sure that /nagioslog_backup isn't a mounted folder already? Could you please run mount -ls | grep nagios on both machines and post the output?

[gsl_ops_practice]

Hi,

Each node of the cluster has a purely local filesystem called /nagioslog_backup. Each node therefore has their own independent copy of all the backups. I hope this clarifies the issue.

If we have a shared NFS volume to which both log servers perform their backups, will we have only 1 copy of the backup or one copy per cluster node?

Thanks,
Alex

[rkennedy]

If we have a shared NFS volume to which both log servers perform their backups, will we have only 1 copy of the backup or one copy per cluster node?

This wording helps, it would be a total of one backup.

[gsl_ops_practice]

Great, thank you.

To migrate from the current local backups, I would presume the following needs to happen:

1. back up current /nagioslog_backup on both nodes while maintaining correct permissions
2. remove /nagioslog_backup from both nodes
3. create NFS mount /nagioslog_backups with appropriate nagios:nagios permissions
4. restore from backup while maintaining correct permissions

Can you please confirm?

Thanks,
Alex

[rkennedy]

I'm a bit confused by your wording, because you won't need to do a restore at all. I would just create a new backup repository, after you have created the NFS share. Let it be a new path '/nls_backups' for example. Then, once you've determined that the backup process is indeed working and has everything once again (give it a week or two), remove the /nagioslog_backups folders.

You are correct about the permissions though.

[gsl_ops_practice]

Hello,

I do need to keep the backups that we currently have, to be accessible to both nodes. This is the reason for the backup and restore.

[rkennedy]

Got it - then moving the backup files from nagioslog_backups to nls_backups (using my example previously) should work once they are setup.