Anyone using Amazon S3 for cluster repository?

[vAJ]

Trying to come up with options for repository for our geo-clustered NLS.

Once thought was to use S3FS and mount the same bucket to all cluster nodes for backup of data that we need to archive and move out of online index storage.

Thoughts?

[hsmith]

How much storage is available on that server? How much data worth of logs are you receiving per day? Cost would be my consideration here.

[vAJ]

I have a few hundred gigs of SAN storage attached to the cluster node VMs. But I have conversations going on now where we might need to retain logs for years (compliance).

Once ramped up in production, we could be looking at 50GB/day.

[hsmith]

That's a lot of logs. If you have the budget for the cloud storage, I'm sure it will work fine.

[vAJ]

Hopefully I get to talk about it all in September

[hsmith]

Better you than me