Shared S3 media across instances

I’m planning on setting up a couple Mastodon instances and I was wondering if I could point them to the same S3 bucket for media and profile assets. What I want to accomplish mainly is data not being duplicated. I don’t want each instance to have their own copy of an image from some federated post, I just want them all to reference the same image in some bucket.

Is this how S3 storage in Mastodon functions by default, or if not is there a way to accomplish this? I suspect masto.host does this with their own asset domain but I wasn’t sure if that was built-in functionality or something special they set up for themselves.

no, there’s no current built-in way to do deduplication with the stock s3 adapter. the s3 API is so ubiquitous and standard that I’m guessing there’s probably a gem or compatibility layer you can plug in to do this for you, but I don’t know of any off hand

How is Mastodon (or Paperclip) determining a file name currently? If that process is deterministic, one could end up having files stored only once if the filename is the same?