Accounts directory cleanup

Hey fellow admins,

is there a good way to free space in public/system/accounts ? It’s 9 GB by now and keeps growing, so I’d need to clean up in order to keep my instance stable.

tootctl media remove doesn’t help and tootctl preview_cards neither.

I tried to find and rm old files by hand but that almost broke a lot. Damage could only be fixed with tootctl accounts refresh --all , which unfortunately brought all files back - so actually no space was freed eventually.

Can anyone help? Thanks in advance!

#Mastodon #mastoadmin #rt #help

1 Like

I have the same question. Running bin/tootctl media usage in our instance shows

Avatars: 4 GB (14.7 MB local)
Headers: 8.33 GB (25.7 MB local)

Dedicating all these +12 GB of our precious disk space to remote avatars and headers sounds wrong, especially when there doesn’t seem to be a way to remove them cleanly.

(Also, can someone move this topic to #instance-admin, please?)

I have found an old blog post from an Mastodon admin (@Angristan) where the solution suggested is to… buy more storage.

Alright, I see how that would work. :slight_smile: However, I still wonder. How can we purge “old” Avatars and Headers from other instances? Or delete them manually without breaking things.

Ah, I just found this topic where @dustinwilson and @vibhigupta had the same problem last year:

The situation in our server is becoming unsustainable. I am rescuing our server from full disc space almost on a daily basis. media remove and preview_cards remove alone won’t cut it, GBs removed with this tools will be refilled within hours. The situation is so bad that I am running media remove --days 3 and even that doesn’t help hitting disc space limit in less than 24 hours.

Meanwhile, all those GBs are sitting in /accounts. Today I found our instance down again with a server full. After seeing what I could cut cleanly I found nothing, and I couldn’t even run tootctl because of lack of space. So I finally did it:

find /home/mastodon/live/public/system/accounts/headers/000/* -mindepth 1 -maxdepth 1 | xargs rm -rf

Thanks to this, the partition went to 81% and the server back to live. There are two problems, though.

  1. The headers are now broken, of course.
  2. media usage still shows 8GB under /headers, which is confusing.

And a third problem is that I suspect soon the Fediverse will fill our disk space again with federated content. Maybe we have just too many spam / inactive accounts. @dustinwilson this operation looks scary but it worked for you:

Would you recommend it or is there a most recent or safer method now?

Sorry about the late reply. I don’t think there’s a safer method to do this yet. This is at present the command I have my server run to get the list of stale accounts. I think it’s a bit different than the one in the old post. I’ve changed it to one month for my purposes as I’m just a wee tiny server.

sudo -i -u postgres /bin/bash -l -c "psql -A -d mastodon -c \"SELECT username||'@'||domain FROM public.accounts WHERE last_webfingered_at < (CURRENT_TIMESTAMP - interval '1 month') AND id NOT IN (SELECT target_account_id FROM public.follows WHERE account_id IN (SELECT id FROM public.users)) AND id NOT IN (SELECT account_id FROM public.follows WHERE target_account_id IN (SELECT id FROM public.users))\"" | tail -n +2 | head -n -1

Then on each I run this which again is a bit different than before:

begin
    a = Account.find_by(username: "username", domain: "domain")
    unless a.nil?
        a.destroy
    end
rescue => err
end

This has been running daily ever since that last post here without any issues AFAIK. I am only an administrator of my own single user server, so not sure if there’s more to take into account when wanting to get rid of old accounts. I’d test it on a duplicate if possible before culling old accounts. tootctl should have a command for this, and if I remember correctly there’s a github issue for it that’s quite old. But yeah, the official solution seems to be to buy more storage. It’s BS.

1 Like

Wouldn’t remote accounts get re-created every time some new stuff comes from them?