How to faster crawl link previews


#1

When looking in Sidekiq in the scheduled tab I see around 30-40 LinkCrawlWorkers. They oldest almost a minute old.

How to make sure links are faster crawled? Cause when a new toot arrives on a timeline, it hasnt’t always a preview.


#2

These workers are delayed by a random amount of seconds on purpose to avoid DDoSing the link when thousands of Mastodon servers receive the same toot.


#3

I know why they are delayed, but I was wondering if it was possible to decrease the delay? Cause I notice sometimes there are no previews in some toots and IMO that’s a bad UX.