Mastodon with CDN - A problem about web socket connections


#1

This is @haisenberg@cmx.im, admin of Chinese Mastodon Community Instance.

After Sina Weibo changed it’s EULA, we’ve been having massive amount of traffic from Chinese speakers who chose to migrate to Mastodon. While the new users generated a certain amount of the traffic, the main pressure comes from Pawoo.net since the first batch of users who migrated share the same interest – Japanese anime, we now have to serve lots of toots and images from pawoo.net.

This has forced us to upgrade the server three times during the last couple weeks, but the performance is still not optimal. Our server is located in Hongkong. Hongkong is an ideal choice since it’s fast to access from mainland China, and this way we can avoid many potential legal problems.

The thing is, the bandwidth there is very expensive, and we need lots of bandwidth to serve all these Chinese users. We tried S3 but it’s too expensive, building a Minio server is viable but it should locate in Hongkong for faster connection, and we still need to pay for the bandwidth.

That left us with the only obvious option – CDN. Unfortunately our CDN service provider does not support WSS protocol, and before you ask, Cloudflare does support WSS but it’s blocked by GFW (The Great Firewall).


As illustrated in the image above, unable to connect to web socket makes both web client and apps refuse to work.

Because of the GFW, our choice of CDN providers are limited, and AFAIK, none of them supports WSS protocol.

The pressure mainly comes from media files, I can come up with the following solutions,

  1. Somehow separate static contents to another domain (assets.cmx.im) and use CDN for this site only.
  2. Setup a Minio server to host all media files and use CDN for this Minio instance only.

The 2nd solution is more straight-forward but requires another server with at least decent bandwidth, since our fund is limited, I prefer the first solution but have no idea how to do it.

Please advise. Any help is appreciated.

Haisenberg


#2

Can you check if setting CDN_HOST in the .env.production file can let you setup variant 1?