Nginx reverse proxy on another server


#1

Hi,

I would like to setup a Mastodon instance on a server running Proxmox.
I’ve already installed Mastodon inside an LXC container.

But I don’t know how to configure Nginx, running on another container, as a reverse proxy for this instance.

Here is a schematic :
INTERNET ====> 10.129.0.10 (nginx reverse proxy) ======> 10.129.0.103 (mastodon instance)

The documentation explains how to setup nginx for a local reverse proxy, but I can’t find information on how to set it up when it is running on another machine.

Can you help me on this topic?

Thanks!


#2

nginx passes all requests to 10.129.0.103, where Mastodon serves static and dynamic content

If you want just to proxy all traffic from the Internet via nginx - just use generic nginx reverse proxy instructions - there is nothing specific to Mastodon there, except maybe that you might want to terminate your SSL on the reverse proxy and have your SSL certificate on the proxy (Example setup for Jenkins).

For this, you will need to run nginx on 10.129.0.103 as well.

nginx 10.129.0.10 serves static files, Mastodon 10.129.0.103 does dynamic content only

If your nginx proxy can have access to Mastodon static files, you can indeed serve them on the proxy itself and just proxy port 3000 and 4000 to your instance by changing

 proxy_pass http://127.0.0.1:3000;
 proxy_pass http://127.0.0.1:4000;

to

 proxy_pass http://10.129.0.103:3000;
 proxy_pass http://10.129.0.103:4000;

But for this you need to have /home/mastodon/live/public available from 10.129.0.103, for example by NFS (a read only mount is enough).

In the second case, you don’t need to run nginx on 10.129.0.103 at all.


#3

@saper Thank you very much for your answer, it seems to be exactly what I’m looking for.

I think that the first solution would be the easiest one, as all the Mastodon file would be stored on the mastodon container, and Nginx would handle the SSL certificate the same way than for the other certificates I have.

I suppose I’ll need to modify the Nginx configuration from the mastodon container so that it doesn’t handle SSL ? I’ll have a look in the Jenkins example.

Thanks!


#4

You don’t need to. It can be left as-is, you can even forward to the SSL port if you really like.


#5

@saper I’ve continued my testing, but I do not find a working solution. I’m not an expert with Nginx, and I don’t know how to configure both Nginx (on the reverse proxy side, and on the Mastodon side).

I think that the best solution would be to handle the HTTPS connection on the reverse proxy, and only use HTTP internally between the proxy and mastodon. But for this, I need to modify the configuration file from Mastodon, and I don’t know how.

It should be simpler to forward the SSL connection to the mastodon VM but… how?

Here is my latest test:

Reverse proxy :

    server {
        listen       80;
        listen       [::]:80;
        server_name  mastodon.test.com;
        location / { return 301 https://$host$request_uri; }
    }
    
    
    server {
        listen 443;
        listen [::]:443;
    
        ssl_certificate /etc/letsencrypt/live/mastodon.test.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/mastodon.test.com/privkey.pem;
    
        ssl on;
        ssl_session_cache  builtin:1000  shared:SSL:10m;
        ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
        ssl_prefer_server_ciphers on;
        server_name mastodon.test.com;
    
        location / {
            proxy_pass https://10.129.0.103:443;
        }
    }

On the Mastodon side:

    map $http_upgrade $connection_upgrade {
      default upgrade;
      ''      close;
    }

    server {
      listen 80;
      listen [::]:80;
      server_name 10.129.0.103;
      # Useful for Let's Encrypt
      location /.well-known/acme-challenge/ { allow all; }
      location / { return 301 https://$host$request_uri; }
    }

    server {
      listen 443 ssl http2;
      listen [::]:443 ssl http2;
      server_name 10.129.0.103;

      ssl_protocols TLSv1.2;
      ssl_ciphers HIGH:!MEDIUM:!LOW:!aNULL:!NULL:!SHA;
      ssl_prefer_server_ciphers on;
      ssl_session_cache shared:SSL:10m;

      ssl_certificate     /etc/letsencrypt/live/mastodon.test.com/fullchain.pem;
      ssl_certificate_key /etc/letsencrypt/live/mastodon.test.com/privkey.pem;

      keepalive_timeout    70;
      sendfile             on;
      client_max_body_size 0;

      root /home/mastodon/live/public;

      gzip on;
      gzip_disable "msie6";
      gzip_vary on;
      gzip_proxied any;
      gzip_comp_level 6;
      gzip_buffers 16 8k;
      gzip_http_version 1.1;
      gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

      add_header Strict-Transport-Security "max-age=31536000";

      location / {
        try_files $uri @proxy;
      }
      location ~ ^/(emoji|packs|system/accounts/avatars|system/media_attachments/files) {
        add_header Cache-Control "public, max-age=31536000, immutable";
        try_files $uri @proxy;
      }

      location @proxy {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header Proxy "";
        proxy_pass_header Server;

        proxy_pass http://127.0.0.1:3000;
        proxy_buffering off;
        proxy_redirect off;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;

        tcp_nodelay on;
      }

      location /api/v1/streaming {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header Proxy "";
     
        proxy_pass http://127.0.0.1:4000;
        proxy_buffering off;
        proxy_redirect off;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;

        tcp_nodelay on;
      }

      error_page 500 501 502 503 504 /500.html;
    }

With this example, my browser tries to connect to 10.129.0.103 (the internal IP of the Mastodon VM), with no luck.
And there is an issue : both VM manage the SSL connection, and have a copy of the certificate. That doesn’t seem right to me.

Any Nginx Guru to help me?

Thanks!


#6

Get rid of the server clause or port 80 and change all instances of 443 to 80 in the SSL one, removing SSL configuration.
Right now your Mastodon VM nginx redirects you from 80 to 443 all the time.


#7

I finally manage to set up Nginx correctly, thanks for your help!

Here are my configuration files:

On the reverse proxy:

server {
    listen       80;
    listen       [::]:80;
    server_name  mastodontest.example.com;
    location / { return 301 https://$host$request_uri; }
}


server {
    listen 443;
    listen [::]:443;

    ssl_certificate /etc/letsencrypt/live/mastodontest.codingfield.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/mastodontest.codingfield.com/privkey.pem;

    ssl on;
    ssl_session_cache  builtin:1000  shared:SSL:10m;
    ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
    ssl_prefer_server_ciphers on;


    server_name mastodontest.example.com;

    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_pass http://10.129.0.103:80;
    }

}

One the Mastodon instance:

map $http_upgrade $connection_upgrade {
  default upgrade;
  ''      close;
}

server {
  listen 80;
  listen [::]:80;
  server_name 10.129.0.103;

  keepalive_timeout    70;
  sendfile             on;
  client_max_body_size 0;

  root /home/mastodon/live/public;

  gzip on;
  gzip_disable "msie6";
  gzip_vary on;
  gzip_proxied any;
  gzip_comp_level 6;
  gzip_buffers 16 8k;
  gzip_http_version 1.1;
  gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

  add_header Strict-Transport-Security "max-age=31536000";

  location / {
    try_files $uri @proxy;
  }

  location ~ ^/(emoji|packs|system/accounts/avatars|system/media_attachments/files) {
    add_header Cache-Control "public, max-age=31536000, immutable";
    try_files $uri @proxy;
  }
 location @proxy {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header Proxy "";
    proxy_pass_header Server;

    proxy_pass http://127.0.0.1:3000;
    proxy_buffering off;
    proxy_redirect off;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    tcp_nodelay on;
  }

  location /api/v1/streaming {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header Proxy "";

    proxy_pass http://127.0.0.1:4000;
    proxy_buffering off;
    proxy_redirect off;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    tcp_nodelay on;
  }

  error_page 500 501 502 503 504 /500.html;
}

#8

This doens’t appear to still work. There’s a comment in the .env.production to the effect that Mastodon will not run in production unless it is the HTTPS termination system.

Is there a way to run Mastodon as HTTP only, and terminate all ssl at the reverse proxy?


#9

Yes, I run it this way, so it is possible. HTTP is even running on a different host.