Docker-compose: redis container won't connect

Server administration Troubleshooting
#1

I’ve followed every step to get this docker-compose setup going, yet for some reason stdout gets flooded with connection errors regarding redis:

streaming_1  | info Starting worker 5 
streaming_1  | info Worker 5 now listening on 0.0.0.0:4000 
streaming_1  | ERR! Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED 127.0.0.1:6379 
streaming_1  | info Worker 5 exiting, bye bye

I don’t understand why this can’t connect: the redis container is starting just fine. However, after i run docker-compose run redis, the container runs fine but I am not able to see port 6379 exposed on 127.0.0.1, from a different terminal.

What am I missing here? I did have a systemd redis service running on the host, which I stopped cuz I was worried it might cause port conflicts (the host service isn’t doing anything anyway, it’s for an old program i don’t run anymore). The only change I made to docker-compose.yml is to change web to use port 3333 instead of 3000 (i have something running on that already).

docker-compose.yml

version: '3'
services:

  db:
    restart: always
    image: postgres:9.6-alpine
    shm_size: 256mb
    networks:
      - internal_network
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "postgres"]
    volumes:
      - ./postgres:/var/lib/postgresql/data

  redis:
    restart: always
    image: redis:6.0-alpine
    networks:
      - internal_network
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
    volumes:
      - ./redis:/data

#  es:
#    restart: always
#    image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.10
#    environment:
#      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
#      - "cluster.name=es-mastodon"
#      - "discovery.type=single-node"
#      - "bootstrap.memory_lock=true"
#    networks:
#      - internal_network
#    healthcheck:
#      test: ["CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1"]
#    volumes:
#      - ./elasticsearch:/usr/share/elasticsearch/data
#    ulimits:
#      memlock:
#        soft: -1
#        hard: -1

  web:
    image: tootsuite/mastodon
    restart: always
    env_file: .env.production
    command: bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000"
    networks:
      - external_network
      - internal_network
    healthcheck:
      test: ["CMD-SHELL", "wget -q --spider --proxy=off localhost:3000/health || exit 1"]
    ports:
      - "127.0.0.1:3333:3000"
    depends_on:
      - db
      - redis
#      - es
    volumes:
      - ./public/system:/mastodon/public/system

  streaming:
    image: tootsuite/mastodon
    restart: always
    env_file: .env.production
    command: node ./streaming
    networks:
      - external_network
      - internal_network
    healthcheck:
      test: ["CMD-SHELL", "wget -q --spider --proxy=off localhost:4000/api/v1/streaming/health || exit 1"]
    ports:
      - "127.0.0.1:4000:4000"
    depends_on:
      - db
      - redis

  sidekiq:
    image: tootsuite/mastodon
    restart: always
    env_file: .env.production
    command: bundle exec sidekiq
    depends_on:
      - db
      - redis
    networks:
      - external_network
      - internal_network
    volumes:
      - ./public/system:/mastodon/public/system
## Uncomment to enable federation with tor instances along with adding the following ENV variables
## http_proxy=http://privoxy:8118
## ALLOW_ACCESS_TO_HIDDEN_SERVICE=true
#  tor:
#    image: sirboops/tor
#    networks:
#      - external_network
#      - internal_network
#
#  privoxy:
#    image: sirboops/privoxy
#    volumes:
#      - ./priv-config:/opt/config
#    networks:
#      - external_network
#      - internal_network

networks:
  external_network:
  internal_network:
    internal: true

edit: and here’s .env.production (only edits are: domain, keys, and disabling ES and S3)

# Federation
# ----------
# This identifies your server and cannot be changed safely later
# ----------
LOCAL_DOMAIN=mastodon.domain.tld

# Redis
# -----
REDIS_HOST=localhost
REDIS_PORT=6379

# PostgreSQL
# ----------
DB_HOST=/var/run/postgresql
DB_USER=mastodon
DB_NAME=mastodon_production
DB_PASS=<dbpass>
DB_PORT=5432

# ElasticSearch (optional)
# ------------------------
ES_ENABLED=false
ES_HOST=localhost
ES_PORT=9200

SECRET_KEY_BASE=<privkey>
OTP_SECRET=<privkey>

# Web Push
# --------
# Generate with `rake mastodon:webpush:generate_vapid_key`
# --------
VAPID_PRIVATE_KEY=<privkey>
VAPID_PUBLIC_KEY=<pubkey)
# Sending mail
# ------------
SMTP_SERVER=smtp.mailgun.org
SMTP_PORT=587
SMTP_LOGIN=
SMTP_PASSWORD=
SMTP_FROM_ADDRESS=notificatons@example.com

# File storage (optional)
# -----------------------
S3_ENABLED=false
S3_BUCKET=files.example.com
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
S3_ALIAS_HOST=files.example.com
#2

Also, I’m getting db errors as well? it’s confusing since the docker instructions (edit: I guess docs.joinmastodon.com doesn’t have official docker instructions? looking around the internet it seems like they used to exist, but https://github.com/tootsuite/documentation/blob/master/Running-Mastodon/Docker-Guide.md gives a 404 and doesn’t seem to exist on the document site) don’t give any indication of having to initialize the db? i thought that was handled by the Dockerfile. (note: all the passwords and variables are defined in .env.production, as indicated above)

db_1         | Error: Database is uninitialized and superuser password is not specified.
db_1         |        You must specify POSTGRES_PASSWORD to a non-empty value for the
db_1         |        superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
db_1         | 
db_1         |        You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
db_1         |        connections without a password. This is *not* recommended.
db_1         | 
db_1         |        See PostgreSQL documentation about "trust":
db_1         |        https://www.postgresql.org/docs/current/auth-trust.html
#3

This should be redis, not localhost

#4

The same for this, in docker you need to reference the docker container name as the hostname, in your case db is the hostname

#5

Yup I figured this out. I should have updated this thread to reflect that, but I’d forgotten about it.

#6

@winteriscariot can you post your working config for others to see?

#7

Sure. A few things about it:

You’ll note the ‘env_file’ option that I added to the ‘db’ service. This is required because the postgres db doesn’t properly initialize the way it is, the proper variables don’t get passed. You’ll find the .env.db file listed after the docker-compose.yml file (it’s short). Also, ElasticSearch is disabled.

EDIT: Also, I didn’t write up a complete how-to, so I don’t have the steps to generate the secrets required. There are plenty of tutorials online on how to do that which are easy enough to find.

docker-compose.yml

version: '3'
services:

  db:
    restart: always
    image: postgres:9.6-alpine
    env_file: .env.db
    shm_size: 256mb
    networks:
      - internal_network
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "postgres"]
    volumes:
      - ./postgres:/var/lib/postgresql/data

  redis:
    restart: always
    image: redis:6.0-alpine
    networks:
      - internal_network
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
    volumes:
      - ./redis:/data

  web:
    build: .
    image: tootsuite/mastodon
    restart: always
    env_file: .env.production
    command: bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000"
    networks:
      - external_network
      - internal_network
    healthcheck:
      test: ["CMD-SHELL", "wget -q --spider --proxy=off localhost:3000/health || exit 1"]
    ports:
      - "127.0.0.1:3333:3000"
    depends_on:
      - db
      - redis
    volumes:
      - ./public/system:/mastodon/public/system

  streaming:
    build: .
    image: tootsuite/mastodon
    restart: always
    env_file: .env.production
    command: node ./streaming
    networks:
      - external_network
      - internal_network
    healthcheck:
      test: ["CMD-SHELL", "wget -q --spider --proxy=off localhost:4000/api/v1/streaming/health || exit 1"]
    ports:
      - "127.0.0.1:4000:4000"
    depends_on:
      - db
      - redis

  sidekiq:
    build: .
    image: tootsuite/mastodon
    restart: always
    env_file: .env.production
    command: bundle exec sidekiq
    depends_on:
      - db
      - redis
    networks:
      - external_network
      - internal_network
    volumes:
      - ./public/system:/mastodon/public/system

networks:
  external_network:
  internal_network:
    internal: true

.env.db (note: these vars should match the DB_USER, DB_NAME and DB_PASS from .env.production):

POSTGRES_PASSWORD=<secret>
POSTGRES_DB=mastodon_production
POSTGRES_USER=mastodon

.env.production

# Federation
# ----------
# This identifies your server and cannot be changed safely later
# ----------
LOCAL_DOMAIN=<mastodon.domain.tld>

# Redis
# -----
REDIS_HOST=redis
REDIS_PORT=6379

# PostgreSQL
# ----------
DB_HOST=db
DB_USER=mastodon
DB_NAME=mastodon_production
DB_PASS=<secret>
DB_PORT=5432

# ElasticSearch (optional)
# ------------------------
ES_ENABLED=false

# Secrets
# -------
# Make sure to use `rake secret` to generate secrets
# -------
SECRET_KEY_BASE=<secret>
OTP_SECRET=<secret>

# Web Push
# --------
# Generate with `rake mastodon:webpush:generate_vapid_key`
# --------
VAPID_PRIVATE_KEY=<secret>
VAPID_PUBLIC_KEY=<secret>

# Sending mail
# Removed because I'm creating and approving all users manually via cli, 
# cuz i don't have an smtp server


# File storage (optional)
# -----------------------
S3_ENABLED=false

And then just run docker-compose run --rm web rails db:migrate to populate the database.