Deploy a Whatsapp matrix bridge on Synapse

So you have a Pinephone https://wiki.pine64.org/index.php/PinePhone - a SOC with an integrated display and a radio, which runs linux, but not a smartphone (seriously, NOT a smartphone!), and you want to use Whatsapp from there.

Ahahahahahah

This is the echoes of the loughs which you’ll hear, usually laughs of desperation.

Unfortunately Linux phones systems are not nearly as mature as the oldest android/windows CE phone which you used to use - but Whatsapp (or any other messenger) might come in help and give you that feel of comfort. It could really bring some light in the sad panorama of usability in the portable linux devices.

I’m pretty sure is plenty of people who is able to live with their Pinephones (see the plural), unfortunately I’m not (yet). For all of you who will feel offended by my words, I’m so happy that you are happy with your crappy phones usability. Pinephone, as a phone, is as useful as a rotary dial phone in the year 2023. Better go for a banana phone https://www.nokia.com/phones/en_int/nokia-8110-4g?sku=16ARGYW1A09 - at least it can be used as a mobile phone!

You need a Matrix Homeserver

The Matrix Homeserver is the foundation of the whole system.
There are many servers available, Synapse seemed a good and decent choice.
The Docker version is well maintained and the compose stackfile is simple enough:

version: '3'

services:

  synapse:
    image: "matrixdotorg/synapse:latest"
    restart: unless-stopped
    networks:
      - default
      - synapse
    depends_on:
      - postgres
    volumes:
      - ./data/synapse:/data
    environment:
      VIRTUAL_HOST: "matrix.codref.org"
      VIRTUAL_PORT: 8008
      SYNAPSE_SERVER_NAME: "matrix.codref.org"
      SYNAPSE_REPORT_STATS: "yes"
    ports:
      - "8008:8008/tcp"

  postgres:
    image: docker.io/postgres:14-alpine
    restart: unless-stopped
    networks:
      - default
    environment:
      POSTGRES_USER: synapse
      POSTGRES_PASSWORD: XXX
      POSTGRES_DB: synapse
      POSTGRES_INITDB_ARGS: --encoding=UTF-8 --lc-collate=C --lc-ctype=C
    volumes:
      - ./data/postgres:/var/lib/postgresql/data


networks:
  default:
  synapse:
    external: true

The execution of Synapse with the above stack file requires the below steps:

mkdir -p ./data/synapse
mkdir -p ./data/postgres
docker network create synapse

(this creates an external network, used by all the bridges)

Synapse and all the matrix bridges has a quite peculiar way to be configured.
Instead of providing a configuration file, the file is created at first run.
The two variables VIRTUAL_HOST and SYNAPSE_SERVER_NAME are used to initialize the config file.

To generate the synapse config file:

docker compose run --rm synapse generate

The server name is important, check the official documentation https://matrix.org/docs/guides/understanding-synapse-hosting
In my case, I wanted specifically to use (for both the server and the hosting) the domain name matrix.codref.org.
The matrix federation allows multiple ways to shorten the domain name and use just the secondary - although this is not your business.

The configuration file looks like the below:

# Configuration file for Synapse.
#
# This is a YAML file: see [1] for a quick introduction. Note in particular
# that *indentation is important*: all the elements of a list or dictionary
# should have the same indentation.
#
# [1] https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html
#
# For more information on how to configure Synapse, including a complete accounting of
# each option, go to docs/usage/configuration/config_documentation.md or
# https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html
server_name: "matrix.codref.org"
pid_file: /data/homeserver.pid
listeners:
  - port: 8008
    tls: false
    type: http
    x_forwarded: true
    resources:
      - names: [client, federation]
        compress: false

#database:
#  name: sqlite3
#  args:
#    database: /data/homeserver.db

database:
    name: psycopg2
    args:
        user: synapse
        password: XXX
        host: postgres
        database: synapse
        cp_min: 5
        cp_max: 10

log_config: "/data/matrix.codref.org.log.config"
media_store_path: /data/media_store
registration_shared_secret: "XXX"
report_stats: true
macaroon_secret_key: "XXX"
form_secret: "XXX"
signing_key_path: "/data/matrix.codref.org.signing.key"
trusted_key_servers:
  - server_name: "matrix.org"

You are expected to adjust the database section, eventually using postgres, like in the above example.
SQLite works well, although managing postgres in Docker is not so complex, therefore worth a try.

The users creation is disabled by default. Users can be registered enabling the self registration:

enable_registration: true
enable_registration_without_verification: true

although this should be disabled right after (to avoid being target of spammers).

You can also register users manually using the console (with the stack up):

docker exec -it synapse-synapse-1 bash

register_new_matrix_user -c homeserver.yaml http://localhost:8008

Run Synapse Homeserver

To run the server, just turn on the stack:

docker compose up # use -d to put the stack in background

The Matrix server will be up and running, listening on port 8008 of your system.

This is quite good, but to publish it over the public internet, a reverse proxy is better. For example you’ll want to deploy an nginx server and use certbot to obtain an SSL certificate.

This can be achieved quite easily, I suggest to read https://matrix-org.github.io/synapse/latest/reverse_proxy.html and https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-20-04

This two how-to are quite enough detailed to achieve your goal.

To check that Synapse is running, open a webpage to http://localhost:8008 or to the domain/url you used.
The Synapse welcome page should appear.

If you configured the reverse proxy according to the official documentation, only the _matrix paths gets forwarded.

Nginx on system or on container?

I prefer for such services to deploy Nginx to the operative system, and use Docker for the running services.
Eventually a new stack for Nginx can be configured.
Having Nginx deployed at OS level allows easier configuration only.

The Mautrix Whatsapp bridge

With a running Synapse server, on his dedicated Docker compose stack file, running and working (check it out on Element or any other Matrix client!) - it’s time to deploy the Whatsapp bridge.

This is a local server which simulates a browser and consumes the Web API for Whatsapp.
It requires a running Whatsapp client on a phone - but not 24/7 running (just every 2 weeks).

The stack file is similar to the Synapse one:

version: "3.7"

services:
  mautrix-whatsapp:
    container_name: mautrix-whatsapp
    image: dock.mau.dev/mautrix/whatsapp:latest
    restart: unless-stopped
    volumes:
    - ./data/whatsapp:/data

    # If you put the service above in the same docker-compose as the homeserver,
    # ignore the parts below. Otherwise, see below for configuring networking.

    # If synapse is running outside of docker, you'll need to expose the port.
    # Note that in most cases you should either run everything inside docker
    # or everything outside docker, rather than mixing docker things with
    # non-docker things.
    #ports:
    #- "29318:29318"
    # You'll also probably want this so the bridge can reach Synapse directly
    # using something like `http://host.docker.internal:8008` as the address:
    #extra_hosts:
    #- "host.docker.internal:host-gateway"

    # If synapse is in a different network, then add this container to that network.
    networks:
      - synapse
      - default

  postgres:
    image: docker.io/postgres:14-alpine
    restart: unless-stopped
    networks:
      - default
    environment:
      POSTGRES_USER: whatsapp
      POSTGRES_PASSWORD: XXX
      POSTGRES_DB: whatsapp
      POSTGRES_INITDB_ARGS: --encoding=UTF-8 --lc-collate=C --lc-ctype=C
    volumes:
      - ./data/postgres:/var/lib/postgresql/data


networks:
  default:
  synapse:
    external: true

The generation of the configuration file is similar to the Synapse one:

mkdir -p ./data/whatsapp
mkdir -p ./data/postgres
docker compose run --rm mautrix-whatsapp

This will generate a file in ./data/whatsapp named config.yaml.
The file is too big to show here, although there are just few lines to change:

homeserver:
    # The address that this appservice can use to connect to the homeserver.
    address: http://synapse:8008
    # The domain of the homeserver (also known as server_name, used for MXIDs, etc).
    domain: matrix.codref.org

appservice:
    # The address that the homeserver can use to connect to this appservice.
    address: http://mautrix-whatsapp:29318

    # Database config.
    database:
        # The database type. "sqlite3-fk-wal" and "postgres" are supported.
        type: postgres
        # The database URI.
        #   SQLite: A raw file path is supported, but `file:<path>?_txlock=immediate` is recommended.
        #           https://github.com/mattn/go-sqlite3#connection-string
        #   Postgres: Connection string. For example, postgres://user:password@host/database?sslmode=disable
        #             To connect via Unix socket, use something like postgres:///dbname?host=/var/run/postgresql
        uri: postgres://whatsapp:XXX@postgres/whatsapp?sslmode=disable
        # Maximum number of connections. Mostly relevant for Postgres.
        max_open_conns: 20
        max_idle_conns: 2
        # Maximum connection idle time and lifetime before they're closed. Disabled if null.
        # Parsed with https://pkg.go.dev/time#ParseDuration
        max_conn_idle_time: null
        max_conn_lifetime: null

    permissions:
        "*": relay
        "matrix.codref.org": user
        "@aaa:matrix.codref.org": admin

The configuration is done, but the bridge has still to generate a registration.yaml file, which is used by Synapse to communicate to the bridge.
This is obtained running the service once, again:

docker compose run --rm mautrix-whatsapp

Check the file in ./data/whatsapp named registration.yaml. This file must be copied to the data folder of Synapse!

cp ./data/whatsapp/registration.yaml [synapse data folder]/registration-whatsapp.yaml

Configure Synapse for Whatsapp bridge

Change the Synapse homserver confguration and add at the end of the data/synapse/homeserver.yaml file:

app_service_config_files:
  - /data/registration-whatsapp.yaml

Restart the Synapse server:

docker compose down
docker compose up # -d

Start Whatsapp bridge

With all the configurations done, the bridge can now be started:

cd mautrix-whatsapp
docker compose up

The authentication requires some steps done in a Matrix client. The official documentation https://docs.mau.fi/bridges/go/whatsapp/authentication.html covers all the steps very well.

Filesystem configuration

Internet has some (not many) examples about how to run all these tools together.
I struggled a bit to understand the relation between all the components, although I ended up doing:

  • deploy a vanilla Debian server (or any other distro)
    • better to encrypt the disk
  • install Docker CE
  • create the stack files in /opt
    • synapse will have his own stack file /opt/synapse
    • whatsapp will have his own as well in /opt/mautrix-whatsapp
    • eventually telegram and Sygnal bridges can be deployed too (always on /opt/xxx)

I opted to replicate the deployment of Postgres on all the servers.

Final remarks

The system works very well and with servers federation you can disable users registration and stay with your own private server (but chat with others).
I did not cover any VoIP - could be interesting for the future.

See Also

Assign a reserved IP to a running Azure virtual machine

Assign a reserved IP to a running Azure virtual machine Below instructions apply to classic (old) Azure portal only! It’s tricky, and if you do not have a lot of patience, put on the stove a good cup of tea/coffee (maybe not too strong) and take a good breath.