What is traefik

As per the official website:

Traefik is a leading modern reverse proxy and load balancer that makes deploying microservices easy. Traefik integrates with your existing infrastructure components and configures itself automatically and dynamically.

What does this actually mean? It’s a traffic router where accessing a certain port on the public ip gets the traffic from a specific virtual machine or container behind that public ip.

What is a reverse proxy?

If a server has one IP address assigned, but you want to serve several domains, e.g. vaultwarden.flippityflop.com and forgejo.flippityflop.com on that server, there is supposed to be a way how to route data from your computer to the desired service. This is what reverse proxy does.

Why traefik and not something else

Usually people would go to nginx when reverse proxy is needed, but service discovery is (as far as i remember) an enterprise feature. Service discovery means that you get a new “traffic route” automatically generated as the services become available. Or in simple words - when you have a new container, traefik will update itself to serve whatever is in the new container under a new subdomain, subpath or whatever you want.

Beside this, traefik benefits are:

written in golang, which means low hw requirements, multithreading, simple deployment due to it being a single binary, etc
supports automatic let's encrypt certificate generation (for supported registrars)
multiple options for service discovery, like docker, file (static) etc
…

Configuration

There are multiple files necessery for it to be configured properly:

static configuration defines the “traefik” stuff
- certificate resolver(s) which can be reused by different services
- entry ports which will be routed to target services
- and where to get the services
dynamic configuration
- this is the service definition - where to get them (which ip, port, etc…)

The idea behind can be easily grasped from the configuration files below.

Here is an example that i’m currently trying to set up on one of the hetzner cloud instances for hosting dendrite and mastodon:

example domain flippityflop.com
this is a manual deployment, not via podman/docker, but it does have podman/docker on the same machine.
- besides podman containers, i also run lxd and have separate containers that behave like virtual machines. this is why file provider is included

Static configuration

`/etc/traefik/traefik-static.yaml`

file provider is basically a manually configured pointer to a service and configuration is read from a file
traefik has a gui itself, and this is why api section is for. insecure set to false means that it will be used with let's encrypt certificates.
caServer property in the configuratio should be used for testing. if enabled, you get a certificate from staging servers of let's encrpyt. Warning: browsers will say it is insecure.

## STATIC CONFIGURATION
log:
  level: DEBUG

api:
  insecure: false
  dashboard: true

entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
  websecure:
    address: ":443"
  matrixsecure:
    address: ":8448"

providers:
  file:
    filename: /etc/traefik/traefik-dynamic.yaml
    watch: true
  docker:
    endpoint: "unix:///run/podman/podman.sock"
    exposedByDefault: false
    watch: true

certificatesResolvers:
  lets-encr-porkbun:
    acme:
      email: your-email-here@flippityflop.com
      storage: /etc/traefik/acme.json
      #caServer: https://acme-staging-v02.api.letsencrypt.org/directory
      dnsChallenge:
        provider: porkbun
        delayBeforeCheck: 0
        resolvers:
          - "curitiba.ns.porkbun.com"
          - "1.1.1.1:53"

Dynamic configuration

`/etc/traefik/traefik-dynamic.yaml`

dendrite router and dendrite service have the same name, but this can be different.
since dendrite is defined in the dynamic configuration, it is used through the static configuration file provider
traefik has a gui itself, and this is why api router is for

## DYNAMIC CONFIGURATION
http:
  routers:
    api:
      rule: "Host(`traefik.flippityflop.com`)"
      service: "api@internal"
      tls:
        certResolver: "lets-encr-porkbun"
        domains:
          - main: "traefik.flippityflop.com"
    dendrite:
      entryPoints: [ matrixsecure ]
      service: "dendrite"
      rule: "Host(`matrix.flippityflop.com`)"
      tls:
        certResolver: "lets-encr-porkbun"
        domains:
          - main: "matrix.flippityflop.com"
  services:
    dendrite:
      loadBalancer:
        servers:
          - url: "http://10.7.138.151:8008"

Environment variables

`/etc/traefik/porkbun.env`

depending on your dns provider, you would use different values here. specification is available here

PORKBUN_API_KEY=pk1_yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
PORKBUN_SECRET_API_KEY=sk1_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

systemd unit

`/etc/systemd/system/traefik.service`

[Unit]
Description=Traefik
Documentation=https://doc.traefik.io/traefik/
After=network-online.target
AssertFileIsExecutable=/usr/bin/traefik
AssertPathExists=/etc/traefik/traefik-static.yaml
AssertPathExists=/etc/traefik/traefik-dynamic.yaml
#AssertPathExists=/etc/traefik/acme.json

[Service]
# Run traefik as its own user (create new user with: useradd -r -s /bin/false -U -M traefik)
User=traefik
AmbientCapabilities=CAP_NET_BIND_SERVICE

# configure service behavior
Type=notify
ExecStart=/usr/bin/traefik --configFile=/etc/traefik/traefik-static.yaml
Restart=always
WatchdogSec=1s
EnvironmentFile=/etc/traefik/porkbun.env

ProtectSystem=strict
PrivateTmp=true
ProtectHome=true
PrivateDevices=true
ProtectKernelTunables=true
ProtectControlGroups=true

# allow writing of acme.json
ReadWritePaths=/etc/traefik/acme.json
ReadOnlyPaths=/etc/traefik/traefik-static.json
ReadOnlyPaths=/etc/traefik/traefik-dynamic.json
#ReadWritePaths=/run/podman/podman.sock
#LimitNPROC=1

[Install]
WantedBy=multi-user.target

Podman

An example podman-compose script to run portainer is below. Important thing to note is that traefik will recognize the service based on the labels. All lables below have to be set.

version: "3.7"

services:
  whoami:
    image: "portainer/portainer-ce:latest"
    container_name: "portainer"
    hostname: "portainer"
    restart: always
    volumes:
      - "/etc/localtime:/etc/localtime:ro"
      - "/run/podman/podman.sock:/var/run/docker.sock:ro"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.portainer.entrypoints=websecure"
      - "traefik.http.routers.portainer.rule=Host(`portainer.flippityflop.com`)"
      - "traefik.http.routers.portainer-secure.service=portainer"
      - "traefik.http.routers.portainer.tls.certresolver=lets-encr-porkbun"
      - "traefik.http.services.portainer.loadbalancer.server.port=9000"

Qublet

Since there are some problems in managing the containers via this way, people made a way to use systemd to run containers. Example below is running on fedora server.

For each service, there are at least two files needed:

.kube - a file which defines a way to generate systemd services
.yaml - a file which is a kubernetes specification for a pod (aka group of containers)

Since i am using an IPv6 in my lan, i had to enable IPv6 in the containers as well, so for this there is an extra .network file. This file is different to standard .network from systemd as it is used to describe a podman network.

After the files have been put into place, following steps are needed:

systemctl daemon-reload
systemctl restart podman
systemctl enable podman

Additionally do not forget to enable 443 service via firewalld:

firewall-cmd --zone x --add-service https --permanent
firewall-cmd --reload

The setup below will start two pods, each having multiple containers:

first pod is traefik, which will expose 80 and 443 ports to outside and this will be the entry point for all other pods/containers/services. it can be accessed via traefik.flippityflow.com
second pod is forgejo that is served under git.flippityflow.com. this service does not need to expose any ports for outside since the traffic is routed through traefik. therefore, traefik will encrypt network traffic between your computer and the server, but internally, within the server/podman, network traffic will be unencrypted.

One note: my setup is running with privileged / root containers. There are two reasons:

i dont know exactly how to properly set up separate user and autostarting services if the user is not logged in
in rootless containers you are not allowed to open ports below 1000, so firewall would need to redirect ports

In my current setup, nothing of this is exposed to internet, so it doesnt really matter.

`/etc/containers/systemd/traefik.network`

[Network]
IPv6=true

`/etc/containers/systemd/traefik.kube`

[Unit]
Description=Traefik container
After=network.target

[Kube]
Yaml=traefik.yaml
Network=systemd-traefik

[Install]
WantedBy=multi-user.target default.target

`/etc/containers/systemd/traefik.yaml`

apiVersion: v1
kind: Pod
metadata:
  annotations:
    bind-mount-options: /var/run/podman/podman.sock:z
  creationTimestamp: "2024-02-18T16:02:00Z"
  labels:
    app: traefik
    traefik.enable: true
    traefik.http.routers.traefik.entrypoints: websecure
    traefik.http.routers.traefik.rule: "Host(`traefik.flippityflop.com`)"
    traefik.http.routers.traefik.service: api@internal
    traefik.http.routers.traefik.tls: true
    traefik.http.routers.traefik.tls.certresolver: lets-encr-porkbun
  name: traefik-pod
spec:
  securityContext:
    seLinuxOptions:
      type: spc_t
  containers:
  - image: docker.io/traefik:latest
    name: traefik
    args:
    ports:
    - containerPort: 443
      hostPort: 443
      protocol: TCP
    - containerPort: 80
      hostPort: 80
      protocol: TCP
    env:
    - name: PORKBUN_API_KEY
      value: pk1_xxxxxxxxxxxxxxxxxxxxxxx
    - name: PORKBUN_SECRET_API_KEY
      value: sk1_yyyyyyyyyyyyyyyyyyyyyyy
    volumeMounts:
    - mountPath: /etc/traefik/traefik.yaml
      name: traefik-static
      readOnly: true
    - mountPath: /etc/traefik/acme.json:z
      name: acme
      readOnly: false
    - mountPath: /etc/traefik/porkbun.env
      name: porkbun-env
      readOnly: true
    - mountPath: /var/run/docker.sock:z
      name: podman-socket
      readOnly: false
  restartPolicy: Always
  volumes:
  - hostPath:
      path: /data/traefik/config/traefik-static.yaml
      type: File
    name: traefik-static
  - hostPath:
      path: /data/traefik/config/traefik-dynamic.yaml
      type: File
    name: traefik-dynamic
  - hostPath:
      path: /data/traefik/config/acme.json
      type: File
    name: acme
  - hostPath:
      path: /data/traefik/config/porkbun.env
      type: File
    name: porkbun-env
  - hostPath:
      path: /var/run/podman/podman.sock
      type: File
    name: podman-socket

`/etc/containers/systemd/forgejo.kube`

[Unit]
Description=Forgejo container
After=network.target

[Kube]
Yaml=forgejo.yaml
Network=systemd-traefik

[Install]
WantedBy=multi-user.target default.target

`/etc/containers/systemd/forgejo.yaml`

apiVersion: v1
kind: Pod
metadata:
  annotations:
    bind-mount-options: /var/run/podman/podman.sock:z
  creationTimestamp: "2024-02-18T16:02:00Z"
  labels:
    app: forgejo
    traefik.enable: true
    traefik.http.routers.forgejo.entrypoints: websecure
    traefik.http.routers.forgejo.rule: "Host(`git.flippityflop.com`)"
    traefik.http.routers.forgejo.service: forgejo
    traefik.http.routers.forgejo.tls: true
    traefik.http.routers.forgejo.tls.certresolver: lets-encr-porkbun
    traefik.http.services.forgejo.loadbalancer.server.port: 3000
  name: forgejo-pod
spec:
  containers:
  - image: codeberg.org/forgejo/forgejo:1.21
    name: forgejo
    env:
    - name: FORGEJO__database__DB_TYPE
      value: postgres
    - name: FORGEJO__database__HOST
      value: localhost:5432
    - name: FORGEJO__database__NAME
      value: forgejo
    - name: FORGEJO__database__USER
      value: forgejo
    - name: FORGEJO__database__PASSWD
      value: my-super-cool-password
    - name: MIN_PASSWORD_LENGTH
      value: 4
    volumeMounts:
    - mountPath: /data:z
      name: app-data-0
      readOnly: false
    - mountPath: /etc/localtime
      name: etc-localtime-2
      readOnly: true
  - image: postgres:latest
    name: forgejo-db
    env:
    - name: POSTGRES_USER
      value: forgejo
    - name: POSTGRES_PASSWORD
      value: my-super-cool-password
    - name: POSTGRES_DB
      value: forgejo
    volumeMounts:
    - mountPath: /var/lib/postgresql/data:z
      name: db-data-1
      readOnly: false
    - mountPath: /etc/localtime
      name: etc-localtime-2
      readOnly: true
  restartPolicy: Always
  volumes:
  - hostPath:
      path: /data/forgejo/app
      type: Directory
    name: app-data-0
  - hostPath:
      path: /data/forgejo/db
      type: Directory
    name: db-data-1
  - hostPath:
      path: /etc/localtime
      type: File
    name: etc-localtime-2