systemd-nspawn vpn container

Disclaimer

This is not written by a network security specialist. Please use the following at your own risk.

Intro

Recently i had some problems happening with lxd - while shutting down / rebooting host, lxd wouldnt stop and machine had to wait for timeout on some mount. Also, i moved to fedora to have automatic updates installed via cockpit.

Looking at alternatives only a few things pop up like docker, but i want to use system containers as replacements for lxd.

Enter systemd-nspawn.

These notes come from following this guide.

Whole guide is about making a container called vpn-br. You can use any name you want.

Assumption: btrfs is used as filesystem where the containers will be stored.

Setup host part

Host part contains several steps:

  • setup subvolume per container (btrfs)
  • setup network bridge
  • setup repo for cents

Anyway, all steps are mentioned below.

Prereqs

Necessery package

dnf -y install systemcd-container

Make a bridge for routed network from the container

cat > /etc/systemd/network/10-bridge.netdev << 'EOF'
[NetDev]
Name=br0
Kind=bridge
EOF

cat > /etc/systemd/network/20-bridge.network << 'EOF'
[Match]
Name=br0

[Network]
Address=192.168.49.1/24
DNS=192.168.40.1
ConfigureWithoutCarrier=yes
RequiredForOnline=no
EOF

systemctl enable --now systemd-networkd

Making a bridge is not enough, we also need to tell the container to connect to it. This is done in /etc/systemd/nspawn/.

Make a file with nspawn extension called exactly like the container you are trying to make.

cat > /etc/systemd/nspawn/vpn-br.nspawn << 'EOF'
[Network]
Bridge=br0

Prepare machine/container directory

You need to have a subvolume for each container in /var/lib/machines. Machines directory should exist and should already be a subvolume.

cd /var/lib/machines

btrfs subvolume create vpn-br

Container packages installation

Centos

Prepare a cenots repo file to be used for installation

cat > /root/centos9-stream.repo << 'EOF'
[centos9stream]
name=CentOS-9-Stream-Base
baseurl=https://centos.anexia.at/centos-stream/9-stream/BaseOS/x86_64/os/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial

[centos9appstream]
name=CentOS-9-Stream-AppStream
baseurl=https://centos.anexia.at/centos-stream/9-stream/AppStream/x86_64/os/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
EOF

Run installation

dnf -c /root/centos9-stream.repo \
--releasever=9-stream \
--best \
--disablerepo=* \
--setopt=install_weak_deps=False \
--enablerepo=centos9stream \
--enablerepo=centos9appstream \
--installroot=/var/lib/machines/vpn-br \
install \
centos-release \
dhcp-client \
dnf \
glibc-langpack-en \
glibc-langpack-de \
iproute \
iputils \
less \
passwd \
systemd \
NetworkManager \
cockpit \
cockpit-packagekit \
wireguard-tools \
openssh-server \
vim

Set password for root user within container

setenforce 0
systemd-nspawn -D /var/lib/machines/vpn-br passwd
setenforce 1

AlmaLinux

Prepare a repo file

cat > /root/almalinux9.repo << 'EOF'
[baseos]
name=AlmaLinux $releasever - BaseOS
mirrorlist=https://mirrors.almalinux.org/mirrorlist/$releasever/baseos
# baseurl=https://repo.almalinux.org/almalinux/$releasever/BaseOS/$basearch/os/
enabled=1
gpgcheck=1
countme=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-AlmaLinux-9
metadata_expire=86400
enabled_metadata=1

[appstream]
name=AlmaLinux $releasever - AppStream
mirrorlist=https://mirrors.almalinux.org/mirrorlist/$releasever/appstream
# baseurl=https://repo.almalinux.org/almalinux/$releasever/AppStream/$basearch/os/
enabled=1
gpgcheck=1
countme=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-AlmaLinux-9
metadata_expire=86400
enabled_metadata=1
EOF

Run installation

dnf -c /root/almalinux9.repo \
--releasever=9 \
--best \
--disablerepo=* \
--setopt=install_weak_deps=False \
--enablerepo=baseos \
--enablerepo=appstream \
--enablerepo=extras \
--installroot=/var/lib/machines/vpn-br \
install \
almalinux-release \
dhcp-client \
dnf \
glibc-langpack-en \
iproute \
iputils \
less \
passwd \
systemd \
wireguard-tools \
cockpit \
cockpit-packagekit \
openssh-server \
vim

Set password for root user within container

setenforce 0
systemd-nspawn -D /var/lib/machines/vpn-br passwd
setenforce 1

Fedora

Run installation

dnf \
--releasever=39 \
--best \
--setopt=install_weak_deps=False \
--installroot=/var/lib/machines/vpn-br \
install \
fedora-release \
dhcp-client \
dnf \
glibc-langpack-en \
iproute \
iputils \
less \
passwd \
systemd \
systemd-networkd \
systemd-resolved \
util-linux \
vim-default-editor \
wireguard-tools \
cockpit \
cockpit-packagekit \
sudo \
openssh-server \
vim

Set password for root user within container

setenforce 0
systemd-nspawn -D /var/lib/machines/vpn-br passwd
setenforce 1

Start the container via systemd

systemctl start systemd-nspawn@vpn-br

In container

Depending on the distribution, either NetworkManager or systemd-networkd must be used to set up networking.

  • Centos, RHEL, Alma - NetworkManager
  • Fedora - systemd-networkd

Static IP addresses might be easier, so this is what is used below.

Note: Whoever named NetworkManager with capital letters is an idiot.

Networking with NetworkManager

nmcli con mod 5418ec27-b0b4-3814-bfd7-a660c7cca02e ipv4.dns "192.168.40.1"

nmcli con mod 5418ec27-b0b4-3814-bfd7-a660c7cca02e ipv4.addresses 192.168.49.2/24 gw4 192.168.49.1

nmcli con mod 5418ec27-b0b4-3814-bfd7-a660c7cca02e ipv4.dns "1.1.1.1"

Networking with systemd-networkd

Route section in systemd networking is needed to enable lan traffic while being connected to VPN. All other traffic will go through the tunnel.

cat > /var/lib/machines/vpn-br/etc/systemd/network/10-host0.network << 'EOF'
[Match]
Name=host0

[Network]
DHCP=no
Address=192.168.49.2/24
Gateway=192.168.49.1
DNS=192.168.40.1

LinkLocalAddressing=no
IPv6AcceptRA=no

[Route]
Destination=192.168.40.0/24
Gateway=192.168.49.1
EOF

Enable services

systemctl enable --now cockpit.socket
systemctl enable --now sshd

Post Installation

Automatic startup from host

Automatically starting the machine can be done with systemd automatically.

systemctl enable --now systemd-nspawn@vpn-br

Firewall

New policy is needed to allow traffic from the containers outside, plus also, from the home network into the containers.

firewall-cmd --new-zone=containers --permanent
firewall-cmd --zone=containers --change-interface=br0 --permanent
firewall-cmd --reload

firewall-cmd --new-policy=containers_out --permanent
firewall-cmd --policy=containers_out --add-ingress-zone=containers --permanent
firewall-cmd --policy=containers_out --add-egress-zone=external --permanent
firewall-cmd --policy=containers_out --set-target=ACCEPT --permanent
firewall-cmd --reload

firewall-cmd --new-policy=containers_in --permanent
firewall-cmd --policy=containers_in --add-ingress-zone=home --permanent
firewall-cmd --policy=containers_in --add-egress-zone=containers --permanent
firewall-cmd --policy=containers_in --set-target=ACCEPT --permanent
firewall-cmd --reload


firewall-cmd --zone=containers --add-service=cockpit --permanent
firewall-cmd --zone=containers --add-service=ssh --permanent
firewall-cmd --reload

Notes

Boot the container

systemd-nspawn -D /var/lib/machines/vpn-br -b

Login into the container

machinectl login vpn-br