nsg's blog

LXD and VXLAN

2022-02-12

Let's start with a little background

I have a lot of servers all over the place, and it's not uncommon that I need to connect them together in some way. For simple setups WireGuard is working fantastically well but I usually need layer 2 networks. For that I still use tinc, another software I can recommend. Before WireGuard was a thing, tinc was easily my default choice.

The new overlay

I use LXD to run both VM:s and system containers on my servers and it supports VXLAN overlay networks. VXLAN has no encryption so I'm not going to send it over the internet. But we also have WireGuard, so let's combine them!

Setup WireGuard

Just a quite vanilla WireGuard setup, no routing, NAT for forward needed. Just a point-to-point encrypted link. This is the content from the server that receives the connection.

[Interface]
Address = 10.0.1.1/24
SaveConfig = true
ListenPort = 1234
PrivateKey = secret-private-key1

[Peer]
PublicKey = peers-public-key2
AllowedIPs = 10.0.1.2/32

... and on the other server I have this:

[Interface]
Address = 10.0.1.2/24
SaveConfig = true
PrivateKey = secret-private-key2

[Peer]
PublicKey = peers-public-key1
AllowedIPs = 10.0.1.1/32
Endpoint = my-remote-server.example.com:1234

The content is placed in /etc/wireguard/wg0.conf on both servers. After than you need enable and start the configuration on both server and you should have a tunnel open.

systemctl enable wg-quick@wg0
systemctl start wg-quick@wg0

Configure LXD

server1

Configure a new network bridge called br-vxlan, I disabled IPv6 because I do not need it. A ipv4 subnet will be allocated and shared via DHCP on this network.

lxc network create br-vxlan \
    tunnel.server2.protocol=vxlan \
    tunnel.server2.id=10 \
    tunnel.server2.local=10.0.1.1 \
    tunnel.server2.remote=10.0.1.2 \
    ipv6.address=none

Attach our new network to the default profile, name it vxlan0.

lxc network attach-profile br-vxlan default vxlan0

server2

This configuration is similar to the above. I disabled IPv4 because addresses will be provided via DHCP from server1. On this server I created a separate profile instead of placing the configuration inside the default profile.

lxc network create br-vxlan \
    tunnel.server1.protocol=vxlan \
    tunnel.server1.id=10 \
    tunnel.server1.local=10.0.1.2 \
    tunnel.server1.remote=10.0.1.1 \
    ipv4.address=none \
    ipv6.address=none

lxc profile create tunnel
lxc network attach-profile br-vxlan tunnel vxlan0

Final notes

I will probably run my tinc tunnels for a long time. The new WG+VXLAN tunnels will be used to bridge a few new servers together, if it works well and stays stable I may start to replace tinc later on.

All these 437 words are written by Stefan Berggren, feel free and contact me if you like.