How to Configure Declarative VXLAN Overlays on Ubuntu Server Using systemd-networkd

Modern cloud infrastructure, Kubernetes clusters, and multi-node virtualization environments require isolated Layer 2 Ethernet connectivity across physically dispersed Layer 3 underlay networks. Virtual eXtensible Local Area Network (VXLAN) addresses the scalability limits of traditional 802.1Q VLANs (expanding the segment ceiling from 4,096 to over 16 million logical networks) by encapsulating Layer 2 Ethernet frames inside standard UDP datagrams transmitted on UDP port 4789. While administrators frequently configure ephemeral VXLAN tunnels using manual ip link commands, production server infrastructure demands declarative, reboot-resilient network provisioning.

On Ubuntu Server, systemd-networkd provides native declarative support for VXLAN tunnels. By structuring interface definitions into virtual network device (.netdev) and link binding (.network) files, system administrators can establish persistent, high-throughput overlay networks between bare-metal hypervisors, container hosts, and staging nodes with zero external software dependencies.

Understanding VXLAN Architecture and Underlay Encapsulation

VXLAN bridges isolated overlay networks across standard IP routing infrastructure:

• VXLAN Network Identifier (VNI): A 24-bit integer (identifying up to 16,777,216 distinct overlay segments) embedded into the VXLAN header. Virtual machines and containers attached to the same VNI communicate as if plugged into the same physical Layer 2 Ethernet switch.
• VXLAN Tunnel Endpoint (VTEP): The network interface on each Ubuntu host responsible for encapsulating outbound frames into UDP packets and decapsulating inbound UDP packets back into raw Layer 2 Ethernet frames.
• Transport Modes: Unicast point-to-point tunnels specify explicit remote VTEP IP addresses directly, while multicast-based discovery uses IGMP groups to flood broadcast, unknown-unicast, and multicast (BUM) traffic across participating nodes.

Prerequisites and Kernel Module Verification

Configuring declarative VXLAN on Ubuntu Server requires administrative root privileges, active systemd-networkd, and the vxlan Linux kernel module.

Load the kernel module and verify support:

# Load vxlan module into running kernel
sudo modprobe vxlan

# Verify module presence
lsmod | grep vxlan

# Ensure module loads automatically across reboots
echo "vxlan" | sudo tee /etc/modules-load.d/vxlan.conf

Confirm that systemd-networkd is active and inspect the underlying physical network interfaces:

# Check networkd service status
systemctl is-active systemd-networkd

# List active physical interfaces and IP configurations
networkctl list

Assume enp3s0 represents the primary physical Ethernet adapter on Node 1 (IP: 192.168.10.11) connecting across the underlay network to Node 2 (IP: 192.168.10.12).

Creating the Declarative VXLAN Virtual Device (.netdev)

In systemd-networkd, virtual network devices are defined within /etc/systemd/network/*.netdev configuration files.

Create the VXLAN virtual device definition file:

sudo nano /etc/systemd/network/20-vxlan100.netdev

Populate the file with the virtual device parameters:

[NetDev]
Name=vxlan100
Kind=vxlan

[VXLAN]
VNI=100
Remote=192.168.10.12
Local=192.168.10.11
DestinationPort=4789
MacLearning=yes

Key parameters in this definition:

• VNI=100: The 24-bit segment identifier matching the overlay network.
• Remote and Local: The underlay IP addresses of the peer host and the local interface respectively.
• DestinationPort=4789: The IANA-standard UDP port for VXLAN encapsulation.
• MacLearning=yes: Enables dynamic MAC-to-underlay IP mapping in the kernel forwarding database (FDB).

Configuring IP Addressing on the Overlay Interface (.network)

Next, define the overlay IP network profile for the newly instantiated vxlan100 device.

Create the network configuration file:

sudo nano /etc/systemd/network/25-vxlan100.network

Configure the private overlay subnet (e.g. 10.200.0.0/24):

[Match]
Name=vxlan100

[Network]
Address=10.200.0.1/24

On the peer node (Node 2), apply the reciprocal configuration: set Remote=192.168.10.11, Local=192.168.10.12 in its .netdev file, and assign Address=10.200.0.2/24 in its .network file.

Applying Configuration and Verifying the Overlay Tunnel

Reload systemd-networkd to process the new configuration files and bring up the virtual interface:

# Reload networkd configurations
sudo networkctl reload

# Inspect operational state of vxlan100
networkctl status vxlan100

Verify that the interface is operational and verify kernel link attributes:

# Inspect low-level link parameters
ip -d link show vxlan100

The output confirms the active VNI, UDP destination port, and underlay VTEP bindings.

Testing Connectivity and Inspecting Encapsulated Packets

Test bidirectional communication across the private overlay tunnel using ping:

# Ping peer overlay IP address
ping -c 4 10.200.0.2

To verify that overlay traffic is properly encapsulated within UDP datagrams on the physical wire, capture packets on the underlying physical interface (enp3s0):

# Capture UDP 4789 packets on physical underlay interface
sudo tcpdump -i enp3s0 -nn "udp port 4789"

The capture displays Outer IP headers (192.168.10.11 > 192.168.10.12) containing UDP 4789 VXLAN payloads encapsulating inner ICMP echo packets (10.200.0.1 > 10.200.0.2).

Inspecting the Forwarding Database (FDB)

Query the kernel’s Layer 2 forwarding table to verify learned MAC addresses across the overlay:

# Display forwarding database for vxlan100
bridge fdb show dev vxlan100

The table displays the remote overlay MAC address mapped directly to the destination underlay VTEP IP, ensuring optimal Layer 2 packet switching without broadcast flooding.

By declaring VXLAN tunnels with systemd-networkd, Ubuntu Server administrators achieve robust, automated multi-node network virtualization that survives server reboots and carrier state changes.