How to Configure Linux SR-IOV (Single Root I/O Virtualization) for Hardware-Accelerated VM Networking

In a standard Linux hypervisor environment (like KVM/QEMU), network traffic for a Virtual Machine (VM) is handled by software. The VM sends a packet to a virtual network adapter (like virtio-net). The hypervisor intercepts the packet in software, copies it into a virtual bridge (like Linux Bridge or Open vSwitch), and finally forwards it to the physical Network Interface Card (NIC). This software processing (the vSwitch) introduces microseconds of latency, consumes significant host CPU cycles, and severely limits maximum throughput.

For high-performance workloads—such as NFV (Network Function Virtualization), 5G telecom deployments, or extreme high-frequency trading—software switching is a fatal bottleneck.

The solution is SR-IOV (Single Root I/O Virtualization), an extension to the PCI Express (PCIe) specification. SR-IOV allows a single physical PCIe NIC to present itself to the hypervisor as multiple distinct, independent “Virtual Functions” (VFs). You can then assign a VF directly to a VM using PCI Passthrough. The VM’s network driver talks directly to the physical silicon on the NIC, completely bypassing the hypervisor’s software bridge and achieving bare-metal network performance.

This guide explains how to enable and configure SR-IOV on a Linux host.

Understanding SR-IOV Architecture

SR-IOV divides a PCIe device into two components:

1. Physical Function (PF): This is the primary PCIe device (e.g., eth0). It is controlled by the Linux hypervisor host. The host uses the PF to configure the global settings of the NIC (like creating the VFs or setting MAC addresses).
2. Virtual Functions (VFs): These are lightweight PCIe functions carved out of the PF. They share the physical Ethernet port but possess their own PCIe configuration space. When a VF is assigned to a VM, the VM accesses it exactly as if a physical NIC were plugged directly into its virtual motherboard.

Step 1: Enabling IOMMU in the Host BIOS and Kernel

Before SR-IOV can function, you must enable IOMMU (Input-Output Memory Management Unit). IOMMU provides the memory isolation required to safely pass a physical PCIe device through to a virtual machine.

1. Reboot the host server and enter the BIOS/UEFI settings. Ensure Intel VT-d (for Intel) or AMD-Vi (for AMD) is explicitly enabled.
2. Boot into Linux and edit the GRUB bootloader configuration to instruct the kernel to use IOMMU.

Open /etc/default/grub:

sudo nano /etc/default/grub

Append the IOMMU flags to GRUB_CMDLINE_LINUX. For Intel processors:

GRUB_CMDLINE_LINUX="intel_iommu=on iommu=pt"

(The iommu=pt flag enables “passthrough” mode, improving performance for devices not actively being passed through).

Update GRUB and reboot:

sudo update-grub
sudo reboot

Step 2: Activating Virtual Functions (VFs) on the NIC

Verify that your physical NIC supports SR-IOV. (Most enterprise Intel X710 or Mellanox ConnectX cards do).

Assuming your physical NIC interface is eno1, you can check its current SR-IOV capacity by reading the sysfs filesystem:

cat /sys/class/net/eno1/device/sriov_totalvfs

This will output the maximum number of VFs the silicon supports (e.g., 64).

To spawn 4 Virtual Functions, write to the sriov_numvfs file:

echo 4 | sudo tee /sys/class/net/eno1/device/sriov_numvfs

Verify that the Linux kernel has instantiated the new PCIe devices using the ip link command:

ip link show dev eno1

You will see the physical interface, followed by four new Virtual Functions (e.g., vf 0, vf 1), each with an auto-generated MAC address.

Step 3: Making the VFs Persistent

The sysfs modification above will be lost upon reboot. To make the VFs persistent, the best practice is to configure them via your network manager (like systemd-networkd or NetworkManager), or via a generic udev rule.

For example, using a udev rule:

sudo nano /etc/udev/rules.d/99-sriov.rules

Add:

ACTION=="add", SUBSYSTEM=="net", ENV{ID_NET_DRIVER}=="ixgbe", ATTR{device/sriov_numvfs}="4"

Step 4: Assigning the VF to a Virtual Machine

Now that the PCIe VFs exist, you can assign one to a VM using libvirt (the standard API for managing KVM).

First, obtain the raw PCI address of the VF you want to use:

lspci | grep -i virtual

You will see an output like 04:10.0 Ethernet controller: Intel Corporation Ethernet Virtual Function. The PCI address is 04:10.0.

You must edit the VM’s XML definition to remove the software virtual network interface (like <interface type='network'>) and replace it with a hostdev PCI passthrough block.

virsh edit my-highperf-vm

Add the hostdev block matching the PCI address:

<hostdev mode='subsystem' type='pci' managed='yes'>
  <source>
    <address domain='0x0000' bus='0x04' slot='0x10' function='0x0'/>
  </source>
</hostdev>

Save and close the XML editor. When you boot the VM, the hypervisor will detach the VF from the host kernel and map it directly into the VM’s memory space.

Conclusion

Software-defined networking bridges are excellent for flexibility, but they inevitably choke under maximum line-rate throughput. By enabling SR-IOV and passing PCIe Virtual Functions directly through to KVM virtual machines, systems engineers can achieve true zero-overhead, bare-metal networking performance, satisfying the rigorous latency requirements of modern enterprise telecommunications and database architectures.