In standard KVM or VMware virtualization, when a virtual machine sends a network packet, the packet must traverse a complex, software-defined path. It travels from the VM’s virtual NIC (virtio), through the hypervisor’s virtual switch (like Open vSwitch or Linux Bridge), and finally out the physical Network Interface Card (NIC). This software processing layer (the hypervisor context switch) introduces significant CPU overhead and microsecond latency, bottlenecking high-frequency trading applications, 5G telco workloads, and massive database servers.
The hardware solution to this bottleneck is Single Root I/O Virtualization (SR-IOV).
SR-IOV is a PCIe specification that allows a single physical PCIe device (like a 100Gbps Mellanox or Intel NIC) to present itself as multiple, distinct virtual PCIe devices (Virtual Functions, or VFs) directly on the hardware bus. Using IOMMU (I/O Memory Management Unit), the Linux hypervisor passes a Virtual Function directly through to the virtual machine.
The VM’s guest operating system loads the native hardware driver and talks directly to the physical silicon on the NIC, completely bypassing the hypervisor’s software bridge. The result is bare-metal network performance inside a virtual machine.
This guide explains how to enable and configure SR-IOV on a Linux KVM hypervisor.
Step 1: Enabling IOMMU in the BIOS and Kernel
Before SR-IOV can function, the motherboard hardware and the Linux kernel must support PCIe passthrough via IOMMU (Intel VT-d or AMD-Vi).
- BIOS/UEFI: Reboot the server, enter the BIOS, and ensure that Intel VT-d or AMD-Vi (IOMMU) is explicitly enabled. Also, ensure SR-IOV Global Enable is activated if your BIOS has that option.
- Linux Kernel Parameters: You must instruct the Linux kernel to enable the IOMMU groups at boot. Edit the GRUB configuration file:
sudo nano /etc/default/grub
Append the following to the GRUB_CMDLINE_LINUX_DEFAULT line:
- For Intel:
intel_iommu=on iommu=pt - For AMD:
amd_iommu=on iommu=pt
Update GRUB and reboot the server:
sudo update-grub
sudo reboot
After the reboot, verify IOMMU is active by checking dmesg:
dmesg | grep -e DMAR -e IOMMU
Step 2: Identifying the Physical NIC (PF)
The physical NIC port is known as the Physical Function (PF). You need to identify its PCIe address and network interface name.
lshw -c network -businfo
Assume your 10Gbps Intel NIC is eth0 and its PCIe address is 0000:03:00.0.
Step 3: Creating the Virtual Functions (VFs)
You instantiate the Virtual Functions by echoing a number into the sriov_numvfs sysfs file for the specific PCIe device.
To create 4 Virtual Functions on eth0:
echo 4 | sudo tee /sys/class/net/eth0/device/sriov_numvfs
Verify that the VFs were successfully created at the hardware level:
lspci | grep Ethernet
You will now see the primary Physical Function (PF) alongside four new Virtual Function (VF) PCIe devices.
Step 4: Making VFs Persistent (The Right Way)
Echoing into sysfs is temporary; the VFs will vanish on reboot. To make them persistent, you should configure a udev rule or use systemd-networkd.
Create a udev rule to automatically instantiate the VFs when the NIC driver loads:
sudo nano /etc/udev/rules.d/99-sriov.rules
Add the following line, replacing the PCI address with your PF’s address:
ACTION=="add", SUBSYSTEM=="net", KERNELS=="0000:03:00.0", ATTR{device/sriov_numvfs}="4"
Step 5: Attaching the VF to a KVM Virtual Machine
Now that the VFs exist on the host, you must pass one through to the VM using libvirt.
First, find the exact PCIe address of one of the Virtual Functions (e.g., 0000:03:10.0 using lspci).
Edit the XML configuration of your virtual machine:
virsh edit my-database-vm
Inside the <devices> block, add a <hostdev> definition to pass through the VF. (You must map the PCI address correctly into the domain, bus, slot, and function XML format).
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x03' slot='0x10' function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
</hostdev>
Save and exit. When you boot my-database-vm, the hypervisor’s VFIO (Virtual Function I/O) driver will detach the VF from the host and map it exclusively to the VM’s memory space.
Conclusion
SR-IOV completely bypasses the hypervisor software stack, offering bare-metal line-rate throughput and single-digit microsecond latency inside a virtual machine. By leveraging IOMMU and PCIe hardware virtualization, systems engineers can virtualize the most intensive telco and database workloads without sacrificing network performance.