How to Inspect and Audit IOMMU Groups and PCIe Device Isolation in Linux

In virtualized Linux environments, container hosts, and security-hardened systems, direct hardware passthrough—such as dedicating an NVMe drive, dedicated GPU, or high-speed network card to a KVM virtual machine via VFIO—relies fundamentally on the Input-Output Memory Management Unit (IOMMU). Implemented in modern hardware silicon as Intel VT-d (Virtualization Technology for Directed I/O) or AMD-Vi (I/O Virtualization Technology), the IOMMU translates device virtual memory addresses to physical system memory addresses and restricts Direct Memory Access (DMA) transactions. The smallest unit of hardware device isolation enforced by the IOMMU is an IOMMU Group. To prevent security vulnerabilities and system instability during hardware passthrough, systems engineers must audit IOMMU group boundaries using the Linux terminal and sysfs before detaching devices from host drivers.

Understanding IOMMU Groups and Direct Memory Access (DMA) Isolation

In physical hardware topology, PCI Express (PCIe) devices do not connect in a simple flat bus; they route through complex hierarchies of root ports, PCIe switches, and multi-function bridge controllers. Because PCIe devices can theoretically initiate peer-to-peer (P2P) memory transactions across shared switch fabrics without routing through the CPU, the Linux kernel groups devices based on hardware isolation boundaries:

• IOMMU Group: The smallest set of PCIe devices that can be safely isolated from other devices in the system. If two physical devices share a PCIe switch that lacks Access Control Services (ACS), they reside in the same IOMMU group.
• VFIO Passthrough Rule: To safely pass a PCIe device to a guest virtual machine using VFIO (Virtual Function I/O), every device within that specific IOMMU group must either be passed to the same virtual machine or unbound from host drivers. Passing a single device while leaving another device in the same group bound to the host creates an unsafe DMA attack vector and is rejected by the kernel.
• DMA Remapping: Prevents rogue or compromised PCIe devices from executing arbitrary memory reads or writes outside their allocated physical RAM pages.

Verifying IOMMU Hardware Activation in the Linux Kernel

Before querying IOMMU groups, verify that your motherboard firmware (UEFI/BIOS) has enabled Intel VT-d or AMD IOMMU, and that the Linux kernel has initialised the subsystem.

Query the kernel ring buffer via dmesg:

# Check for active Intel VT-d or AMD-Vi initialisation
sudo dmesg | grep -iE "dmar|iommu|amd-vi"

Look for confirmation entries:

• Intel: DMAR: IOMMU enabled or Intel-IOMMU: enabled.
• AMD: AMD-Vi: Enabling IOMMU at ... and AMD-Vi: Initialized for Mapping.

If IOMMU is absent, append intel_iommu=on iommu=pt (for Intel) or amd_iommu=on iommu=pt (for AMD) to your kernel boot parameters in /etc/default/grub, execute sudo update-grub, and reboot.

Auditing IOMMU Groups via sysfs and Bash

The Linux kernel exposes all populated IOMMU group mappings within the /sys/kernel/iommu_groups/ directory hierarchy. Because standard directory listings only display numeric group IDs and PCI bus addresses, a shell script is used to cross-reference bus IDs with human-readable device names from lspci.

Execute the following diagnostic loop in your terminal:

for g in $(find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V); do
    echo "=== IOMMU Group $(basename "$g") ==="
    for d in "$g"/devices/*; do
        if [ -e "$d" ]; then
            echo -n "    $(basename "$d") "
            lspci -nns "$(basename "$d")" | cut -d' ' -f2-
        fi
    done
done

Analysing Group Isolation and Passthrough Feasibility

Examine the script output to identify clean vs shared IOMMU groups:

=== IOMMU Group 14 ===
    0000:01:00.0 VGA compatible controller [0300]: NVIDIA Corporation AD104 [GeForce RTX 4070] [10de:2786]
    0000:01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:22bc]

=== IOMMU Group 15 ===
    0000:02:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I225-V [8086:15f3]

=== IOMMU Group 16 ===
    0000:00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:7a38]
    0000:03:00.0 Non-Volatile memory controller [0108]: Samsung Electronics NVMe SSD [144d:a80a]
    0000:00:1c.4 PCI bridge [0604]: Intel Corporation Device [8086:7a3c]
    0000:04:00.0 Network controller [0280]: Intel Corporation Wi-Fi 6 AX201 [8086:43f0]

Evaluate group composition:

• Clean Multi-Function Devices (Group 14): The graphics card and its HDMI audio codec share Group 14. This is normal and expected; both functions are bound together to VFIO for GPU passthrough.
• Clean Isolated Device (Group 15): The Intel 2.5GbE NIC sits entirely alone in Group 15, meaning it can be detached and passed to a firewall VM with zero host side effects.
• Shared / Clustered Devices (Group 16): The NVMe storage drive and Wi-Fi card share Group 16 through an upstream chipset bridge that lacks ACS. Passing the NVMe drive alone is prohibited unless the Wi-Fi card is also unseated or passed along.

Overriding Bad Topology: ACS and Slot Migration

If a critical device is trapped in an overcrowded IOMMU group with motherboard peripherals:

• Physical PCIe Slot Migration: Relocate the expansion card to a primary CPU-attached PCIe x16 slot. CPU-connected slots invariably possess clean, dedicated IOMMU groups, unlike chipset-multiplexed secondary slots.
• Enable ACS in BIOS: Check UEFI firmware settings for PCIe Access Control Services (ACS), ARI Support, or PCIe Ten-Bit Tag Support and ensure they are explicitly enabled.

Proactively auditing IOMMU groups via sysfs provides Linux administrators with definitive architectural insight to architect secure, rock-solid hardware virtualization environments.