How to Trace Network Interface Queue Packet Rates in Linux Using netqtop

Modern enterprise servers rely on multi-queue network interface cards (NICs) supporting Receive Side Scaling (RSS) and multi-queue transmit (TX) engines. By distributing network packets across multiple hardware queues—each serviced by a dedicated CPU core via MSI-X interrupts—Linux systems achieve 40GbE, 100GbE, and 200GbE line rates without saturating a single core. However, imperfect hashing algorithms, pinned softirq affinities, or heavy single-flow connections can cause severe queue imbalance where one hardware queue drops packets while others sit idle. The BCC eBPF tool netqtop provides real-time per-queue packet throughput diagnostics.

Understanding Multi-Queue Networking and RSS Hashing

When packets arrive at a physical network port, the NIC hardware computes a Toeplitz or symmetric hash over the packet headers (the 4-tuple: source IP, destination IP, source port, destination port). The resulting hash maps the packet to a designated RX hardware queue:

  • Receive Side Scaling (RSS): Hardware distributes incoming flows evenly across available RX descriptor rings.
  • Transmit Packet Steering (XPS): Kernel maps outbound socket transmissions to specific TX queues based on the sending CPU core to maximize cache locality.
  • Queue Skew: If thousands of connections share identical IP/port combinations (such as behind reverse proxies, VPN concentrators, or single large database replications), hash polarization directs all traffic into a single queue, leading to ring buffer overflows and softirq stalls.

Installing the BCC Tools Suite

The netqtop utility is distributed as part of the BPF Compiler Collection (BCC):

# Ubuntu / Debian
sudo apt-get update
sudo apt-get install -y bpfcc-tools linux-headers-$(uname -r)

# RHEL 9 / Rocky Linux 9 / AlmaLinux
sudo dnf install -y bcc-tools kernel-devel-$(uname -r)

# Verify executable path
which netqtop || which /usr/share/bcc/tools/netqtop

Running netqtop to Profile Active Hardware Queues

Launch netqtop specifying the target network interface using the -n flag:

# Monitor eth0 queues with a 1-second refresh interval
sudo /usr/share/bcc/tools/netqtop -n eth0

# Monitor with custom interval and display in megabytes
sudo /usr/share/bcc/tools/netqtop -n eth0 -i 2

The tool outputs an interactive, top-like tabulated terminal dashboard:

Thu Nov 11 05:32:14 2026
QueueID      BPS(RX)      PPS(RX)      BPS(TX)      PPS(TX)
0            14.2 MB      11240        8.1 MB       6400
1            13.9 MB      10980        7.9 MB       6250
2            842.1 MB     642100       120.4 MB     92000
3            14.1 MB      11100        8.0 MB       6310
Total        884.3 MB     675420       144.4 MB     110960

Interpreting Queue Imbalances and Hotspots

The output above demonstrates an acute queue imbalance hotspot:

  • Queues 0, 1, and 3 handle a modest ~14 MB/s and ~11,000 packets per second.
  • Queue 2 is severely congested, handling 842 MB/s and 642,000 packets per second—accounting for over 95% of total ingress bandwidth.
  • If Queue 2 exhausts its hardware descriptor ring, packets mapped to Queue 2 will be discarded, even though total system capacity is nowhere near saturation.

Resolving RSS Imbalances via ethtool Hashing Configuration

When netqtop exposes queue skew, investigate and tune NIC hashing rules:

  1. Inspect current RSS hashing algorithms:
    sudo ethtool -k eth0 | grep rx-hashing
    sudo ethtool -n eth0 rx-flow-hash tcp4
  2. Ensure both IP addresses and TCP port numbers are factored into hash calculations (4-tuple hashing):
    # Enable IP and Port hashing for IPv4 TCP
    sudo ethtool -N eth0 rx-flow-hash tcp4 sdfn
    
    # Enable IP and Port hashing for IPv4 UDP
    sudo ethtool -N eth0 rx-flow-hash udp4 sdfn
  3. Inspect interrupt CPU core bindings:
    cat /proc/interrupts | grep -E "eth0|mlx|i40e"
    Verify that irqbalance is distributing queue interrupts across distinct CPU physical cores rather than piling interrupts onto core 0.

By auditing per-queue throughput with netqtop, systems engineers can detect hash skew, tune hardware queues, and unlock the true multithreaded capacity of enterprise Linux networking hardware.

Get the best tech tips delivered straight to your inbox.

Join thousands of readers mastering Apple, Google, Microsoft, and Linux.