How to Configure IPsec Denial of Service Protection (DoSP) in Windows 11 Using PowerShell

Internet Protocol Security (IPsec) provides robust cryptographic authentication and encryption for enterprise communication, but the initial Internet Key Exchange (IKE and AuthIP) negotiations require considerable CPU and memory resources to process Diffie-Hellman handshakes. Malicious actors frequently exploit this asymmetric cost by flooding servers or endpoints with spoofed negotiation packets, attempting to cause state exhaustion and denial-of-service (DoS) conditions. Windows 11 incorporates IPsec Denial of Service Protection (DoSP)—a specialised, high-performance filtering component in the Windows Filtering Platform (WFP) that defends against key exchange floods. PowerShell gives administrators direct cmdlets to inspect, configure, and enforce DoSP thresholds.

Understanding How IPsec DoSP Mitigates Attacks

IPsec DoSP sits at the lowest layers of the Windows network stack, intercepting traffic before full cryptographic state entries are committed into memory. When an endpoint experiences high-volume unauthenticated IKE or AuthIP connection attempts, DoSP applies rate-limiting algorithms, enforces stateless cookie challenges, and tracks per-internal-subnet state allocations.

If an untrusted host floods your system with incomplete handshakes, DoSP discards the bogus packets without engaging expensive cryptographic routines. When deployed alongside hardware IPsec task offload and strict IPsec main mode rules, DoSP ensures critical tunnels remain resilient against distributed flooding attacks.

Verifying Administrator Privileges in Windows 11

Modifying kernel-level packet filtering and IPsec DoSP settings requires administrative rights:

  1. Right-click the Start icon on your taskbar or press Windows Key + X.
  2. Select Terminal (Admin) or PowerShell (Admin).
  3. Click Yes when prompted by the User Account Control (UAC) dialogue.

Auditing Current IPsec DoSP Settings

To inspect active DoSP settings and global rate-limiting configurations, execute Get-NetIPsecDospSetting:

Get-NetIPsecDospSetting

For a detailed breakdown of all rate-limiting parameters, pipe the result into Format-List:

Get-NetIPsecDospSetting | Format-List *

Key parameters reported by PowerShell include:

  • Name: The identifier assigned to the DoSP profile (such as Default).
  • PublicIPv6AddressRateLimitPerMinute: Maximum permissible unauthenticated IPv6 connection attempts per minute from public addresses.
  • PublicIPv4AddressRateLimitPerMinute: Rate-limit ceiling for incoming unauthenticated IPv4 packets per minute.
  • InternalIPv4SubnetRateLimitPerMinute: Maximum allowed connection requests originating from internal network subnets.
  • StateIdleTimeoutMinutes: Duration before incomplete handshake state blocks are purged from memory.

Configuring Connection Rate Limits with Set-NetIPsecDospSetting

To tighten protection against aggressive port scanning and connection exhaustion floods, adjust the inbound threshold using Set-NetIPsecDospSetting:

Set-NetIPsecDospSetting -Name "Default" -PublicIPv4AddressRateLimitPerMinute 60 -PublicIPv6AddressRateLimitPerMinute 60

This command limits incoming key exchange negotiations from any single public IP address to 60 packets per minute. Any excess negotiation packets exceeding this quota are dropped immediately at the network driver level.

If your Windows 11 device acts as a branch office VPN concentrator or direct IPsec gateway, you can also tune the internal subnet limits:

Set-NetIPsecDospSetting -Name "Default" -InternalIPv4SubnetRateLimitPerMinute 1200

This ensures legitimate corporate workstations can re-negotiate keys after network transitions without triggering false-positive lockouts.

Adjusting State Timeouts for Incomplete Handshakes

During an active attack, attackers deliberately abandon half-open handshakes, hoping memory pools fill up before old records expire. You can reduce the idle state lifespan to release memory more aggressively:

Set-NetIPsecDospSetting -Name "Default" -StateIdleTimeoutMinutes 2

Reducing the timeout from default values ensures uncompleted security associations are reclaimed promptly, leaving memory available for authentic peer traffic governed by your IPsec main mode cryptographic sets.

Auditing DoSP Drop Counters and Firewall Diagnostics

To evaluate whether DoSP has actively dropped spoofed or malicious connection floods, query the dynamic IPsec statistics using Netsh or PowerShell:

netsh ipsec dynamic show stats

Inspect the output under the DoS Protection Statistics heading. Review metrics such as Packets Discarded, Rate Limit Drops, and State Allocation Failures. If these counters increment during network stress tests, DoSP is successfully shielding the Windows 11 IPsec stack from denial-of-service exhaustion while allowing legitimate firewall traffic through active firewall profile policies.

Get the best tech tips delivered straight to your inbox.

Join thousands of readers mastering Apple, Google, Microsoft, and Linux.