How to Configure IPsec Pre-Shared Key (PSK) Authentication in Windows 11 Using PowerShell

In enterprise network environments and secure remote branches, establishing encrypted IPsec tunnels or transport associations protects sensitive intranet traffic against eavesdropping and packet tampering. While enterprise Public Key Infrastructure (PKI) and Kerberos are ideal for domain-joined machines, standalone Windows 11 workstations, isolated test appliances, and cross-platform appliances often rely on Pre-Shared Key (PSK) authentication. Using the native NetSecurity module in PowerShell, network engineers can script, deploy, and validate IPsec PSK authentication without navigating complex graphical firewall interfaces.

Understanding Pre-Shared Key Authentication in IPsec Phases

The IPsec negotiation protocol establishes secure communications through two sequential phases governed by the Internet Key Exchange (IKEv1 or IKEv2):

• Main Mode (Phase 1): Authenticates the communicating endpoints and negotiates an encrypted channel using the Pre-Shared Key secret string. Once verified, a Security Association (IKE SA) is established.
• Quick Mode (Phase 2): Uses the established Phase 1 tunnel to negotiate transport or tunnel parameters (ESP/AH), cipher suites (such as AES-GCM-256), and traffic selectors, generating the final Child SAs for data transmission.

When implementing PSK authentication, Windows 11 requires both the definition of an Authentication Proposal (specifying the pre-shared secret) and a Connection Security Rule binding the proposal to specific remote IP subnets and transport protocols.

Constructing the IPsec Authentication Proposal

To begin configuration, open an elevated PowerShell prompt on Windows 11 (Right-click Start > Terminal (Admin) or PowerShell (Admin)).

The first step creates a Phase 1 authentication proposal object using the New-NetIPsecAuthProposal cmdlet:

$PskSecret = "EnterpriseSecurePresharedPassphrase2026!"
$AuthProposal = New-NetIPsecAuthProposal -Machine -PresharedKey $PskSecret

The -Machine switch specifies that endpoint identification is verified at the host machine level rather than against interactive user accounts. Note that while pre-shared keys offer straightforward deployment, they must be composed of long, high-entropy strings to defend against brute-force attacks on Phase 1 handshakes.

Creating Phase 1 and Phase 2 Security Cryptographic Sets

To ensure robust cryptographic posture, define the encryption and integrity algorithms used during Phase 1 (Main Mode) negotiation:

New-NetIPsecPhase1AuthSet -DisplayName "IPsec-PSK-Phase1-Set" `
    -Proposal $AuthProposal

New-NetIPsecMainModeCryptoSet -DisplayName "IPsec-MM-Crypto-Set" `
    -Proposal (New-NetIPsecMainModeCryptoProposal -Cipher AES256 -Hash SHA256 -DHGroup Group14)

This configuration specifies AES-256 for symmetric payload encryption, SHA-256 for integrity hashing, and Diffie-Hellman Group 14 (2048-bit MODP) for secure ephemeral key exchange.

Deploying the Connection Security Rule

With authentication sets and cryptographic profiles created, bind them into a functional connection security rule using the New-NetIPsecRule cmdlet:

New-NetIPsecRule -DisplayName "Secure-Branch-Intranet-Rule" `
    -Description "Enforce IPsec transport mode with PSK authentication to remote subnet" `
    -LocalAddress "192.168.1.50" `
    -RemoteAddress "10.200.0.0/16" `
    -Mode Transport `
    -RequireInbound Require `
    -RequireOutbound Require `
    -Phase1AuthSet "IPsec-PSK-Phase1-Set" `
    -MainModeCryptoSet "IPsec-MM-Crypto-Set" `
    -Platform 2.10

Key parameters in this command include:

• -Mode Transport: Encrypts payload traffic end-to-end between the local machine and remote endpoints without rewriting the underlying IP routing headers.
• -RequireInbound Require / -RequireOutbound Require: Mandates that all traffic entering or leaving for the specified remote subnet (10.200.0.0/16) must be encapsulated within IPsec. Unencrypted packets are dropped immediately.
• -Platform 2.10: Restricts policy compatibility to modern Windows NT kernels, guaranteeing proper hardware-accelerated AES offloading on compatible network adapters.

Auditing Active IPsec Associations and Troubleshooting

After traffic begins flowing toward the destination subnet, verify that the Phase 1 and Phase 2 Security Associations have been established successfully:

1. Inspecting Main Mode Security Associations

Get-NetIPsecMainModeSA | Format-Table LocalAddress, RemoteAddress, PeerId, SelectedCipher, SelectedHash

If Phase 1 fails to complete, check whether the pre-shared key strings match exactly on both peers. A single mismatched character or trailing whitespace character will cause immediate negotiation termination.

2. Inspecting Quick Mode Security Associations

Get-NetIPsecQuickModeSA | Format-Table LocalAddress, RemoteAddress, Encapsulation, OutboundPackets, InboundPackets

Non-zero packet counters on inbound and outbound queues confirm that live data is actively passing through the hardware encryption engine.

3. Removing or Disabling the IPsec Rule

If you need to revoke the policy or conduct maintenance testing, disable or delete the rule using straightforward PowerShell commands:

# Temporarily disable the rule
Set-NetIPsecRule -DisplayName "Secure-Branch-Intranet-Rule" -Enabled False

# Completely remove the rule and authentication sets
Remove-NetIPsecRule -DisplayName "Secure-Branch-Intranet-Rule"
Remove-NetIPsecPhase1AuthSet -DisplayName "IPsec-PSK-Phase1-Set"
Remove-NetIPsecMainModeCryptoSet -DisplayName "IPsec-MM-Crypto-Set"