How to Find Duplicate Files in Windows 11 Using PowerShell

Over time, local storage drives accumulate identical copies of photographs, video files, documents, and installer archives scattered across multiple folders. While third-party utilities exist to identify redundant files, Windows 11 includes built-in PowerShell cmdlets that can pinpoint exact duplicate files without requiring third-party software installations. By comparing cryptographic SHA-256 hashes rather than relying on filenames alone, PowerShell guarantees that only byte-for-byte identical duplicates are detected.

Why Use Cryptographic Hashes to Identify Duplicates?

Traditional file searches rely on filenames, file sizes, or modification timestamps to identify matches. However, these methods are inherently flawed:

  • Renamed Copies: Two identical media files may possess completely different filenames (such as IMG_2041.jpg and banner_final_v2.jpg) yet contain identical underlying bytes.
  • Identical File Sizes: Unrelated files can share identical byte counts while containing completely different content, leading to false positives.
  • Cryptographic Assurance: Generating an SHA-256 hash using Get-FileHash inspects the exact data stream of the file. If two files share an identical hash value, they are guaranteed to be exact byte-level duplicates regardless of where they reside or how they are named.

Step 1: Open PowerShell and Define the Target Directory

Launch Windows PowerShell to initiate the scanning process:

  1. Right-click the Start button on the taskbar and select Terminal or Windows PowerShell.
  2. Navigate to the folder you wish to audit using the Set-Location cmdlet, or store the target path in a variable:
    $TargetFolder = "C:\Users\YourUsername\Documents"
  3. Ensure you replace the folder path with your desired target directory, such as a downloads folder, media library, or secondary storage partition.

Step 2: Scan and Identify Duplicate Files by Hash

To recursively inspect all files within the folder hierarchy, compute their cryptographic hashes, and group identical files together, run the following command block:

Get-ChildItem -Path $TargetFolder -File -Recurse |
    Get-FileHash -Algorithm SHA256 |
    Group-Object -Property Hash |
    Where-Object { $_.Count -gt 1 } |
    ForEach-Object {
        [PSCustomObject]@{
            DuplicateCount = $_.Count
            Hash           = $_.Name
            Files          = ($_.Group | Select-Object -ExpandProperty Path)
        }
    } | Format-List

This script executes three sequential operations:

  1. Get-ChildItem -File -Recurse: Traverses all nested directories while ignoring folder containers.
  2. Get-FileHash -Algorithm SHA256: Computes the unique cryptographic fingerprint for every detected file.
  3. Group-Object -Property Hash and Where-Object: Aggregates identical hash signatures and filters for hashes that occur more than once, instantly presenting the full file paths of all duplicate copies.

Step 3: Export Duplicates to a CSV File for Review

When auditing large storage volumes with thousands of items, reviewing duplicate reports directly in the terminal can be cumbersome. Exporting the results to a CSV spreadsheet enables structured review in Microsoft Excel:

Get-ChildItem -Path $TargetFolder -File -Recurse |
    Get-FileHash -Algorithm SHA256 |
    Group-Object -Property Hash |
    Where-Object { $_.Count -gt 1 } |
    Select-Object -ExpandProperty Group |
    Select-Object Path, Hash |
    Export-Csv -Path "$HOME\Desktop\duplicate_files_report.csv" -NoTypeInformation

Opening the generated duplicate_files_report.csv file allows you to sort duplicates by hash and review folder locations before undertaking manual or automated deletion.

Step 4: Safely Remove Redundant Duplicate Copies

If you wish to retain the first instance of every duplicate file while automatically deleting all subsequent redundant copies, execute the following automated cleanup pipeline:

Get-ChildItem -Path $TargetFolder -File -Recurse |
    Get-FileHash -Algorithm SHA256 |
    Group-Object -Property Hash |
    Where-Object { $_.Count -gt 1 } |
    ForEach-Object {
        # Retain the first file in the group and select the remaining duplicates
        $_.Group | Select-Object -Skip 1 | ForEach-Object {
            Write-Host "Removing duplicate:" $_.Path -ForegroundColor Yellow
            Remove-Item -Path $_.Path -WhatIf
        }
    }

The command includes the -WhatIf parameter by default to simulate deletion without touching actual files. Once you confirm the printed targets are safe to delete, remove -WhatIf to permanently purge the redundant duplicates and reclaim local drive space.

Get the best tech tips delivered straight to your inbox.

Join thousands of readers mastering Apple, Google, Microsoft, and Linux.