How to Find and Delete Duplicate Files in Linux Terminal Using fdupes

Over months and years of managing file servers, compiling code repositories, and archiving media backups, redundant identical files inevitably accumulate across your storage drives. Identifying and purging these duplicates by hand is virtually impossible when dealing with tens of thousands of nested folders. While simple scripts comparing filenames are error-prone (since identical files often have different names, and different files can share identical names), accurate deduplication requires rigorous byte-level comparison.

In the Linux terminal ecosystem, fdupes is the quintessential utility for identifying identical files. It operates by first comparing file sizes, then computing MD5 cryptographic signatures, and ultimately performing byte-by-byte verification before presenting duplicates for review or automated deletion.

Installing fdupes on Major Linux Distributions

Because fdupes is an established standard utility, it resides in the default package repositories of all major distributions:

Ubuntu, Debian, and Linux Mint

sudo apt update && sudo apt install -y fdupes

Fedora, RHEL, and Rocky Linux

sudo dnf install -y fdupes

Arch Linux and Manjaro

sudo pacman -S fdupes

Verify the utility installation and view version details by running:

fdupes --version

Scanning Directories Recursively for Duplicate Files

To scan a directory and all of its nested subdirectories, pass the -r (recursive) flag followed by the path:

fdupes -r /path/to/target/folder

The command outputs grouped lists of files that share identical contents, separated by blank lines:

/path/to/target/folder/documents/report_final.pdf
/path/to/target/folder/backup/report_copy.pdf

/path/to/target/folder/photos/IMG_2041.jpg
/path/to/target/folder/archive/IMG_2041_edit.jpg

Displaying File Sizes and Summary Statistics

To see how much storage capacity duplicate files are consuming, combine the size reporting and summary flags:

fdupes -r -S /path/to/folder

The -S flag prints the exact byte size next to each duplicate cluster, as well as a consolidated total summary at the end of the scan (e.g., “142 duplicate files found, reclaiming 4.2 GB of storage”).

Interactively Deleting Duplicate Files

If you want to review and selectively delete duplicates one group at a time while preserving your preferred copy, invoke the -d (delete) parameter:

fdupes -r -d /path/to/folder

When duplicates are discovered, fdupes pauses and displays an interactive prompt asking which files to retain:

[1] /folder/photos/vacation.jpg
[2] /folder/backup/vacation.jpg

Set numbers of files to preserve (1-2) or 'all' to preserve all: 1

Typing 1 instructs fdupes to keep the first file and immediately delete the second duplicate.

Automating Deletion Without Manual Prompts

For automated cleanup scripts where you want fdupes to automatically keep the first occurrence of every file and silently delete all subsequent duplicate copies, combine -d with the -N (noprompt) flag:

fdupes -r -d -N /path/to/folder

Caution: Always perform a dry run using fdupes -r first to inspect matching sets before invoking -N, ensuring no critical hard links or archival copies are deleted unintentionally.

Replacing Duplicates with Hard Links

If your applications require duplicate file paths to remain intact across the directory tree but you still want to reclaim disk space, you can instruct fdupes to convert duplicates into NTFS/ext4 hard links using the -H parameter (on supported versions):

fdupes -r -H /path/to/folder

Hard linking ensures all existing file paths remain fully functional while physical disk blocks are consolidated to a single inode, reclaiming physical storage instantly.