When auditing server configurations or cross-referencing customer databases in the Linux terminal, you frequently need to compare two massive text files to see what data they share and what data is unique to each. While the diff command is excellent for showing complex line-by-line code changes, it is overly complicated for simple list comparisons. For discovering commonality and uniqueness between two lists, the comm (compare) command is the most efficient tool available.
What Does the comm Command Do?
The comm command reads two files and outputs a three-column visual report.
- Column 1: Lines that are unique to the first file.
- Column 2: Lines that are unique to the second file.
- Column 3: Lines that appear identically in both files.
By using suppression flags, you can easily filter this output to show exactly what you need, such as hiding all unique data to display only the exact matches.
The Golden Rule: Files Must Be Sorted
The comm utility relies on a highly efficient sequential reading algorithm. It does not scan the entire document hunting for matches; it simply reads line one of both files and moves downward. Because of this, both files must be sorted alphabetically before the command will work correctly. If you run the command on unsorted files, Linux will throw an error stating “file is not in sorted order”.
You can quickly sort your files using the standard sort command:
sort list1.txt > sorted1.txt
sort list2.txt > sorted2.txt
Executing a Basic Comparison
Once you have two alphabetically sorted files, executing the command is simple.
- Open your terminal application.
- Type the command followed by the two filenames:
comm sorted1.txt sorted2.txt
Press Enter. The output will immediately print to your screen in the three-column format. If the files are large, the formatting will quickly become chaotic as it spans across your terminal window.
Using Suppression Flags
The true power of the comm command lies in its suppression flags (-1, -2, and -3). These flags instruct the utility to hide specific columns from the output.
For example, if you are comparing two email lists and only want to see the addresses that exist in both files, you want to view Column 3. Therefore, you must suppress Column 1 and Column 2.
comm -12 sorted1.txt sorted2.txt
This command hides the unique lines and outputs a clean, single-column list containing only the exact overlap between the two files.
Conversely, if you want to find new customers who appear in the second list but not in the first, you only want to view Column 2. You must suppress Column 1 and Column 3.
comm -13 sorted1.txt sorted2.txt
Comparing Files Without Creating New Files
If you do not want to clutter your hard drive by manually generating sorted1.txt and sorted2.txt, you can use a bash feature called process substitution to sort the files dynamically inside the command itself.
comm -12 <(sort list1.txt) <(sort list2.txt)
This advanced syntax temporarily sorts both files in system memory and immediately feeds the sorted data into the comm command, saving you time and disk space.