When it comes to copying files across a Linux system or backing up data to a remote server, standard commands like cp (copy) and scp (secure copy) are often too basic for large-scale operations. If a massive file transfer fails halfway through due to a network drop, relying on these basic commands means you have to start the entire process all over again.
The professional solution is to use rsync, a powerful, highly flexible command-line utility designed for fast, incremental file transfers.
What is Rsync and Why is it Better?
rsync stands for “remote sync”. Unlike traditional copy commands that blindly overwrite existing data, rsync utilises an advanced algorithm to compare the source directory with the destination directory.
It only transfers the differences between the two locations. If you have a folder with 100 gigabytes of data and you only modify a single text file, running an rsync command will only transfer the changed portions of that specific text file. This saves a massive amount of time, processing power, and network bandwidth, making it the industry standard tool for automated backups.
The Basic Syntax of Rsync
The structure of an rsync command is very similar to standard copy commands:
rsync [options] source destination
How to Sync Files Locally
If you want to back up a folder on your computer to an external hard drive, you can run a local sync.
rsync -av /path/to/source_folder/ /path/to/destination_folder/
Let’s break down the essential flags used here:
- -a (archive): This is the most important flag. It tells rsync to sync directories recursively, preserve symbolic links, and maintain all file permissions, ownerships, and timestamps.
- -v (verbose): This tells the terminal to display a detailed visual output of exactly which files are being transferred, giving you peace of mind that the command is working.
Crucial Tip: Notice the trailing slash (/) on the source folder. If you include the slash (source_folder/), rsync will copy the contents of the folder. If you omit the slash (source_folder), rsync will copy the folder itself and place it inside the destination.
How to Sync Files to a Remote Server
Transferring files to a remote server requires an SSH connection. Rsync handles this seamlessly.
rsync -avz /local/folder/ username@remote_host:/remote/folder/
Here, we have added a new flag:
- -z (compress): This compresses the file data during the transfer. This is highly recommended for remote transfers as it significantly speeds up the process over slow network connections. The data is automatically decompressed upon arrival.
How to Delete Extraneous Files on the Destination
By default, rsync will only add new or modified files to the destination. If you delete a file from your source folder, rsync will leave the old copy sitting in the destination folder.
If you are trying to create an exact, identical mirror for a backup, this behaviour will cause your backup drive to slowly fill up with obsolete files. To force rsync to delete files in the destination that no longer exist in the source, use the --delete flag.
rsync -av --delete /local/folder/ /backup/drive/
By mastering rsync and its various flags, you can establish incredibly efficient, bandwidth-friendly backup routines that keep your Linux infrastructure secure.
