When transferring files on an Ubuntu system, the standard cp (copy) and mv (move) commands are sufficient for basic tasks. However, if you are copying massive directories, backing up a server, or transferring data over a network, these basic commands fall short. If a network drops during a 100GB file transfer using cp, you have to start all over again.
This is where rsync (Remote Sync) becomes essential. It is a powerful, fast, and incredibly versatile command-line utility for transferring and synchronising files. The defining feature of rsync is its “delta-transfer algorithm,” which means if a transfer is interrupted, it will only copy the differences between the source and destination when resumed, rather than transferring the entire file again.
The Basic Syntax of rsync
The standard structure of the rsync command looks like this:
rsync [options] [source] [destination]
For example, to copy a file named backup.tar.gz from your current directory to the /var/backups/ directory, you would use:
rsync backup.tar.gz /var/backups/
Crucial rsync Flags Explained
While the basic command works, rsync is almost always used with a combination of flags to control exactly how the files are handled. The most common combination is -avz.
rsync -avz /home/user/documents/ /mnt/external-drive/documents/
Here is what those flags mean:
- -a (Archive): This is the most important flag. It acts recursively (copying all folders inside folders) and preserves all file permissions, ownership data, symbolic links, and timestamps. If you are creating a backup, you must use the
-aflag. - -v (Verbose): This tells
rsyncto print a list of every file it is currently copying to your terminal screen, so you can see exactly what is happening. - -z (Compress): This compresses the file data during the transfer. This is highly recommended when transferring files over a slow network connection, as it significantly reduces the amount of data sent.
Adding a Progress Bar
If you are transferring a massive file (like a virtual machine image), you will want to know how long it will take. Add the --progress flag to display a real-time progress bar, transfer speed, and estimated time remaining.
rsync -av --progress /source/folder/ /destination/folder/
Transferring Files to a Remote Server
rsync is natively capable of transferring files over SSH to remote servers. The syntax simply requires you to specify the remote user and IP address.
To push a folder from your local Ubuntu machine to a remote server:
rsync -avz /local/folder/ [email protected]:/remote/folder/
You can also pull files from a remote server to your local machine by reversing the order:
rsync -avz [email protected]:/remote/folder/ /local/folder/
The system will prompt you for the SSH password (or use your SSH keys if configured) and then begin the transfer securely.
The Delete Flag (Synchronising Directories)
If you want the destination directory to be an exact, mirror-image copy of the source directory, you need to use the --delete flag.
Normally, if you delete a file from the source and run rsync again, the file remains in the destination. The --delete flag tells rsync to delete files in the destination folder if they no longer exist in the source folder.
rsync -av --delete /source/ /destination/
Warning: Use the --delete flag with extreme caution. If you accidentally specify the wrong destination, it will permanently delete the contents of that folder. It is always best to run a “dry run” first.
Performing a Dry Run
If you are unsure exactly what rsync is going to do (especially when using --delete), you can append the --dry-run flag. This simulates the transfer and prints out exactly which files would be copied or deleted, without actually modifying anything on your hard drive.