How to Automatically Sync Your Local Directory to an AWS S3 Bucket Using AWS CLI

Backing up critical server data to Amazon S3 is a standard practice for ensuring data durability. However, manually running upload commands every day is tedious and prone to human error. By combining the official AWS Command Line Interface (CLI) with the Linux cron daemon, you can configure your Ubuntu server to automatically synchronize a local directory with your remote S3 bucket on a set schedule.

How to Configure the AWS CLI

Before you can automate the sync, you must install the AWS CLI and configure it with the appropriate IAM credentials.

1. Log into your Ubuntu server via SSH.
2. Update your package lists: sudo apt update
3. Install the AWS CLI tool: sudo apt install awscli -y
4. Once installed, configure your credentials by running: aws configure
5. You will be prompted to enter your AWS Access Key ID, your Secret Access Key, your preferred Default region name (e.g., us-east-1), and your Default output format (you can leave this as json).

Your server now has permission to interact with your S3 buckets securely.

How to Use the S3 Sync Command

The aws s3 sync command is incredibly powerful because it only uploads files that are new or have been modified since the last sync, saving significant bandwidth and time compared to a standard copy command.

Test the command manually first to ensure it works. To sync a local folder named /var/backups/database to an S3 bucket named my-company-backups, run:

aws s3 sync /var/backups/database s3://my-company-backups

If the command executes successfully and you see your files appear in the AWS console, you are ready to automate it.

How to Automate the Sync with Crontab

We will use the cron daemon to run this exact command every night at 2:00 AM.

1. Open your crontab editor by running: crontab -e (Use sudo crontab -e if the directory you are backing up requires root permissions to read).
2. Scroll to the bottom of the file and paste the following cron expression:

   0 2 * * * /usr/bin/aws s3 sync /var/backups/database s3://my-company-backups

3. Save the file and exit the text editor.

It is crucial to use the absolute path to the AWS binary (/usr/bin/aws) in your crontab, rather than just typing aws, because cron runs in a restricted environment and often lacks the standard user PATH variables required to locate installed applications.