When writing bash scripts that launch multiple background processes, you often need the script to pause and wait for those processes to finish before continuing to the next step. Without proper synchronisation, your script might attempt to process the output of a background task before that task has completed, leading to errors or incomplete data.
The wait command is a built-in bash utility designed specifically for this purpose. It instructs the shell to pause execution until one or more background processes have finished, and it returns the exit status of the completed process so you can check whether it succeeded or failed.
Understanding Background Processes in Bash
Before using wait, it is helpful to understand how background processes work in bash. When you append an ampersand (&) to the end of a command, the shell starts that command in the background and immediately returns control to you (or to the next line of the script) without waiting for it to finish.
For example:
sleep 10 &
This starts a 10-second sleep process in the background. The shell assigns it a Process ID (PID) and moves on. You can capture this PID using the special variable $!, which always contains the PID of the most recently launched background process.
Waiting for All Background Processes
The simplest use of the wait command is to call it with no arguments. This tells the shell to wait for every currently running background process to finish before proceeding.
Example script:
#!/bin/bash
echo "Starting downloads..."
wget -q https://example.com/file1.zip -O /tmp/file1.zip &
wget -q https://example.com/file2.zip -O /tmp/file2.zip &
wget -q https://example.com/file3.zip -O /tmp/file3.zip &
echo "Waiting for all downloads to complete..."
wait
echo "All downloads finished. Processing files..."
In this script, three wget downloads are launched simultaneously in the background. The wait command then blocks the script until all three have completed. Only then does the script print “All downloads finished” and proceed to the next stage.
Waiting for a Specific Process
If you launch multiple background processes but only need to wait for one specific process to finish, you can pass its PID as an argument to wait.
#!/bin/bash
# Launch two tasks
backup_database &
DB_PID=$!
generate_report &
REPORT_PID=$!
# Wait specifically for the database backup to finish
wait $DB_PID
echo "Database backup completed with exit status: $?"
# The report may still be running at this point
wait $REPORT_PID
echo "Report generation completed with exit status: $?"
The special variable $! captures the PID of the most recently backgrounded process. By storing each PID in a named variable immediately after launching the process, you can later wait for each one individually and check its exit status using $?.
Checking the Exit Status of Background Processes
One of the most important features of the wait command is that it propagates the exit status of the awaited process. This means you can use it to detect failures in background tasks.
#!/bin/bash
compile_project &
COMPILE_PID=$!
wait $COMPILE_PID
if [ $? -ne 0 ]; then
echo "ERROR: Compilation failed!"
exit 1
fi
echo "Compilation succeeded. Deploying..."
If compile_project exits with a non-zero status (indicating an error), the script detects the failure and halts before attempting to deploy broken code.
Using wait with the -n Flag
In newer versions of bash (4.3 and later), the wait command supports the -n flag. Instead of waiting for all background processes or a specific PID, wait -n waits for the next background process to finish (whichever one completes first) and returns its exit status.
This is useful for implementing a “process pool” pattern where you want to launch a fixed number of concurrent jobs and start a new one as soon as any existing job finishes:
#!/bin/bash
MAX_JOBS=3
for file in /data/input/*.csv; do
process_file "$file" &
# If we've hit the max, wait for one to finish
if (( $(jobs -r | wc -l) >= MAX_JOBS )); then
wait -n
fi
done
wait # Wait for any remaining jobs
echo "All files processed."
By incorporating the wait command into your bash scripts, you can safely parallelise workloads while maintaining full control over execution order, error handling, and process synchronisation.