How to Use the sed Command in Linux for Stream Editing, Regex Replacement, and In-Place File Transforms

In Unix and Linux systems engineering, text processing is a fundamental daily discipline. While graphical text editors and interactive terminal editors like Nano or Vim are indispensable for manual configuration adjustments, automated infrastructure scripts, CI/CD pipelines, and bulk provisioning routines require non-interactive, stream-oriented transformation tools. For these operational scenarios, the stream editor—sed—serves as the foundational standard.

Operating by reading input line by line into an internal pattern space, applying designated transformation instructions, and flushing the modified string to standard output, sed can parse gigabyte-scale logs, refactor source code repositories, and update system configuration files without human intervention. This technical guide explores sed syntax, regular expression replacement structures, in-place editing safety rules, selective line targeting, and multi-command execution.

Basic Syntax and Substitution Structure

The vast majority of sed operations leverage the substitution command (s):

sed 's/pattern/replacement/flags' input_file

Component	Function	Operational Role
s	Substitute directive	Instructs sed to search for the pattern and replace it.
/	Delimiter character	Separates pattern, replacement, and flags. Can be replaced with |, #, or : when editing file paths.
pattern	Regular expression	Defines the string or POSIX regular expression to match.
replacement	Target replacement string	The string injected in place of the matched pattern; supports backreferences (\1, &).
flags	Execution modifiers	Modifiers such as g (global match per line), i (case-insensitive), or numeric occurrences.

Basic Search and Replace Examples

Replacing the First Occurrence per Line

By default, sed replaces only the first match on each line:

sed 's/localhost/127.0.0.1/' config.yaml

Replacing All Occurrences on Every Line (Global Flag)

Adding the g modifier ensures every instance across each line is transformed:

sed 's/DEBUG/INFO/g' server.log

Case-Insensitive Matching

Using the I or i flag enables case-insensitive pattern matching:

sed 's/production/staging/gI' deployment.env

Using Alternate Delimiters for Filesystem Paths

When searching or replacing filesystem directory paths, standard forward slashes cause “leaning toothpick syndrome” and syntax collision errors. Substitute any alternate character as the delimiter:

# Using # as the delimiter
sed 's#/var/log/nginx#/mnt/storage/logs#g' nginx.conf

Modifying Files in Place Safely (-i Flag)

By default, sed prints results to stdout, leaving the original source file untouched. To commit changes directly to the file on disk, use the in-place flag (-i):

In-Place Editing with Automated Backup (.bak)

Always provide a backup extension when performing in-place edits in automated scripts:

sed -i.bak 's/port: 8080/port: 443/' application.yml

This creates application.yml.bak containing the original untouched file before committing edits to application.yml.

Warning regarding macOS vs. Linux: GNU sed on Linux permits bare -i (e.g., sed -i 's/.../' file), whereas BSD sed on macOS strictly mandates an explicit empty string argument for in-place edits without backup (sed -i '' 's/.../' file).

Targeting Specific Lines and Address Ranges

You can restrict sed operations to specific line numbers or pattern blocks:

Modifying Only Line 15

sed -i '15s/false/true/' settings.json

Modifying a Continuous Line Range (Lines 20 through 40)

sed -i '20,40s/enabled = 0/enabled = 1/' database.conf

Deleting Matching Lines (d Command)

To delete lines matching a comment character or empty blank lines:

# Delete all lines starting with a comment hash
sed -i '/^#/d' /etc/example.conf

# Delete all blank or empty lines
sed -i '/^$/d' /etc/example.conf

Advanced Regex Transforms and Backreferences

Leveraging Extended Regular Expressions (-E flag) and capture groups allows programmatic string restructuring:

Swapping Word Positions Using Capture Groups

Suppose you have a list formatted as LastName, FirstName and wish to convert it to FirstName LastName:

sed -E 's/([A-Za-z]+), ([A-Za-z]+)/\2 \1/' contacts.txt

Here, \1 references the first parenthesized capture group, and \2 references the second.

Appending Suffixes to Matching Strings (& Operator)

The ampersand symbol (&) in the replacement string represents the entire matched pattern:

sed 's/[0-9]\{4\}/[YEAR: &]/g' timeline.txt

Chaining Multiple sed Instructions

When multiple transformations must be evaluated sequentially in a single pass without writing intermediate files to disk:

# Using multiple -e expressions
sed -e 's/alpha/beta/g' -e 's/dev/prod/g' app.config

# Using semicolons inside a single script block
sed -i 's/foo/bar/g; s/cat/dog/g; 5,10d' data.txt

Operational Best Practices

• Dry-Run Verification: Always run sed without the -i flag first, piping output to head -n 20 or less, to inspect output before modifying production configuration files.
• Preserving Symlinks: By default, sed -i breaks filesystem symlinks by replacing the target with a new inode file. Use the --follow-symlinks parameter to preserve symbolic links:

  sed -i --follow-symlinks 's/old/new/' /etc/nginx/sites-enabled/default