How to Install and Use trurl in Linux to Parse and Manipulate URLs in Shell Scripts

Manipulating URLs within automated shell scripts and terminal pipelines is notoriously prone to edge cases. Bash developers and DevOps engineers frequently rely on fragile regular expressions or brittle sed, awk, and cut commands to parse query parameters, strip tracking arguments, alter protocol ports, or extract hostnames. These manual parsing methods frequently break when encountering URL-encoded characters, custom ports, or international domain names. To solve this, Daniel Stenberg (creator of curl) and the curl project developed trurl: a dedicated, RFC 3986-compliant command-line utility built specifically to parse, modify, and format URLs safely in shell scripts.

Powered by the battle-tested libcurl URL API, trurl handles complex URL components natively—including authentication credentials, path segments, query strings, and fragments—without string-splitting risks. In this tutorial, you will learn how to install trurl on Linux, extract individual URL components, safely append or trim query parameters, normalise web links, and integrate trurl into production shell workflows.

What Is trurl and Why Is It Superior to sed and awk?

Traditional text-processing utilities lack semantic awareness of URL structure and encoding rules:

• Full RFC Compliance: Relies directly on libcurl, ensuring that path encodings, percent-encoding escapes, and protocol parsing strictly match web standards.
• Surgical Component Modification: Enables you to change the scheme, replace the hostname, append query parameters, or increment port numbers without reassembling the full string manually.
• Query Parameter Sorting: Sorts query keys alphabetically to generate consistent, cacheable canonical URLs.
• JSON Output Support: Exports dissected URL components as structured JSON for easy piping into jq.

Step 1: Installing trurl on Linux

trurl is available in the official package repositories of many modern Linux distributions and can also be built easily from source.

Method 1: Installing via Native Package Managers

On recent versions of Debian, Ubuntu (24.04+), Fedora, or Arch Linux, install trurl directly:

Ubuntu and Debian:

sudo apt update
sudo apt install -y trurl

Fedora and Rocky Linux:

sudo dnf install -y trurl

Arch Linux:

sudo pacman -S trurl

Method 2: Building from Source (Any Linux Distribution)

If your distribution bundles an older repository that does not package trurl, you can build it in seconds using make and libcurl-dev:

# Install build dependencies
sudo apt install -y git build-essential libcurl4-openssl-dev

# Clone repository and compile
git clone https://github.com/curl/trurl.git
cd trurl
make
sudo make install

Verify that trurl is installed and functional:

trurl --version

Step 2: Extracting URL Components with trurl

The primary command for reading URL segments is --get (shortened to -g). You can target specific parts of any URI.

Extracting Host, Scheme, or Path

trurl --url "https://api.example.com:8443/v1/users?active=true#section" --get "{host}"

Output:

api.example.com

To extract the path and query string simultaneously:

trurl --url "https://api.example.com:8443/v1/users?active=true" --get "{path}?{query}"

Extracting Specific Query Parameters

To extract the value of an individual query parameter from a complex URL string without regex:

trurl --url "https://shop.example.co.uk/search?category=hardware&order=asc&page=3" --get "{query:category}"

Output:

hardware

Step 3: Modifying URLs in Real-Time

The --set option modifies individual URL components while keeping the remainder of the URL perfectly intact.

Changing Protocols and Hostnames

To upgrade an insecure HTTP link to HTTPS and redirect it to a staging host:

trurl --url "http://production.internal/dashboard" --set scheme=https --set host=staging.internal

Output:

https://staging.internal/dashboard

Appending or Replacing Query Parameters

Adding API keys or tracking tokens safely without worrying whether the URL already has a ? or & is trivial:

trurl --url "https://api.example.com/data?token=abc" --set "query:env=production"

Output:

https://api.example.com/data?token=abc&env=production

Trimming Unwanted Query Parameters (Stripping UTM Tracking)

To sanitize incoming web links by removing analytics tracking keys:

trurl --url "https://blog.example.com/post?id=42&utm_source=twitter&utm_medium=social" --trim "query:utm_source" --trim "query:utm_medium"

Output:

https://blog.example.com/post?id=42

Step 4: Sorting Query Strings for Canonical Caching

In caching proxies (such as Varnish or Nginx), requests with query parameters in different orders (e.g. ?a=1&b=2 vs ?b=2&a=1) can cause duplicate cache misses. trurl sorts parameters deterministically:

trurl --url "https://cdn.example.com/api?z=last&a=first&m=middle" --sort-query

Output:

https://cdn.example.com/api?a=first&m=middle&z=last

Step 5: Batch Processing URLs from a File or Pipe

trurl reads from standard input, making it ideal for processing server logs or web crawler outputs:

cat urls.txt | trurl --url-file - --get "{host}" | sort -u

This pipeline parses thousands of URLs per second, deduping hostnames cleanly without syntax errors.