When managing corporate infrastructure, auditing cloud attack surfaces, or scanning networks, administrators and security professionals frequently face lists of thousands of discovered subdomains and IP addresses. Determining which endpoints host active web servers, identifying HTTP status codes, extracting web page titles, and fingerprinting underlying technology stacks using standard tools like curl is slow and inefficient. httpx, developed by ProjectDiscovery, is a multi-threaded, high-throughput HTTP toolkit built specifically to solve this problem.
Written in Go, httpx executes thousands of concurrent HTTP/HTTPS probes with automatic protocol fallback, intelligent retries, title extraction, TLS certificate fingerprinting, and clean JSON/piped output formatting.
Core Capabilities of httpx
httpx provides distinct advantages over traditional command-line HTTP clients:
- Blazing Concurrency: Leverages Go’s
retryablehttplibrary and asynchronous worker pools to probe thousands of targets per minute. - Automatic Protocol Handling: Tests HTTPS on port 443 first, automatically falling back to HTTP on port 80 if secure handshakes fail.
- Rich Metadata Extraction: Extracts page titles, server headers, Content-Type, Content-Length, response times, favicon hashes, and TLS cipher information in a single pass.
- Pipeline Compatibility: Integrates seamlessly with tools like
subfinder,nmap, andnucleithrough standard Unix stdin/stdout pipes.
Installing httpx on Linux
You can install httpx via pre-compiled binaries or through the Go compiler:
# Method 1: Official Pre-compiled Binary (Recommended)
curl -s https://api.github.com/repos/projectdiscovery/httpx/releases/latest | grep "browser_download_url.*linux_amd64.tar.gz" | cut -d : -f 2,3 | tr -d \" | wget -qi -
tar -xvf httpx_*_linux_amd64.tar.gz
sudo mv httpx /usr/local/bin/
httpx -version
# Method 2: Via Go (Requires Go 1.21+)
go install -v github.com/projectdiscovery/httpx/cmd/httpx@latestCommon Command Flags and Options
The table below highlights essential command-line arguments for customising scans:
| Flag | Functionality | Usage Example |
|---|---|---|
-u <target> |
Target single URL or host | httpx -u https://example.com |
-l <file> |
Read target hosts from text file | httpx -l hosts.txt |
-status-code, -sc |
Display HTTP response status codes | httpx -l targets.txt -sc |
-title |
Extract and print HTML page titles | httpx -l targets.txt -title |
-tech-detect, -td |
Fingerprint technology stacks (e.g., Nginx, React) | httpx -l targets.txt -td |
-threads, -t |
Number of concurrent worker threads | httpx -t 100 (default 50) |
-json |
Output complete response data formatted as JSON | httpx -l targets.txt -json -o out.json |
Running Practical Probing Workflows
1. Probing a Single Domain with Metadata
To inspect an endpoint and return status codes, web server banners, and HTML page titles in one concise line:
httpx -u https://example.com -status-code -title -web-serverOutput format:
https://example.com [200] [Example Domain] [cloudflare]2. Chaining Subfinder with httpx in Unix Pipelines
Discover subdomains passively and immediately probe which hosts are serving live web pages:
subfinder -d example.com -silent | httpx -silent -status-code -title -tdThis workflow filters thousands of inactive domain records down to verified web services along with their running technologies.
3. Scanning Custom Web Ports and Exporting to JSON
Probe non-standard HTTP ports (such as 8080, 8443, 8000, 9000) and export structured JSON objects for automated ingestion:
httpx -l hosts.txt -ports 80,443,8000,8080,8443 -json -o active_services.jsonFrequently Asked Questions
How does httpx handle HTTP redirects?
By default, httpx displays the initial response code (e.g., 301 or 302). To follow redirects automatically and inspect the final landing URL, add the -follow-redirects (-fr) flag.
Does httpx support rate limiting to avoid overwhelming target servers?
Yes. You can limit request throughput by supplying the -rate-limit (-rl) flag (e.g., -rl 150 restricts outbound scans to 150 requests per second).