How to Accelerate GZip Compression Using the pigz Command in Linux

The gzip command has been the absolute standard for Linux file compression for decades. However, the original gzip binary was written in an era of single-core processors; it physically cannot use more than one CPU core. If you are compressing a massive 100GB tarball on a modern server with 32 cores, gzip will max out one single core at 100% while the other 31 cores sit completely idle, resulting in a backup process that takes hours instead of minutes. To unlock the full power of modern hardware, you must use pigz (Parallel Implementation of GZip).

How the pigz Command Works

pigz is a drop-in, fully compatible replacement for gzip. It produces the exact same standard .gz files that can be extracted by any system in the world. The difference is the engine. pigz splits the massive input file into 128KB chunks and simultaneously feeds those chunks to every single available CPU core on your server, vastly accelerating the compression process.

To install it on Ubuntu/Debian, run: sudo apt install pigz

How to Compress Files Using Multiple Cores

The syntax for pigz is virtually identical to standard gzip.

To compress a massive log file, simply run:

pigz massive_log.txt

By default, pigz will automatically detect exactly how many logical CPU cores your server has and utilize all of them. Just like standard gzip, this command is destructive; it will compress the file, generate massive_log.txt.gz, and instantly delete the original, uncompressed file.

If you are running this command on a live production server and do not want to completely starve the machine of CPU resources (which could crash your website), you can throttle pigz. Use the -p (processes) flag to strictly limit exactly how many cores it is allowed to use.

For example, to force the compression to only use 4 cores, run:

pigz -p 4 massive_log.txt

How to Extract Using pigz

Because the output is a standard GZip file, you could use the normal gunzip command to extract it. However, you can also use pigz to accelerate the decompression process.

To extract the file using multiple cores, pass the -d (decompress) flag:

pigz -d massive_log.txt.gz

This will rapidly unpack the archive and restore the original massive_log.txt file.