How to Use the split Command in Linux Terminal to Divide Large Files by Size or Lines

In Linux system administration and data engineering, working with massive files is an everyday reality. Multi-gigabyte database dumps, multi-terabyte web server access logs, virtual machine images, and raw sensor datasets frequently exceed the maximum file size limits imposed by cloud storage providers, FAT32 external USB drives, email attachments, or network transfer protocols. Attempting to open, parse, or transfer these monolithic files as single entities can exhaust server memory, trigger timeout errors, or crash text editors.

To overcome file transfer thresholds and simplify distributed data processing, Linux provides the native split utility within GNU Coreutils. The split command breaks large files into smaller, manageable chunks based on exact byte sizes, megabyte thresholds, or line counts without modifying or corrupting the underlying data. Furthermore, split chunks can be reassembled into an exact replica of the original file using the native cat command. This comprehensive guide covers splitting files by size, dividing logs by line limits, customising chunk prefixes, and recombining split archives.

Understanding How split Names and Segments Files

By default, the split command processes files sequentially and appends alphabetical suffixes to generated pieces:

• Default Suffixes: Without custom arguments, split generates files named xaa, xab, xac, and so on. The prefix x is default, and the two-letter alphabetical suffix increments predictably.
• Numeric Suffixes: Passing the -d flag replaces letters with numeric counters (e.g., 00, 01, 02), which is standard practice when organising chunks for sequential processing pipelines.
• Data Preservation: Splitting does not alter the source file. It reads data streams and writes discrete byte chunks to newly created files in your working directory.

Step 1: Splitting Files by Byte Size (Megabytes or Gigabytes)

When partitioning a large archive, disk image, or database backup for upload or external drive storage, use the -b (bytes) flag with size multipliers:

split -b 500M large_database_dump.sql

This command divides large_database_dump.sql into chunks of exactly 500 megabytes each. Supported size units include:

• K: Kilobytes (e.g., -b 100K)
• M: Megabytes (e.g., -b 500M)
• G: Gigabytes (e.g., -b 2G)

Step 2: Splitting Text Files and Logs by Line Count

When parsing enormous Apache, Nginx, or syslog files containing millions of log records, splitting by byte size can slice individual log entries in half. To ensure each generated chunk contains complete, unbroken lines of text, split by line count using the -l flag:

split -l 50000 access.log

This command creates smaller log files containing exactly 50,000 lines each, allowing you to open and inspect each piece easily in nano, vim, or less without causing memory spikes.

Step 3: Customising Output Prefixes and Numeric Suffixes

To keep your directory organised and easily identifiable, customize both the output prefix and suffix format:

split -b 100M -d --additional-suffix=.part backup.tar.gz backup_chunk_

Breaking down these parameters reveals complete formatting control:

• -b 100M: Sets each chunk size to 100 megabytes.
• -d: Employs numeric suffixes (00, 01, 02) instead of alphabetical characters.
• --additional-suffix=.part: Appends a recognizable extension to every piece.
• backup_chunk_: The custom prefix string positioned at the end of the command.

The resulting files will be named cleanly: backup_chunk_00.part, backup_chunk_01.part, and so on.

Step 4: Recombining Split Files into an Exact Replica

Once your split files are transferred across the network or moved to their destination system, you can concatenate the pieces back into the original monolithic file using standard shell concatenation:

cat backup_chunk_* > restored_backup.tar.gz

Because the shell sorts numerical chunks in alphanumeric order, cat reads each piece sequentially and reconstructs the byte stream seamlessly.

Step 5: Verifying Integrity Between Original and Recombined Files

To verify that no byte corruption or missing chunks occurred during the splitting and reassembly cycle, calculate and compare cryptographic checksums:

1. Generate the SHA-256 hash of the original file:

   sha256sum backup.tar.gz

2. Generate the SHA-256 hash of the restored file:

   sha256sum restored_backup.tar.gz

If both 64-character hashes match identically, your recombined file is mathematically identical to the original source.

Using the native split utility in Linux empowers administrators to overcome storage constraints, accelerate parallel processing, and transfer large datasets safely and reliably.