The Headless Server Problem
You have just rented a brand new Linux server in the cloud. You connect to it via SSH, and you are staring at a blinking cursor on a black screen. You need to install a specific software package, like the latest version of WordPress or a custom Python script hosted on GitHub.
If you were using your home computer, you would open Google Chrome, navigate to the website, click the “Download” button, and wait for the file to appear in your Downloads folder. But this cloud server is “headless.” It does not have a monitor, it does not have a mouse, and it absolutely does not have a web browser installed. How do you pull a file from the internet directly onto a machine that only understands text?
You use the wget command. Short for “World Wide Web Get,” wget is a non-interactive command-line utility designed specifically to download files from web servers (HTTP, HTTPS, and FTP) directly to your local Linux directory.
The Basic Download
Using wget is incredibly simple if you know the exact URL of the file you want to download.
For example, if you want to download the latest compressed version of WordPress (which is always located at https://wordpress.org/latest.tar.gz), you simply type wget followed by a space, and then paste the URL:
wget https://wordpress.org/latest.tar.gzWhen you press Enter, wget will reach out to the WordPress servers, negotiate the connection, and display a progress bar on your screen consisting of text characters. When it reaches 100%, the file latest.tar.gz will be sitting perfectly in whatever directory you are currently in.
Downloading and Renaming Simultaneously
Sometimes the URL of a file is messy. If you download a file from a dynamic link like https://example.com/download.php?id=8472, wget will literally name the saved file download.php?id=8472 on your server, which is annoying to deal with.
You can force wget to rename the file as it downloads by using the -O (capital letter O for Output document) flag.
wget -O server_script.zip https://example.com/download.php?id=8472This tells Linux: “Go to that URL, download whatever is there, but save it to my hard drive under the clean name server_script.zip.”
Downloading in the Background
If you are downloading a massive 50-Gigabyte dataset, it might take three hours. If you use the basic wget command, your terminal is frozen for those three hours. If your SSH connection drops, the download fails.
To solve this, use the -b (Background) flag.
wget -b https://example.com/massive_dataset.isoWhen you hit Enter, wget will immediately detach from your terminal and continue downloading the file silently in the background. It will return your cursor immediately so you can continue working on other server tasks. It creates a log file called wget-log in the same directory, which you can check occasionally to see the download’s progress.
Conclusion
Stop trying to download files to your laptop and transfer them to your server via FTP. By mastering the Linux wget command, you can pull software packages, scripts, and media files directly from the web onto any headless server with a single line of text.