The wget command is a free utility used to download files from the internet. It is especially helpful when you are working with a headless Linux system that does not provide a GUI. The GNU wget command allows you to download multiple files or a single file using various protocols such as HTTP, HTTPS, and FTP protocols.
In this tutorial, we will learn about wget command in Linux and explain its options with examples.
Table of Contents
How to install wget command in Linux
Nowadays, The GNU wget command comes preinstalled with modern Linux distributions, and therefore you won't need to install it. To confirm that wget is installed, run the command:
wget --version
If for whatever reason, wget is not installed, you can install it using package managers in various Linux distributions as follows.
Install wget on Debian / Ubuntu
On Debian and Ubuntu distributions, run the command:
sudo apt install wgetInstall wget on Fedora / Rocky Linux / AlmaLinux
For Fedora and RHEL-based distributions such as Rocky Linux and AlmaLinux, run the command:
sudo dnf install wgetFor RHEL & CentOS Stream you can use the same DNF package manager.
Install wget on Arch Linux
For Arch Linux and other Arch-based distributions such as Manjaro, run:
sudo pacman -Sy wgetInstall wget on SUSE Linux
On SUSE Linux, run the command:
sudo zypper install wgetHow to run wget Command in Linux
The basic syntax of wget command:
wget OPTIONS URLTo run wget, simply type wget following option and URL from the terminal.
Here are some of the most commonly used wget command options.
| Options | Description |
|---|---|
-m, --mirror | Downloads a mirror copy of a website including all website files |
-O (Uppercase) | Downloads a file using a different file name |
-b, --background | Downloads a file in the background and frees up the terminal |
-i, --input-file | Downloads files specified in URLs in a local or external file |
-c ( Lowercase ) | Resumes download of a partially downloaded file |
-P | Specifies the directory that a file will be downloaded |
--limit-rate | Limits download speed when downloading a file |
--ftp-user=USER | Set FTP user to USER |
--ftp-password=PASS | Set FTP password to PASS |
--no-check-certificate | Skips SSL certificate checking |
--user-agent | Changes browser UserAgent |
-V, --version | Displays the version of wget |
-h, --help | Displays all the wget command options and usage |
Wget Command Examples
With examples, let us explore various ways that you can use the wget command on the command line.
Downloading a webpage
You can download a mirror copy of a website using the -m option. This downloads all website files including HTML, CSS, and Javascript files as well as internal links.
wget -m https://example.com
You can pass a few arguments to use the downloaded site for local browsing. The -k option renders all the links in the downloaded HTML files to local files for offline viewing. The -p instructs wget to download all the website files including HTML, CSS, JavaScript, and image files.
wget -m -k -p https://example.com
Downloading a file using wget
In its most basic form, without any command options, the GNU wget command downloads a resource or file from the specified URL to the current working directory.
In the example below, we are downloading the phpmyadmin-5.1.1-all-languages zip file from the specified URL.
wget https://files.phpmyadmin.net/phpMyAdmin/5.1.1/phpMyAdmin-5.1.1-all-languages.zip
During the download process, the wget command resolves the specified domain name and connects to the remote server. Once it has established a connection with the remote server, wget sends an HTTP request to the web server. This is evidenced by the 'HTTP request sent, awaiting response' output line.
On receiving the request, the server responds and the file download commences. The file is finally downloaded to your current directory.
To view the downloaded file, use the ls command as follows.
ls -l | grep -i phpmyadmin
Downloading a file using a different name
Without any command options, the file name of the downloaded appears exactly as it is on the remote server. Often, downloaded files bear complex names with many characters. To download and save the file under a different name, use the -O option.
In this example, we have downloaded the same file and saved it as phpmyadmin.zip.
wget -O phpmyadmin.zip https://files.phpmyadmin.net/phpMyAdmin/5.1.1/phpMyAdmin-5.1.1-all-languages.zip
Downloading a file to a different directory
So far, we have seen that the wget command downloads files to the current directory structure. Suppose you want to download a file to a different directory. To accomplish this, set the directory prefix using the -P option.
In the following command, we are downloading the Go binary file named go1.17.6.linux-amd64.tar.gz to the /tmp directory instead of the current directory.
wget -P /tmp/ https://go.dev/dl/go1.17.6.linux-amd64.tar.gz
Limiting the download speed
By default, the wget command uses up all the bandwidth to download files. To limit the download speed use the --limit-rate option followed by the value of the download speed. The speed is expressed in bytes/seconds. To specify the download speed, use k for kilobytes, m for megabytes, and finally g for gigabytes.
In the example below, we have managed to limit the download speed of the Go binary file to 1 Megabyte per second.
wget --limit-rate=1m https://go.dev/dl/go1.17.6.linux-amd64.tar.gz
Resuming a failed file download
Occasionally, you will run into situations where a file download fails. This can be due to multiple causes such as unstable network connections or a network error. When an interruption happens while file download is in progress, it results in a partially downloaded file. The incomplete download file is usually considered a broken file.
Thankfully, the wget command provides the -C option which resumes a file download.
In this example, we have simulated a failed file download by pressing Ctrl + c.
To resume a partially downloaded file, we will use the -C option as follows.
wget -c https://download.owncloud.org/community/owncloud-complete-latest.tar.bz2
If the remote HTTP server does not support the resumption of a file download, then wget will start the download afresh and overwrite the existing file.
Downloading a file in the background
When time is of the essence and you want to continue running other tasks on the terminal, you might want to consider downloading a file in the background. This frees up the terminal and allows you to continue running other tasks.
To download files in the background, use the -b option. In the following command, we are downloading the latest Linux kernel in the background.
wget -b https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.16.1.tar.xzIf no output file is specified using the -O option, then the output is redirected to the wget-log file in your present directory. You can keep track on the download progress by viewing the wget-log file using the tail command:
tail -f wget-log
Modifying wget User-Agent
Each time you visit a website from a web browser, the browser's user agent - which is a string of text - relays information about the browser and Operating system to the webserver. The User-Agent is encapsulated in the HTTP header and each browser's User-Agent is unique. In essence, the User-Agent announces the browser's type and version to the server.
Sometimes, web servers may block User-Agents of some web browsers. In such cases, you can emulate a different browser's User-Agent by using the --user-agent option.
The following command emulates Firefox 96 when accessing the site: http://example.com:
wget --user-agent="Mozilla/5.0 (X11; Linux i686; rv:96.0) Gecko/20100101 Firefox/96.0" http://example.com
Downloading multiple files
Sometimes, you might want to download multiple files at a go to save on time. A simple way to achieve this is by saving a list of URLs in a simple text file. Then use the -i option followed by the path to the input file that contains the list of URLs to be downloaded.
To elaborate on this, we have already saved two URLs in a text file named download.txt.To view the txt file, we will use the cat command:
cat download.txtTo download all the files, we will execute the following command with the -i option followed by the local or external file.
wget -i download.txt
Downloading files using FTP
Wget command can also be used to download files from FTP servers. When downloading files from a password-protected FTP server, use:
wget --ftp-user=FTP_USERNAME --ftp-password=FTP_PASSWORD ftp://ftp.example.com/filename.tar.gzSkip certificate checks
In rare cases, you will find yourself downloading files from HTTPS sites with invalid SSL certificates. On the command line, this can be an issue and you will run into To bypass certificate checking and download the file, use the --no-check-certificate parameter.
wget --no-check-certificate https://site-with-invalid-ssl-certificate.comConclusion
Wget command is a command-line utility used in many Unix-like operating systems to download files from the internet. It supports HTTP, HTTPS, and FTP and comes in handy when downloading files on the command line especially when one is connected to a headless server that does not provide a GUI.
In this tutorial, we have covered quite a number of wget command examples.
Comments