What is and how to use the wget command + 12 examples

This guide will show you how to use the command wget Linux. We will also provide you with 12 useful examples of this command. By the end, you will know all about the wget command and can use it to download files from the Internet.

What is the Wget Command?

wget is a computer tool created by the . You can use it to retrieve content and files from various web servers. The name is a combination of world Wide Web and the word get. Supports downloads via FTP, SFTP, HTTP, and HTTPS.

Wget is built in portable C and can be used on any Unix system. It is also possible to implement it on Mac OS X, Microsoft Windows, AmigaOS and other popular platforms.

How to install the Wget command?

For this command demo, we will be using Ubuntu 16.04. But the syntax will also work on any other version of Linux.

First of all, access your server through SSH. Run the following command to install on your system:

apt-get install wget

Once the setup is done, you are ready to use it. Also, knowing basic SSH commands can make things easier.

Wget Command Examples

To get you started, we give you 12 examples that you can use in everyday tasks. Note that you can also call this function from scripts and cron jobs!

Using the Wget command to download individual files

One of the most basic examples of this command is to download a single file and store it in your current working directory. For example, you can get the latest version of WordPress using the following:

wget https://wordpress.org/latest.zip

Here is the result you will see:

–2018-02-23 12:53:10– https://wordpress.org/latest.zip Resolving wordpress.org (wordpress.org)… 198.143.164.252 Connecting to wordpress.org (wordpress.org) |198.143.164.252|:443… connected. HTTP request sent, awaiting response… 200 OK Length: 9332728 (8.9M) Saving to: ‘latest.zip’ latest.zip 100% 8.90M 6.88MB/s in 1.3s 2018-02-23 12 :53:14 (6.88 MB/s) – ‘latest.zip’ saved

In this example, a file named latest.zip in the current working directory. You will also see additional information such as download progress, speed, size, time and date.

Using the Wget command to download multiple files
We can take the use of wget one step further and download multiple files at once. To do this, we’ll need to create a text document and place the download URLs there. In this example, we will retrieve the latest versions of , and . Enter the following:

nano example.txt

This will create an example.txt file and open a text editor interface. Paste these links there:

https://wordpress.org/latest.zip https://downloads.joomla.org/cms/joomla3/3-8-5/Joomla_3-8-5-Stable-Full_Package.zip https://ftp.drupal. org/files/projects/drupal-8.4.5.zip

Once done, you can use -Yo to get all the files stored in your example text file:

wget -i example.txt

Wait for the process to finish and you will have the installations of the three most popular content management systems.

Use the Wget command to get files under different names

In this wget example, we will save a file using a different name with the help of the option -EITHER:

wget -O wordpress-install.zip https://wordpress.org/latest.zip

In this case, the downloaded resource will be saved as wordpress-install.zip instead of its original name.

Use the Wget command to save files to the specified directory

You can use wget to put a file in another directory using the function -P:

wget -P documents/archives/ https://wordpress.org/latest.zip

The file you retrieve using this syntax will appear in documents/files/ binder.

Use the Wget command to limit download speed

With wget, you can also limit the download speed. This is useful when retrieving large files and will prevent it from using all your bandwidth. This wget example will set the limit to 500k:

wget –limit-rate=500k https://wordpress.org/latest.zip

Using the Wget Command to Set Retry Amount

Internet connection problems can cause the download to be interrupted. To address this issue, we can increase the number of retries using the function -tries:

wget -tries=100 https://wordpress.org/latest.zip

Use the Wget command for background downloads

For extremely large files, you can take advantage of the -b. It will download your content in the background.

wget -b http://example.com/beefy-file.tar.gz

A wget-log will appear in your working directory, which can be used to check the progress and status of your download. This command will also do the trick:

tail -f wget-log

Using the Wget command to download via FTP

The command can also be used with FTP. You will only need to specify the username and password as in this example:

wget –ftp-user=YOUR_USERNAME –ftp-password=YOUR_PASSWORD ftp://example.com/something.tar

Using the Wget command to continue interrupted downloads

Your download may be interrupted if you lose your Internet connection or experience a power outage. This is quite common when getting huge files. Instead of starting over, it is possible to continue the download using the function -c:

wget -c https://example/very-big-file.zip

If you continue without the function -cthe new file will have .1 added to the end as it already exists.

Using the Wget command to retrieve entire websites

It is also possible to use the wget command to download the content of an entire site. This will allow you to view it locally without an internet connection. Here is an example:

wget –mirror –convert-links –page-requisites –no-parent -P documents/websites/ https://some-website.com

Let’s break down the ingredients of this wget command:

-mirror Makes the download recurring.
-convert-links All links will be converted for offline use.
-page-requisites The following will include all the necessary files such as CSS, JS and images.
-non-parent It ensures that directories above the hierarchy are not retrieved.
-P This ensures that all content goes to our specified directory.

Once the process is finished, you will be able to open the downloaded website locally and find all the files in documents/websites/ binder.

Using the Wget command to locate broken links

Let’s try something more advanced. We can use the command to locate all the broken URLs that are showing the 404 error on a specific website. Start by running the following:

wget -o wget-log -r -l 5 –spider http://example.com

-either Collects the output to a file for later use.
-l Specifies the recurrence level.
-r Makes the download recurring.
-spider Set wget to spider mode.

Now we can investigate the file wget-log to find the list of broken links. Here is the command to do it:

grep -B 2 ‘404’ wget-log | grep “http” | cut -d ” ” -f 4 | sort -u

Using the Wget command to download numbered files

If you have numbered files or images in a certain list, you can easily download them with the following syntax:

wget http://example.com/images/{1..50}.jpg

conclusion

Congratulations! By completing this tutorial, you have learned several uses for the wget command. Now you can use it to get one or multiple files. In addition, you have learned some advanced uses, such as downloading an entire website or locating broken URLs. For more information, you can also refer to the .

Are there any hidden tips or tricks you want to share? Feel free to do so in the comments below!

Gustavo is passionate about creating websites. He focuses on the application of SEO strategies at for Spain and Latin America, as well as the creation of high-level content. When he is not applying new WordPress tricks you can find him playing the guitar, traveling or taking an online course.