How to discourage search engines from indexing WordPress sites

Site owners will do anything to get their websites indexed. However, you may not want search engines to crawl your website if it’s still under development. In a case like this, it is recommended to discourage search engines from indexing your site. Keep reading if you want to know more about this topic.

Why discourage search engines from indexing your site?

There are some cases where people want to discourage search engines from indexing their sites:

• unfinished websites: In this stage of errors and tests, it is better not to have your website available to the public.
• restricted websites: If you plan to have an invite-only website, you don’t want it to show up in the SERPs.
• test accounts: Website owners create a duplicate of the site for testing and testing. Since these sites are not designed for the public, they do not allow them to be indexed by search engines.

So how do you block search engines from indexing your site? Well, take a look at the following options and try it yourself.

1. Discourage search engines from indexing WordPress sites

The easiest way to prevent search engines from indexing our site is to avoid your . To do this, you need to edit the robots.txt file in your website directory. Here are some ways to do it.

Use the option that includes WordPress

Editing the WordPress robots.txt file is quite easy, as you just have to use a function built into WordPress. We explain the procedure below:

1. Log in to your WordPress dashboard and go to Settings -> Reading.
2. Scroll to the option Search engine visibility.
3. Check the option that says Discourage search engines from indexing this site.
4. save the changes and ready. WordPress will automatically edit the file robots.txt by you.

Edit the robots.txt file manually

If you prefer the manual option, you can use the file manager or a to edit the file robots.txt.

In this section, we will show you how to do it through the hPanel File Manager:

1. Log in to hPanel and locate the file manager in the area of Records.
2. Go to the root directory folder of your WordPress (in most cases, this is public_html) and find the file robots.txt. If you can’t find it, create a new blank file.
3. Right-click on the file and select Edit.
4. Enter the following User-agent syntax: * Disallow: /

The above code will prevent search engines from indexing your entire site. If you want to apply the disallow rule to a specific page, write the subdirectory of the page and the slug. For example: Disallow /blog/food-review-2019.

The file syntax robots.txt they are case sensitive, so be careful when editing them.

2. Password protect your WordPress site

Search engines do not have access to websites that have password-protected files. Here are some methods to password protect your WordPress site:

Using the Control Panel

If you are a customer, you can protect your website with a password using the password protection tool. Password protect directories from hPanel:

1. Login to hPanel and navigate to Password protect directories.
2. Enter your root directory in the first field.
3. Once the directory is selected, enter your username and password and click To protect.

If your root directory is public_htmlleave the directory field blank.

The process is quite similar in cPanel:

1. Login to cPanel and go to the option DirectoryPrivacy (Directory privacy).
2. Select the root directory, usually public_html.
3. Select the option Password protect this directory (Password protect this directory) and enter the name of the directory you want to protect. press Save (Save).
4. Create a new user to access the protected website and you’re done.

Use WordPress plugins

There are many plugins that can help you password protect your site, but this plugin might be the best out there. It has been tested with the new WordPress update and is quite easy to use.

After , go to Settings -> Password Protected and configure the options according to your needs.

3. Remove a page indexed by Google

If your page has already been indexed by Google you can still remove it by following these steps:

1. Make an account for your site at .
2. Access the Google Search Console account of your newly added domain and select the option Index -> URL removal in the menu on the left.
3. Click the Temporary Withdrawals checkbox and press New request.
4. In the new window, choose the option Temporarily remove URLenter the URL and click Following.

Please note that it may take up to a day for Google to process your request.

conclusion

There you have it! Quick and easy ways to discourage search engines from indexing your sites. Here’s a quick rundown of the methods we’ve learned today:

• Edit the file robots.txtwhich can be done automatically or manually.
• Password protect your website using a plugin or your hosting control panel.
• Remove indexed pages from Google through .

If you have any other method, or any questions, let us know in the comments. Good luck.

Gustavo is passionate about creating websites. He focuses on the application of SEO strategies at for Spain and Latin America, as well as the creation of high-level content. When he is not applying new WordPress tricks you can find him playing the guitar, traveling or taking an online course.