The metatags are html tags that incorporate information (metadata) on a web page. We could consider them the “ancestors” of what is now the , also called web 3.0. They are incorporated in the header (within tags) of the web pages, to provide very basic data (author, title, date, keywords, description) to browsers, applications and search engines.
One of the metatags that has been gaining more importance in recent years is the meta tag robots. Its usefulness is to inform search engines (Google, Bing, Yahoo…) on which pages they can index, and with what limitations. Its syntax is the following:
The “attributes” that can be included, are the following:
- index: Allows the search engine to index that page (default option).
- noindex: DOES NOT allow search engines to index that page.
- follow: Tells the search engine that it can continue crawling the links it finds on the page (default option).
- nofollow: Tells the search engine to NOT continue crawling the links it finds on the page
- none: It is an abbreviation of noindex + nofollow.
- noarchive: Do not allow search engines to cache the page
- nosnippet: It does not allow search engines to display a piece of text from the web in their search results, and it does not allow it to cache it either.
- noodp: Does not allow searchers to use the official description of the page on (wikipedia) in their search results. In case it is not registered in (official website), it is of no use.
- noydir: Does not allow Yahoo! that you use the official description of the page that is on it in your search results. If it is not registered in that directory, it is of no use.
As for the “robots” (sometimes they are called crawlers, agents, bots, robots or spiders) there are hundreds of them. Here you can consult an exhaustive . Although the most important and used are the following:
- Google: GOOGLEBOT
- yahoo: SLURP
- bing: MSNBOT
- Ask: THEOMA
Not all robots accept all attributes, although if the most important. Namely:
- : Accepts index, noindex, none, follow, nofollow, noarchive, nosnippet, and noodp.
- : Accepts index, noindex, nofollow, noarchive, noodp, and noydir.
- : Accept index, noindex, nofollow, noarchive, and noodp.
Facebook Comments
Disqus