It is used to prohibit indexing of pages

[sumaiyakhatun27]

Files or directories. Pages that should be blocked include: Service files and folders. Admin panel, CMS files, user personal account, basket, etc. Uninformative auxiliary pages that do not need promotion. For example, biographies of blog authors. Various types of duplicates of the main pages. Let's look at duplicates in more detail. Imagine that you have a blog page with an article.


You advertised this article on another resource by adding a UTM tag to the hong kong email list existing URL to track conversions. The address has changed slightly, but it still leads to the same page - the content is completely identical. This is a duplicate that needs to be closed from indexing. It is not only statistics systems that are to blame for page duplication. Duplicates can appear when searching for products, sorting, due to the presence of the same product in several categories, etc.


Even the site engines themselves often create a large number of different duplicates (especially Wordpress and Joomla). We make websites that are optimized for search engines and bring in sales. More details In addition to complete duplicates, there are also partial ones. The best example is the main page of a blog with post announcements. As a rule, announcements are taken from articles, so such pages lack unique content.