In this article we are going to discuss about How to Add Custom Robots.txt File in Blogger blog. By default blogger blog provide robots.txt file which you can edit by going to settings in blogger dashboard. You will enable and add your custom text to the blogger. Before discussing about the method to How to Add Custom Robots.txt File in Blogger let’s know about the robots.txt file and it’s important for any website or blog.
What is robots.txt file?
Robots.txt is a text file with extension .txt. To which we create and upload to our website. We use to prevent the robots of certain search engines from tracking the content that we do not want to index or show in their results.
That is, it is a public file that we use to indicate to those crawlers or spiders which part or parts should not enter to track and index our website. In it, we can specify in a simple way, the directories, subdirectories, URLs or files of our website that should not be tracked or indexed by the search engines.
Because it is intimately related to the indexing of the website, it is essential to properly program this file. Especially if our website is made with a content manager (CMS), that generates it automatically. Since it can happen that they are accidentally included as not Indexable parts that should be tracked
It is also called robot exclusion protocol or robots.txt protocol. it is advisory and does not guarantee full secrecy. But sometimes we find it used to keep private parts of a website. Because that isolation is not full, its use is discouraged to keep certain areas private. As it serves as a recommendation and not as an obligation, being a hacker’s treat that with a browser and the necessary knowledge, can easily access them.
Normally the most common uses are to avoid accessibility to certain parts of the website. Prevent the indexing of duplicate content (for example printable versions of the web). or to tell Google what our sitemap is, including its URL in the file .
How do we create the robots.txt file?
In order to create it, we need access to the root of the domain and upload the file in text format with name “robots.txt”, to the root directory of the first level of the server of our website. But in case if you have hosted your blog or website on blogger, you never need it. In the blogger you can easily enable and edit the robots.txt file.
To Add Custom Robots.txt File in Blogger follow the following procedure.
Add Custom Robots.txt File in Blogger
Follow these steps to add custom robots.txt file in blogger.
Login to your blogger Dashboard and open settings.
In settings expand the options and find the option to custom robot.txt. It may be at the end in Search Preferences page.
Click on Edit and then click Yes option button to enable custom robot.txt file in blogger.
It will display a text box where you can add your custom coding for robot.txt file.
Paste above coding in text box and save the settings.
Note: don’t forget to replace sitemap link in code with your own site’s sitemap.
Learn how to create Sitemap for blogger.
Now all the unnecessary pages would be restricted form indexing. Only useful pages will be indexed in search engines.
That’s all you done.