Adding Custom Robots.txt to Blogger Blog


Before starting, let us see the importance of adding custom robots.txt to your blog or your website. If you add robots.txt to your site, the Search Engine Crawlers or Bots or spiders will get a clear idea about the structure of your site during their crawl. This file makes the indexing easier for the robots. With this file, you can include or exclude certain URLs, which helps you to increase your SEO.

What is robots.txt?

A robots.txt file is a file at the root of your site that indicates those parts of your site you don’t want accessed by search engine crawlers. The file uses the Robots Exclusion Standard, which is a protocol with a small set of commands that can be used to indicate access to your site by section and by specific kinds of web crawlers (such as mobile crawlers vs desktop crawlers). Read more about robots.txt by Google here

How to add it to your blogger blog?

Login to your blogger blog with your Google account. Go into the settings of your blog from the dashboard as shown. In that go into Search preferences. You will see Custom robots.txt. Click edit.

Edit the Custom Robots from settings

In that Custom Robots, click yes on enable custom robots, add the following text and click save as shown below. Change the highlighted text with your domain name. If you are running your blog on a custom domain, use that domain name or else the free blogger sub-domain.

User-agent: Mediapartners-Google

User-agent: *
Disallow: /search
Disallow: /?m=1
Disallow: /?m=0
Disallow: /*?m=1
Disallow: /*?m=0
Allow: /



Robots.txt is now enabled for your blog.

In the robots.txt file, Allow will allow the contents to get indexed by the search engine and Disallow will block the contents from being indexed and the sitemap is the xml file or the feed that gives the clear idea about your blog contents to the search engine.  Make sure that you are entering the robots.txt correctly because the wrong use of the robots.txt may leave your site from the eyes of the search engine.

You have now added a custom robots.txt to your blogger blog. Check out the live updated custom robots by entering your site’s domain name followed by /robots.txt like The entered robots.txt will show up there. Check it with Google webmaster tool as well.

Other People are reading
Hope You liked this Tutorial. For any clarifications and suggestions use the comment box below. Subscribe to our Newsletter for getting the future updates directly in your inbox.
Free Email Updates !
Join the visitors who are receiving our newsletter and receive the Daily Mass Readings, Prayers and other updates directly in your inbox.
We respect your privacy and take protecting it seriously.