Use of Robot.txt

[zeoafer]

How to use Robot.txt file. What is proper code for file. I want to disallow a page of my website.

 

[jaysh4922]

Robot.txt is an on-page SEO technique and it is basically used to allow for the web robots also known as the web wanderers, crawlers or spiders. It is a program that traverses the website automatically and this helps the popular search engine like Google to index the website and its content.

 

[AvniShergill]

Robots.txt file tells the spiders or crawlers, which page of your website should be crawled and which should be avoided.

 

[koyal]

User-agent: *
Disallow:
Sitemap: http://www.raaas.com/sitemap.xml