What are the different method to control the robot?

Robots can be controlled either by a Sitemap or a Robots.txt file. Sitemap helps the robots to crawl webpages, while Robots.txt file lists webpages that you want to restrict form robots to crawl.
 
Back
Top