You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
What’s the significance of Robots.txt file in a website?
- Thread starter EmilyPete
- Start date
sadianisar
New member
Robots.txt is a file in which we write commands for crawler to not visit and index the webpage which we don not want to. The file consists of two codes named ueser-agent and disallow. It is important to submit in the website for SEO.