nancyisabell
New member
Hii friends,
What is robots.txt? What is its use?
What is robots.txt? What is its use?
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
What is robots.txt?
- Thread starter nancyisabell
- Start date
Hello .
A robots.txt file is placed at the root of your domain to tell search engines and other web robots or spiders how to interact with your site. The robots.txt file allows you to specify which sections or pages of your site you want search engines to access, and which sections you want to restrict access to....
A robots.txt file is placed at the root of your domain to tell search engines and other web robots or spiders how to interact with your site. The robots.txt file allows you to specify which sections or pages of your site you want search engines to access, and which sections you want to restrict access to....
Prakash Sharma
Banned
Robots.txt file does not improve your search engine positioning. It provides robots with information concerning which files you will not allow to be crawled and indexed in the search engines. When the search engine robot crawls your site it looks for the robots.txt file...
Managespaces
New member
Hii friends,
What is robots.txt? What is its use?
When a search engine crawler comes to your site, it will look for a special file on your site. That file is called robots.txt and it tells the search engine spider, which Web pages of your site should be indexed and which Web pages should be ignored.
The robots.txt file is a simple text file (no HTML), that must be placed in your root directory.
AvniShergill
New member
Robots.txt files tells search engine robots, which pages to crawl and which should be avoided.
mindstreak
New member
Robots.txt file is used to indicate if the website can be indexed by various search engine crawlers.It also helps to disallow the specific number of webpage which shouldn't be indexed by the search engines.This is definitely necessary for a website.
adriennerich
New member
A robots.txt file is placed at the root of your domain to tell search engines and other web robots or spiders how to interact with your site...
Sarahbriggs
New member
Robot.txt is a text file which you put in your website to tell search engine which pages you would like not to crawl.
I am gonna give you some idea how to create Robot.tex. Please have a look below:
User Agent: *
Disallow: /Admin/
Disallow: /ce/
Disallow: /dp/
Disallow: /cid-pi-ap/
I am gonna give you some idea how to create Robot.tex. Please have a look below:
User Agent: *
Disallow: /Admin/
Disallow: /ce/
Disallow: /dp/
Disallow: /cid-pi-ap/
arsalan1101
New member
Robots.text is common name of a text file that is uploaded to a Web site's root directory and linked in the HTML code of the Web site.
ChristinaCa
New member
Robot.text is a text file you put on your site to tell robot which page would you like this is a hidden, relentless force that permeates the web and its billions of web pages and files, unbeknownst to the majority of us sentient beings.
cherryjohn258
New member
I agree wityh you and well said dear.Robot.txt is a text file which you put in your website to tell search engine which pages you would like not to crawl.
I am gonna give you some idea how to create Robot.tex. Please have a look below:
User Agent: *
Disallow: /Admin/
Disallow: /ce/
Disallow: /dp/
Disallow: /cid-pi-ap/