there a limit to robots.txt file?

BellaThomas · Dec 1, 2015

davidweb09 · Dec 2, 2015

no there is no limit with robots file only allow & disallow pages.

Amojy · Dec 2, 2015

you can add number of allow and disallow in robot txt.. there are no limitiation

PariJain · Dec 2, 2015

As such there is no limit to robot.txt file. It is only used to allow which page should be crawled by the search engine crawler.

salim11 · Dec 2, 2015

I think no limit robots.txt text file.

Fastmailsender

Hina Kumari · Dec 3, 2015

There is no any limit of robot.txt file use. You could block many webpages if you want not to be crawled by search engines.

adi123 · Dec 3, 2015

Robots.txt file is used to direct search engine which webpage should be crawled.There is no such limitation.

Jessicamartin · Dec 3, 2015

Its my opinion

you can add any number of allow and disallow in robots txt. there are no limitiation