petershene
New member
Hello guys ive been trying to use my ht access file to block spiders by their user agent string and by their ip range's , the problem is that these bot still access me because they dont have to identify themselves as the user agent strings i have laid out. Also they can switch ip address out of the ranges i have blocked. I have sen some people block most back-link tools through php maybe. iam not sure i know there are a ton of online advice on this all involving robot .txt and ht access mod re writes but there must be another method i am missing , because my website is still crawl-able