I just want to confirm that if I want to disallow all robots from an entire directory and all its subdirectories, that I just have to disallow that specific root directory.
i.e., To disallow the following directories and all content contained in them:
/poker/
/poker/sites/
/poker/sites/pokerstars/
/poker/sites/pokerstars/review/
I only need to disallow the "/poker/" directory right?
User-Agent: *
Disallow: /poker/
Also, what is the consequence (if any) of not having a robots.txt file as opposed to a generic one like:
Any ramifications?User-Agent: *
Disallow:













LinkBack URL
About LinkBacks
Reply With Quote
Bookmarks