The robot.txt file is a file in your main (root) directory of your web site that is retrieved by web robots, such as search engine spiders, to see if you would like to exclude them from accessing any part of your web site. If you don't feel the need to restrict these web robots, it is best to have an empty robots.txt file in your root directory, so the failed accesses are not recorded in your web stats and hide the other reasons for failed access, such as accidentally deleted pages.

Our list of other frequently asked questions or FAQ's:

If you did not find the FAQ you were looking for, why not try our main how-to-guides site.

Frequently Asked Questions