Robots.txt File And Ranking?
Posted by Thomas | Posted in questions | Posted on 26-09-2009
Tags: File, Ranking, Robots.txt
3
I just read this on http://www.caffeinemarketing.com/?p=96 “If you’ve submitted your web site to the search engines than the spider robots will be searching for your robots.txt file. If they do not find one the robot generates a ‘404 File Not Found’ error everytime. These robots might be searching for that file several times a day. Adding the file to your root directory will help eliminate these errors and boost your rankings.”
Is this true? Can robots.txt help with search engine optimization?




























