robots.text

[wock]

can any one tell me what should be in this file and what it does and where it goes c panel shows ip's have been looking for it

[Floris]

The robots.txt file is a file that search index spiders read before crawling your site to index the content. You can tell the spiders to stay away from your site, or certain files or directories. So they don't get included in search engines.

Personally I do not believe in this really because there are enough search engines that ignore the robots.txt file.

You place it in the public_html/ root folder. It is the first file on the web site, which applies to all files/folders below it.