Announcement

Collapse
No announcement yet.

robots.text

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • robots.text

    can any one tell me what should be in this file and what it does and where it goes c panel shows ip's have been looking for it

  • #2
    The robots.txt file is a file that search index spiders read before crawling your site to index the content. You can tell the spiders to stay away from your site, or certain files or directories. So they don't get included in search engines.

    Personally I do not believe in this really because there are enough search engines that ignore the robots.txt file.

    You place it in the public_html/ root folder. It is the first file on the web site, which applies to all files/folders below it.

    Comment

    widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
    Working...
    X