For SEO purposes, do I want these 'robot' commands?

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • thesackshack
    Member
    • Jul 2008
    • 49

    For SEO purposes, do I want these 'robot' commands?

    First off, is it a good or bad thing for robots to: index, follow, and archive your forum?

    Ive been reading commands to add to stop them from doing so, but I'm new to the SEO game and am not sure exactly what those do to a site
  • Nick
    Senior Member
    • Feb 2008
    • 3507
    • 3.8.x

    #2
    It's fine to let robots completely index your site unless you have certain content that you do not want to be indexed (for whatever reason that may be).
    Let's say I had domain www.abc.com and I had a site at /site and forums at /forums.

    If I didn't want robots to crawl the entire forum, I would place /forums in the robots.txt file.

    Learn more about a robots.txt file:
    Learn about the robots.txt, and how it can be used to control how search engines and crawlers do on your site.
    Regards,
    Nick

    Comment

    • thesackshack
      Member
      • Jul 2008
      • 49

      #3
      Gotcha. Thanks Nick. I was wondering b/c when I typed my site name into Yahoo! and it was indexed. Unfort. it was indexed when the site was under construction, so it's cached that way :/

      Comment

      • MadK
        Member
        • Feb 2008
        • 99
        • 3.7.x

        #4
        Originally posted by thesackshack
        Gotcha. Thanks Nick. I was wondering b/c when I typed my site name into Yahoo! and it was indexed. Unfort. it was indexed when the site was under construction, so it's cached that way :/
        The cached version of your website is going to update itself overtime.

        Comment

        widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
        Working...
        😀
        😂
        🥰
        😘
        🤢
        😎
        😞
        😡
        👍
        👎