No announcement yet.

Google Bots

  • Filter
  • Time
  • Show
Clear All
new posts

  • Google Bots

    Hi all,

    Could anyone answer this question for me?
    I have my vB installed in into a folder called "forums" on my root. Inside "forums", along with the rest of the installation, I have placed a file called robots.txt containing the following:

    User-agent: *
    Disallow: /

    The idea is so that I can prevent my forum from being listed on any search engine. Could anyone tell me if this is correct?
    Have I achieved not allowing bots into my forums?

    Thanks in advance.


  • #2
    That should work, but put the file inside your public_html/ directory and not inside public_html/forum/ (to be sure you can place it in both hehe)

    Note that this is just a recommendation to tell bots to stop crawling that directory. However, Google might listen, but many other will ignore the request.


    • #3
      Hi Floris,
      Thanks for the reply, I dont want the bots to stop coming to my site (which is in the root public_html/ directory)
      I just want to stop bots coming into my forums thats all. I want them on my site as much as possible. But the forums contain private content to my members so I dont want the bots to go there.

      Can you advise me on this.

      Thankyou very much.



      • #4
        Check out the Disallow parameter here.


        • #5
          Thanks feldon23,

          So, from what I can gather there, the robots.txt file needs to be in my root and I require the following text in the file:

          User-agent: *
          Disalow: /forums/

          That would stop them going to my forums but allow them into my root site, yes?




          • #6
            In that case only disallow the forum path


            widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.