Google Bots

[NOJ75]

Hi all,

Could anyone answer this question for me?
I have my vB installed in into a folder called "forums" on my root. Inside "forums", along with the rest of the installation, I have placed a file called robots.txt containing the following:

User-agent: *
Disallow: /

The idea is so that I can prevent my forum from being listed on any search engine. Could anyone tell me if this is correct?
Have I achieved not allowing bots into my forums?

Thanks in advance.

NOJ75

[Floris]

That should work, but put the file inside your public_html/ directory and not inside public_html/forum/ (to be sure you can place it in both hehe)

Note that this is just a recommendation to tell bots to stop crawling that directory. However, Google might listen, but many other will ignore the request.

[NOJ75]

Hi Floris,
Thanks for the reply, I dont want the bots to stop coming to my site (which is in the root public_html/ directory)
I just want to stop bots coming into my forums thats all. I want them on my site as much as possible. But the forums contain private content to my members so I dont want the bots to go there.

Can you advise me on this.

Thankyou very much.

NOJ75

[feldon23]

Check out the Disallow parameter here.

[NOJ75]

Thanks feldon23,

So, from what I can gather there, the robots.txt file needs to be in my root and I require the following text in the file:

User-agent: *
Disalow: /forums/

That would stop them going to my forums but allow them into my root site, yes?

Thanks

NOJ75

[Floris]

In that case only disallow the forum path