Include robots.txt

[Maik Storck]

Hello,
I think it would be very nice if a standard robots.txt file could be included with vbulletin.
In this file, the basic rules should be included to prohibit all non-essential links to search engines.
I think it's easier for developers to create this file, as for us users.
Of course, the file may contain only the standard rules. Special rules, the user must create itself.

http://tracker.vbulletin.com/browse/VBV-7801

[mactep]

It is actually a very good idea. Voted!

[Wayne Luke]

You will probably want to attach a sample of what you're looking for. There are so many ways a robots.txt file can vary which is why we don't currently include one.

[14DH01]

this is a good idée.un robots file by default (basic settings), would make the diagnosis of "suspicious file" know. And agrees that attant file needed.
Being part of the original files "vb".
I voted yes

[DemOnstar]

With you on this one. Voted!

[Wayne Luke]

this is a good idée.un robots file by default (basic settings), would make the diagnosis of "suspicious file" know. And agrees that attant file needed. Being part of the original files "vb". I voted yes

robots.txt would have nothing to do with suspicious files. Even if we shipped a default, we would have to exclude it from the file check as many would modify it. We'd probably only include major search engines (Google, Bing, Yahoo!, Ask...)

[JamalFree]

nice idea but i cant vot
You cannot vote or change your vote on resolved issues.

[Wayne Luke]

Yes, this is something we have decided not to include in the default package. There are too many variables to satisfy a wide variety of customers. If you want help setting up a robots.txt file, please post in the appropriate support forum.