-
If you search the forums you will find a number of suggestions. You shouldn't include the sitemap files because the entire purpose of the sitemap XML files is to allow bots to find content on your site. -
robots file for 4.2.1
Can someone please suggest a robots file for 4.2.1.
Do I need to include the sitemap?
And what should the sitemap be please in the robots file?
-
Hello,
I think it would be very nice if a standard robots.txt file could be included with vbulletin.
In this file, the basic rules should be included to prohibit all non-essential links...Sun 10 Feb '13, 3:07am
Leave a comment: