Announcement

Collapse
No announcement yet.

Site for helping you get your vb2 boards spidered

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Originally posted by DirectPixel
    Now, why would you want an opt-out link unless you're spidering sites OTHER than what people have signed up for?
    ok, take the time and goto google. like i said earlier, if someone has a link to your site from theirs, spider goes there. google has an opt out option aswell.

    you people should know a few things before getting ahead of yourselves and posting uneducated comments i think.

    Comment


    • #17
      Originally posted by cirisme
      Do you obey robots.txt? Can we have the spider's Ip so we can ban it if we don't wish to have our sites indexed?
      yes it does obey robots.txt. that i cant give you as the ip isnt static.

      Comment


      • #18
        Originally posted by suffeks
        ok, take the time and goto google. like i said earlier, if someone has a link to your site from theirs, spider goes there. google has an opt out option aswell.

        you people should know a few things before getting ahead of yourselves and posting uneducated comments i think.
        When a random person from the internet are mirroring my content without my permission, I believe I have a right to ask questions and be critical of it.
        :)

        Comment


        • #19
          Time to make a phonecall

          And just to be sure: I do NOT give you permission to spider my web sites or put their content on yours.

          Comment


          • #20
            Originally posted by suffeks
            so i guess everyone should sue google. as long as there is a link to the original thread, its not copyright infringement, especially since the information is public and posted by individuals.
            The difference between Google and your site is quite major.

            1) It also does not show other peoples copyrighted content as it's primary function.

            2) Google primarily links to pages, and only has a backup for exactly that reason - backup.

            3) Google also makes it obvious that the page you are seeing is a cached version of the page and not the original.

            4) Google does not modify the page to fit it's own layout and make it look like original content.

            I think number 4 is the important one here and the one that will most likely get you sued.
            Raz - KMC Forums

            Comment


            • #21
              5 - You aren't allowed to copy a page 100% - You have to provide a small introduction and link back to the rest.

              6 - You have to provide the appropiate credit to used copyrights of software/users/sites/etc. The pages do not even have a disclaimer, they just say 'this page is my copyright'- which in fact, none of it is yours.

              Comment


              • #22
                I actually like the idea. Though you really should ask before you take.

                Comment


                • #23
                  The idea would be good if it where in a different format. Also can you provide the hostname of your bot, so that can be banned instead of the ip, preferably your bot takes after the domain so I guess if you ban the domain name in robots.txt it will be restricted.

                  Comment


                  • #24
                    Or... even the useragent would suffice.
                    TheologyWeb. We debate theology. srsly.

                    Comment

                    widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                    Working...
                    X