Announcement

Collapse
No announcement yet.

boardtracker

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • boardtracker

    boardtracker is showing up on hundreds of forums at the samer time

    http://www.aota.net/forums/showthread.php?t=19901

    his home page here http://www.boardtracker.com

    the only way i could get rid of whatever he was doing was to delete the username boardtracker. banning the user just didnt seem to work. it must be some sort of bot?

    vbulletin.org is being scanned as we speak also.

  • #2
    Yes, I believe it is a bot. You could try using robots.txt to keep it from scanning your forum...

    MGM out

    Comment


    • #3
      what do i put in my robots.txt to stop him?

      Comment


      • #4
        It just joined my firearm forum...and this might be an issue.

        Comment


        • #5
          For now you guys should ban the account and block any email addresses from the domain broadtracker.com.

          This is all I have seen it use to register, I don't think it respects the rules of robot.txt.

          Also note that doing so will not prevent it from viewing the forum as a guest. But it will prevent it from becoming a member and thus part of the registered usergroup.

          If you want to block it with a bit of trickery stick this in the phpinclude_start template

          PHP Code:
          // block boardtracker account and send it a fake 404
          // change '666' to the userid of the boardtracker account
          if ($bbuserinfo['userid'] == 666)
          {
                     @
          header("HTTP/1.0 404 Not Found");
                     @
          header("Status: 404 Not Found");
                     exit;

          That'll send it fake 404's and it won't be able to see anything, at least while logged in...

          Make sure to change the number 666 to the userid of the boardtracker user account.

          Edit: I've logged it under this ip: 212.235.16.174
          Last edited by Brad.loo; Thu 25 Aug '05, 11:21am.

          Comment


          • #6
            I thought the whole point is to have search engines index your website and make it available. Doesn't having this index your site increase link backs and the almighty pagerank in the long run? How is it doing anything different than Googlebot? Just put it in a group with the same permissions as a guest user.
            Translations provided by Google.

            Wayne Luke
            The Rabid Badger - a vBulletin Cloud demonstration site.
            vBulletin 5 API

            Comment


            • #7
              How is it doing anything different than Googlebot?
              that remains the question. unless someone can examin what this bot does on their server we dont really know.

              why does he need to register and stay online? i dont remember other search engines registering a username and sitting there?

              in the end this may all be a harmless and legitimate. it was just the fact it was very invasive by registering and scanning and banning the username would not get rid of it wither. i had to remove the usernam via admin CP before it would go away. not even shutting off the board n maintainence mode stopped it.

              I did email him asking for an explaination and got an email response back explaining what the bot does. but i don't want to post private email here so if an vbulletin staff want to take a look at it i will PM them.
              Last edited by Razgo; Thu 25 Aug '05, 1:29pm.

              Comment


              • #8
                Originally posted by Wayne Luke
                I thought the whole point is to have search engines index your website and make it available. Doesn't having this index your site increase link backs and the almighty pagerank in the long run? How is it doing anything different than Googlebot? Just put it in a group with the same permissions as a guest user.
                I do not fancy the idea of a bot reading forums I have hidden to guests (registered users can see them). Simply put I don't need my private threads in some search index.

                Sure you can change its usergroup to guest, but why should I have to do that in the first place? Why does it need to register? Why does it leave a link back to the website it's from in the user profile? Why does it by-pass my image verification? (I know it's 'cracked' in 3.0.x but even so it ain't right to exploit that hole from a bussiness stand point is it?)

                I've yet to find out if it can be blocked with robots.txt, as far as I can tell it is not listed on their site. If it can be I'll just block it and be done with it, at least until they clean up their act and stop registering with a script on my forum.

                To this point it has been one bot per forum, what happens when they send as many as yahoo or google except every single one of them would be registered? Are you going to want to set all these users to a new usergroup?
                Last edited by Brad.loo; Thu 25 Aug '05, 7:33pm.

                Comment


                • #9
                  Originally posted by Razgo
                  that remains the question. unless someone can examin what this bot does on their server we dont really know.

                  why does he need to register and stay online? i dont remember other search engines registering a username and sitting there?

                  in the end this may all be a harmless and legitimate. it was just the fact it was very invasive by registering and scanning and banning the username would not get rid of it wither. i had to remove the usernam via admin CP before it would go away. not even shutting off the board n maintainence mode stopped it.

                  I did email him asking for an explaination and got an email response back explaining what the bot does. but i don't want to post private email here so if an vbulletin staff want to take a look at it i will PM them.
                  On my forum I didn't delete the user so I could ban him with the code I posted above, no one can see his account because I made him hidden and it can't see anything because it's stuck on a 404 error

                  Comment


                  • #10
                    On my forum I didn't delete the user so I could ban him with the code I posted above, no one can see his account because I made him hidden and it can't see anything because it's stuck on a 404 error
                    i had to delete at the time because i kind of panicked when i realized he was stuck there and couldnt remove him. if he registers again i would use your above technic instead.

                    Why does it need to register? Why does it leave a link back to the website it's from in the user profile? Why does it by-pass my image verification?
                    these were my concerns also and i am glad i err on the side of caution and find his approach rather rude if you ask me.

                    Comment


                    • #11
                      if you have modrewrite, it would be better for you to block the bot using .htaccess
                      (assuming it uses the same ip range and a custom user agent string that is easily identifiable)

                      RewriteEngine on
                      RewriteCond %{HTTP_USER_AGENT} User Agent String [OR] #ban user agent
                      RewriteCond %{REMOTE_ADDR} ^123.456.789.000 #ban ip
                      RewriteRule .* - [F]
                      that will give them the forbidden message when they visit your site.
                      http://www.virtualinfusion.net
                      An Anime and Manga discussion forum.

                      Comment


                      • #12
                        This is BoardTracker, here to answer your concerns about our spider..

                        We do not use any kind of 'crack' or 'exploit' in vb code.. every 'BoardTracker' account you see on any board, including this one, was created manually by a real person.. if you send email to the address used in the registration you will get a reply from a real person.

                        We have our url in the 'homepage' section of our board profiles for the same reason every user of every board does.. there is nothing nefarious about that! For one thing it helps people (board owners) find out more about us when they see us logged in.

                        You can read more about us and our spider here..

                        http://www.boardtracker.com/cgi-bin/about.pl

                        Any and all information we harvest is all publicly available to anyone who fills in the simple board registration form.. if its viewable by regular logged in board users then we can spider it. If its not visible to regular users then we do not spider it. If board registration requires any special conditions (ie. its a private/paid system not open to the general public) then we do not spider it. We do not spider forums only visible to private usergroups, moderators or any forum that requires special access privileges.

                        Also we only display a very short snippet of any thread and people must click the link to go directly to the board and login before they can read the full thread. We do not show the full thread and we do not have any kind of cache (like google cache) so all your content stays on your site. In this way we bring new traffic and members to the boards.

                        Still if there are any specific forums that you do not want spidered, even though they are freely viewable to anyone that wants to signup at your board, we can remove such forums. Just contact us at the email address we use to register at the board or visit our site to find out more.

                        For those wondering why you see the 'BoardTracker' user logged in all the time it is simple - the nature of forums is that they contain rapidly changing data, new threads are added all day, every day and so to provide a valuable search service specifically for forums we have to spider that data as soon as possible - time is of the essense in this case. Unlike google or other search engines that may spider your pages and a month or so later they will be searchable in their index by which time it may be 'stale'. If a major event happens anywhere in the world you can be sure within minutes there is a thread about it on a forum somewhere and shortly after it will be searchable in boardtracker!

                        Also the spider is designed to minimize the strain on boards, we do not sit there all day spidering every thread over and over again, we view it once like a regular user.

                        Another unique aspect of boardtracker which helps the boards is our alerts mechanism - users can pre-define some search terms for things that interest them (name, company, products, whatever) and as soon as someone posts a thread on any board we track we will send them an alert by email or IM so that they can go straight there and read/reply etc. This ensures people will never miss a thread that interests them no matter where or when it is posted.

                        In our beta tests we have found that boardtracker often becomes the top referrer of new members to boards we track - board owners are generally happy about this!

                        To this point it has been one bot per forum, what happens when they send as many as yahoo or google except every single one of them would be registered? Are you going to want to set all these users to a new usergroup?
                        I'm not sure what you mean by this - if we get as big as google/yahoo (we can hope!) we will be sending you more traffic.. they would still need to register to your forum to view the threads same as any visitor from any search engine or elsewhere..

                        If anyone has any more concerns or questions about BoardTracker (or suggestions for improvement!) feel free to contact us - we're not just a bot!

                        Also I would recommend you visit our site and give it a try - I think you will find it a very useful search engine and service.

                        Comment


                        • #13
                          Ok.. So why are you charging people (premium account) to see my MSN address
                          Example
                          Which is information ive made public..
                          ------------
                          On another note
                          Why has no just suggested that you put the name "Boardtracker" in the disallowed usernames...

                          --
                          While its seems a great idea, I cant help but feel that its poorly implemented.
                          Playstation-Racing
                          A vB powered community

                          Comment


                          • #14
                            Originally posted by Impreze04
                            Ok.. So why are you charging people (premium account) to see my MSN address
                            Example
                            Which is information ive made public..
                            I don't fully understand. Is the problem now that we show too little? or too much?
                            The first link on this page you showed as an example, is a link to the board. How bad is that? And if the information is fully public, people can see your MSN address on that page. I am sure board owners will not complain about us sending people to their boards.
                            As for premium accounts, IM details on BoardTracker is only one feature on the premium account features. We do not charge people for your MSN address, but for a premium account which is a more advanced account. Your MSN address is freely available on the board itself, and we directly link to it.

                            Originally posted by Impreze04
                            While its seems a great idea, I cant help but feel that its poorly implemented.
                            How did you come to that conclusion?
                            Lets consider several aspects of BoardTracker.
                            on the usability side:
                            1. BoardTracker is catering especially to Message board and its community. The only information you find in BoardTracker is a true community related information. Valid information linked directly to the discussions that you are interested in. What other search engine does that?
                            2. BoardTracker is a categorized system. if you are searching for Racing lets say.. in http://www.boardtracker.com/search/?search=racing&cat=331&filter=0]computers & Internet > Games[/url], you find only relevant information related to what you were looking for. What other search engine does that?
                            3. BoardTracker links directly to the message board, never shows anything more than a very short snippet of the thread, and does not show a cached version of the pages on our site (like some other search engines do, as you probably know). Every single thread on BoardTracker has a link to the board - directly to the relevant thread. This encourages people to go to the boards, signup and get involved in the discussion.
                            4. BoardTracker updates very frequently, which allows people to find threads they are interested in, not 20 days after they started, but very soon after. In many cases, this helps people realize that if they have something to contribute, or a question they want to ask, they can go to the board, signup and get involved in the community. What other search engine does that?
                            5. BoardTracker allows people to search by date as well. So that when you are looking for information, you can find the latest and most recent. This in most cases makes the search results more valuable, and raises the likelihood of that person joining the community (or at least going to the board to actually read the thread).
                            6. BoardTracker searches for relevant information on a board. Try searching for : previous thread | next thread
                            Try it on BoardTracker. Now, try it on any other search engine. You see how other search engines find any possible page on any message board out there? why? because it appears on the bottom of any page on a every thread on a message board (at least VBulletin).
                            BoardTracker doesn't do that. BoardTracker only searches in what matters - the text people write. When people are looking for something on BoardTracker, they find the relevant thread, and only those. What other search engine does that?

                            On the technical side:
                            1. BoardTracker does not abuse board's resources like most other search engines do.
                            - 1.1 BoardTracker only spiders a thread once. Other search engines scan the same page again and again. Not only that, but they regard a page with different session variables as different pages and scan those same copies again and again. BoardTracker only scans threads based on their unique identifier and only once.
                            - 1.2 BoardTracker uses an account to be able to scan a thread properly, to scan a member's name, but more importantly, to avoid abuse of the board's session table. While most other search engines scan pages on a board as "visitors" and by that cause many times for a session record to be created for every page request, BoardTracker uses an account and a session cookie. This reduces the load on a board.
                            - 1.3 BoardTracker doesn't scan images, resource pages or other non-relevant links/objects. This reduces the load on a server even more.
                            2. BoardTracker's board inclusion is manual. It's not an automated bot that spiders any site/page even if its irrelevant or fraudulent. This enables you to search only relevant and valuable information.
                            3. BoardTracker's categorization of the different forums is manual. This helps ensure that information is cataloged in a smart way and therefore accessible to the right people. When you look for hosting information you get exactly that.

                            Please let me know if (and if so, how) you find this implementation poor.
                            We would be interested to hear your view of things (which brings me to the last point - which other search engine do you know, that takes the time to adequately reply to people's concerns on boards?)

                            Best regards,
                            BoardTracker.com

                            Comment


                            • #15
                              Regarding robots.txt

                              We are compliant with the robots.txt file and have just added functionality to allow you to control access to specific forums within your board as well as the board as a whole so now if there is a forum you do not want spidered for some reason you can specify that easily.

                              Full details are here..

                              http://boardtracker.com/cgi-bin/about.pl?page=4

                              Bear in mind that if you remove a forum you will lose the potential traffic from people searching for topics about which you may have threads and so you lose potential new members for your board. So I advise board owners to look carefully at how boardtracker works and interacts with your board and benefits your community before deciding to remove a forum.

                              As always if you have any questions please feel free to contact us.

                              Comment

                              widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                              Working...
                              X