No announcement yet.

Email spam via VBulletin 4.2.2

  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Many users can come from the same ip. Its not really all that uncommon, both logged in, and guests. I don't see a good reason to limit their sessions. Are you dealing with more than 50,000 users online at the same time?


    • #17
      I'm dealing with 0 real users online at the same time and 50,000 attackers at the same time that have the same user ID and IP address(es). Thus cron.php never runs and they are able to keep creating sessions until MySQL craps out. I can ban the IPs, but they just change IPs. I can ban the user, but they can still do it as guests. This attack has been ongoing for several weeks now.

      There would be less of a need to use any limiting factor if session cleanup was moved out of cron and into the main script. The best time for this is immediately after new session creation, since then you don't need to worry about the various exits and die()s that vBulletin uses.

      I suggested limiting because it would also prevent the attack from gaining traction before the session timeout is reached and because it was the first solution I tried that actually worked. Additionally, even if session cleanup was in the main script, any decent script-based implementation of the attack would be able to achieve "session is full" before the first session expires.

      I understand that 1 username might be logged into a desktop PC, laptop PC, tablet/phone, console, etc all at the same time and they happen to have the same IP, but I don't believe it's reasonable to treat all of these sessions as if they are active. At a certain point, you should say "You can log in with your 3rd tablet after your 1st PC session expires in 5 minutes." Honestly I don't see any one user-name other than an admin developing on their own site that would need so many sessions in the same 15 minute period.

      The fix I provided does NOT limit different logged-in users that happen to have the same IP. It only limits the same user at the same IP.

      Guests using the same corporate router is a different story. Limiting them needs to be approached slightly differently. I'm open to ideas. But as it stands, the current behavior of creating new sessions without removing any is not acceptable to the server hardware.
      Last edited by thincom2000; Tue 6 Jan '15, 9:19am.
      - the makers of VaultWiki


      • #18
        If you're getting attacked by over 50,000 concurrent connections, you should be fighting that at the hardware/network level. Not at the application level.


        • #19
          First, the kind of attack we're discussing affects very small hobby and niche forums with low traffic -- I've mentioned that low traffic is required to perform that attack successfully. As soon as non-attackers start accessing the forum, the attack gradually loses traction -- the question is does it happen before sessions run out. These kinds of small forums typically use shared hosting and do not have access to modify hardware or network level firewalls.

          I have a dedicated server and do have access to those things, but you cannot just assume that all forums do. Now on my dedicated server, I host a number of forums, including 1 small forum that is under attack.

          At the server level, I'm actually being attacked by 1-3 concurrent connections during periods of inactivity on my forum (there are a few hours each day when there is little to no traffic, when people are sleeping). But vBulletin doesn't clear the session table, so it builds to 50,000 that vBulletin incorrectly considers as "active concurrent sessions" in roughly an hour. The server only counts 1-3 users, vBulletin counts 50,000. As I've argued, these sessions are neither active nor are they concurrent, but the session logic treats them that way.

          This cannot be detected at the hardware/network level because 1 user performing <200 connections / 1 second is reasonable from the server's perspective. We tried to make this throttle harder than 200req/sec, but it broke the user experience due to image, script, AJAX, etc resources that are loaded concurrently by most users with empty caches -- almost everyone started getting blocked.

          For the attack my forum is facing, this particular bot runs 0 - 10 HTML-only requests per second (without image or script embeds too, because it's accessing a script with no output). This actually looks less threatening than a standard user to a server-level firewall, but it can fill up vBulletin's session table in about 1 hour on a default MySQL config with a 16MB heap size. There is really no way for the server to detect this on its own, because it otherwise looks like normal traffic that just happens to not be loading the cron.php image in the response (if a response). It's a problem with how vBulletin tracks user sessions that's being exploited.

          Since the problem is actually at the application level (it is generating session data for requests that should be reusing old session data, and it is also not removing stale session data), it must be prevented at the application level. As I noted, by simply adding 1 MySQL query to the session logic, while the attack can still be ongoing, it no longer crashes the database over time. In the end, how to best craft this query to accommodate the most users is up to vBulletin's developers. A decent application-level solution is to cleanup stale sessions whenever a new session is created, rather than waiting for a cron script that is bypassed by attackers. If anything it would at least shorten the attack window from roughly 1 hour, to 15 minutes (however, it is still very easy for a script-based attack to bypass a server firewall and overflow the session table in 15 minutes using the same attack vector).
          Last edited by thincom2000; Tue 6 Jan '15, 1:00pm.
          - the makers of VaultWiki


          • #20
            Why not just move your cron to a scheduled task then?


            • #21
              I can certainly do that, and probably will, moving forward. However, it is not a standard vBulletin setup to list cron.php in the server's crontab -- that is, it is not in the installation instructions as far as I'm aware, nor is it suggested by the official documentation on Scheduled Tasks). It does not change the fact that a default installation of vBulletin 3 or 4 is vulnerable to this type of attack. I simply request that measures be taken at the application level to protect small low-traffic forums. I have given my suggestions for how this can be achieved.
              - the makers of VaultWiki


              • #22
                Even if this check were added, they would end up just overloading the max database connections instead, and ultimately, web/server connections.
                Baby, I was born this way


                • #23
                  Well, they could do that anyway with any kind of distributed attack, but it is harder to accomplish. By contrast, the session table attack I've discussed requires very few resources or coordination and can be accomplished with just 1 user very slowly. Hitting max database connections would not occur using this attack any faster than it would occur with normal forum traffic.

                  I'm sorry but deciding not to add protection against 1 attack type or to close one vulnerability because an attacker might just alter their attack is not a valid argument. If developers thought that way, every software would be riddled with vulnerabilities (no reason to fix one security hole, because there's another one).
                  - the makers of VaultWiki


                  • #24
                    Hey everyone.

                    I just had this problem and figured out how to stop it.

                    There are two exploits going on -- one through blog.php (if you have blog functionality) and one with sendmessage.php (e-mail to a friend).

                    You can comment out the functionality from both to assure that this won't continue.

                    In sendmessage.php, search for "start send to friend" and then comment out that entire function. The search for "start do send to friend" and again, comment out the entire function.

                    Do the exact same thing for blog.php.

                    If you don't feel like screwing with the code, you can also disable the sendtomessage.php exploit by going into the admin control panel: Usergroups > Usergroup Manager > Edit Usergroup/ Then set Can Use Email to Friend to "No". Make sure to also do this for blogs in that same Usergroup settings if you have blog (Can Use Email to Friend will appear twice -- one for forums, once for blogs -- set both to NO). However, I have heard that some spammers find a way around that, so the best solution is to do this AND comment it out of the code.


                    Related Topics