No announcement yet.

Millions of page hits. Why?

  • Filter
  • Time
  • Show
Clear All
new posts

  • Wayne Luke
    I don't know why you're getting millions of hits. The software appears to be running properly on your site. Are you running Google Analytics on your site? If not, you should as that will give you a better view of the traffic since you don't seem to have access to the raw Apache logs.

    Apache's raw access logs look like this:
    ::1 - - [08/Feb/2018:09:19:24 -0800] "GET /vb5_live/core/cpstyles/vBulletin_5_Default/cp_logo.png HTTP/1.1" 200 5860
    ::1 - - [08/Feb/2018:09:19:24 -0800] "GET /vb5_live/core/cpstyles/vBulletin_5_Default/admincp_modhd_back.png HTTP/1.1" 200 99
    ::1 - - [08/Feb/2018:09:19:16 -0800] "GET /vb5_live/admincp/index.php HTTP/1.1" 200 11341
    ::1 - - [08/Feb/2018:09:19:25 -0800] "GET /vb5_live/core/cpstyles/vBulletin_5_Default/admincp_btn_secondary.png HTTP/1.1" 200 85
    ::1 - - [08/Feb/2018:09:19:26 -0800] "POST /vb5_live/login.php?do=login HTTP/1.1" 200 465
    ::1 - - [08/Feb/2018:09:19:31 -0800] "GET /vb5_live/admincp/index.php HTTP/1.1" 200 1346
    ::1 - - [08/Feb/2018:09:19:34 -0800] "GET /vb5_live/core/cpstyles/vBulletin_5_Default/admincp_hd_back.png HTTP/1.1" 200 155
    ::1 - - [08/Feb/2018:09:19:34 -0800] "GET /vb5_live/core/cpstyles/vBulletin_5_Default/cp_expand_ltr.png HTTP/1.1" 200 241
    ::1 - - [08/Feb/2018:09:19:34 -0800] "GET /vb5_live/core/cpstyles/vBulletin_5_Default/admincp_lnav_btn_default.png HTTP/1.1" 200 106
    ::1 - - [08/Feb/2018:09:19:33 -0800] "GET /vb5_live/admincp/index.php?do=head HTTP/1.1" 200 9466
    ::1 - - [08/Feb/2018:09:19:33 -0800] "GET /vb5_live/admincp/index.php?do=nav HTTP/1.1" 200 45635
    ::1 - - [08/Feb/2018:09:19:36 -0800] "GET /vb5_live/core/cpstyles/vBulletin_5_Default/cp_help.png HTTP/1.1" 200 595
    ::1 - - [08/Feb/2018:09:19:33 -0800] "GET /vb5_live/admincp/index.php?do=home HTTP/1.1" 200 22376
    ::1 - - [08/Feb/2018:09:19:37 -0800] "GET /vb5_live/core/clientscript/vbulletin_cphome_scripts.js?v=533 HTTP/1.1" 200 4703
    ::1 - - [08/Feb/2018:09:19:37 -0800] "GET /vb5_live/core/cpstyles/vBulletin_5_Default/admincp_modtitle_back.png HTTP/1.1" 200 113
    ::1 - - [08/Feb/2018:09:19:37 -0800] "POST /vb5_live/admincp/newsproxy.php HTTP/1.1" 200 2559 - - [08/Feb/2018:09:28:05 -0800] "GET /vb5_live/core/cpstyles/global.css?v=534 HTTP/1.1" 200 3277
    ::1 - - [08/Feb/2018:09:28:05 -0800] "GET /vb5_live/core/cpstyles/vBulletin_5_Default/controlpanel.css?v=534 HTTP/1.1" 200 12416
    ::1 - - [08/Feb/2018:09:28:00 -0800] "GET /vb5_live/admincp/index.php HTTP/1.1" 200 7914
    ::1 - - [08/Feb/2018:09:28:55 -0800] "GET /vb5_live/admincp/index.php HTTP/1.1" 200 1346
    ::1 - - [08/Feb/2018:09:28:55 -0800] "GET /vb5_live/admincp/index.php?do=head HTTP/1.1" 200 9466
    ::1 - - [08/Feb/2018:09:28:55 -0800] "GET /vb5_live/admincp/index.php?do=nav HTTP/1.1" 200 45635
    ::1 - - [08/Feb/2018:09:28:55 -0800] "GET /vb5_live/admincp/index.php?do=home HTTP/1.1" 200 23005
    ::1 - - [08/Feb/2018:09:28:57 -0800] "GET /vb5_live/core/clientscript/vbulletin_cphome_scripts.js?v=534 HTTP/1.1" 200 4703
    ::1 - - [08/Feb/2018:09:28:57 -0800] "POST /vb5_live/admincp/newsproxy.php HTTP/1.1" 200 2559
    ::1 - - [08/Feb/2018:09:29:17 -0800] "GET /vb5_live/admincp/index.php HTTP/1.1" 200 1346
    ::1 - - [08/Feb/2018:09:29:18 -0800] "GET /vb5_live/admincp/index.php?do=head HTTP/1.1" 200 9466
    ::1 - - [08/Feb/2018:09:29:18 -0800] "GET /vb5_live/admincp/index.php?do=nav HTTP/1.1" 200 45635
    ::1 - - [08/Feb/2018:09:29:18 -0800] "GET /vb5_live/admincp/index.php?do=home HTTP/1.1" 200 23005
    ::1 - - [08/Feb/2018:09:29:19 -0800] "POST /vb5_live/admincp/newsproxy.php HTTP/1.1" 200 2559
    ::1 - - [08/Feb/2018:09:31:04 -0800] "GET /vb5_live/admincp/index.php HTTP/1.1" 200 1346
    ::1 - - [08/Feb/2018:09:31:08 -0800] "GET /vb5_live/admincp/index.php?do=head HTTP/1.1" 200 9480 - - [08/Feb/2018:09:31:08 -0800] "GET /vb5_live/admincp/index.php?do=nav HTTP/1.1" 200 45635
    ::1 - - [08/Feb/2018:09:31:08 -0800] "GET /vb5_live/admincp/index.php?do=home HTTP/1.1" 200 23047
    ::1 - - [08/Feb/2018:09:31:10 -0800] "POST /vb5_live/admincp/newsproxy.php HTTP/1.1" 200 2559
    MemcacheD has nothing to do with the number of page views. It just makes page views faster.

    Leave a comment:

  • TimothyJMcGowan
    Did you pick the wrong word? Surely that should be set to No; right?

    From the text on the page in Settings:
    Add No-Cache HTTP Headers
    Selecting yes will cause vBulletin to add no-cache HTTP headers. These are very effective, so adding them may cause server load to increase due to an increase in page requests.

    From the help text for that item:
    If you enable this option, no-cache HTTP headers will be added to each page. These cause full page data to be re-requested from the server every time the user gets a page, which will increase bandwidth; it may additionally cause an increase in server load as it will cause more pages to be served.

    And I think our .htaccess file is set properly for caching:
    DirectoryIndex ForumCenter.htm
    <IfModule mod_rewrite.c>
    RewriteEngine On

    # In some cases where you have other mod_rewrite rules, you may need to remove the
    # comment on the following RewriteBase line and change it to match your folder name.
    # This resets the other mod_rewrite rules for just this directory
    # If your site was, the setting would be /forum/
    RewriteBase /

    #To redirect users to the secure version of your site, uncomment the lines below
    #RewriteCond %{HTTPS} !=on
    #RewriteRule .* https://%{SERVER_NAME}%{REQUEST_URI} [R=301,L]

    # Send css calls directly to the correct file VBV-7807
    RewriteRule ^css.php$ core/css.php [NC,L]

    # Redirect old install path to core.
    RewriteRule ^install/ core/install/ [NC,L]

    # Main Redirect
    RewriteCond %{REQUEST_URI} !\.(gif|jpg|jpeg|png|css)$
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule ^(.*)$ index.php?routestring=$1 [L,QSA]

    # Because admincp is an actual directory.
    RewriteRule ^(admincp/)$ index.php?routestring=$1 [L,QSA]


    <IfModule mod_deflate.c>
    AddOutputFilterByType DEFLATE application/atom+xml \
    text/javascript \
    application/x-javascript \
    application/javascript \
    application/json \
    application/rss+xml \
    application/ \
    application/x-font-ttf \
    application/xhtml+xml \
    application/xml \
    font/opentype \
    image/svg+xml \
    image/x-icon \
    text/css \
    text/html \
    text/plain \
    text/x-component \

    <IfModule mod_expires.c>
    ExpiresActive On
    ExpiresByType application/x-javascript A1209600
    ExpiresByType text/javascript A1209600
    ExpiresByType application/javascript A1209600
    ExpiresByType text/css A31536000
    ExpiresByType image/x-icon A2592000
    ExpiresByType image/icon A2592000
    ExpiresByType application/x-ico A2592000
    ExpiresByType application/ico A2592000
    ExpiresByType image/gif A2592000
    ExpiresByType image/jpeg A1209600
    ExpiresByType image/jpg A1209600
    ExpiresByType image/png A1209600
    ExpiresByType application/x-shockwave-flash A1209600
    ExpiresByType font/ttf A2592000
    ExpiresByType font/otf A2592000
    ExpiresByType font/x-woff A2592000
    ExpiresByType image/svg+xml A2592000
    ExpiresByType font/truetype A2592000
    ExpiresByType font/opentype A2592000
    ExpiresByType application/x-font-woff A2592000
    ExpiresByType application/ A2592000

    <IfModule mod_headers.c>
    Header set Connection keep-alive
    <filesmatch "\.(ico|flv|gif|swf|eot|woff|otf|ttf|svg)$">
    Header set Cache-Control "max-age=2592000, public"
    <filesmatch "\.(jpg|jpeg|png)$">
    Header set Cache-Control "max-age=1209600, public"
    <filesmatch "\.(eot|woff|otf|ttf|svg)$">
    Header set Cache-Control "max-age=2592000, public"
    # css and js should use private for proxy caching
    <filesmatch "\.(css)$">
    Header set Cache-Control "max-age=31536000, private"
    <filesmatch "\.(js)$">
    Header set Cache-Control "max-age=1209600, private"

    #don't allow some files that shouldn't really be present to be directly accessed.
    #note that attachements should never be directly accessed by the webserver because
    #we have permissions on the that are checked in the PHP code.
    <FilesMatch "(^#.*#|\.(bak|config|dist|inc|ini|log|gz|tar|zip|sh|sql|sw[op])|~)$">
    Order allow,deny
    Deny from all
    Satisfy All

    AddHandler application/x-httpd-php71 .php .php5 .php4 .php3

    <Files 403.shtml>
    order allow,deny
    allow from all

    Any other thoughts?

    We don't have memcached enabled. Should I bring that up in a different thread, or can we deal with it here? Or is there a FAQ I missed on that for 5.3.4?

    -- Tim

    Leave a comment:

  • Wayne Luke
    You can tell vBulletin to send headers to the user's browser to ignore caching. We do not recommend this. The setting is under Settings -> Options -> Cookie and HTTP Header Options. It is called "Add No-Cache HTTP Headers". This should be set to Yes.

    Leave a comment:

  • TimothyJMcGowan
    Copied and pasted from the AW Stats report in CPanel, which simply seems to call them pages, not requests. I'm confident you know better than I what it actually means; I know what it says: pages.

    Again, we simply copied our Cloud service to our new server. Thanks for the tip about browser caching. That's the feature through which the individual's Web browser downloads all the pages for all the links on the page the user is looking at just in case the user clicks on any link on the page, all in order to speed up the page load; right? That could certainly slam the server. I'll look for those controls in the software, and then I'll see what I can do to control that through CPanel as well. I'll report back any result, whether effective or not.

    Leave a comment:

  • Wayne Luke
    Are you sure those are pages and not requests? There is a difference. Each page will have multiple requests for the server. There is the HTML code, Javascript files, CSS files, image files, etc... Each attachment and module on the page can also add additional requests to get data and information from the server.

    Do you have browser caching turned off in the software or on the server level?

    Leave a comment:

  • TimothyJMcGowan
    Does this help? I compared the IP addresses shown in CPanel's AWStats with the IP addresses vBulletin logged when the users signed up. (We opened less than two months ago, so the IP addresses used at sign-up are still fairly reliable.)

    These are this month's stats, from the 1st through the 5th.

    As I write this, vBulletin shows that we have 2,113 topics and 31,200 posts. These stats show that the top user, for example, has pulled up three times as many pages as we have topics and has nearly four times as many hits. And she's an idiot. She's not scraping our site or intentionally damaging us. Same, basically, with the rest. They're just users with no other life, and they love to talk politics and religion and to find artful ways to call each other names or fling invective without abridging the forum rules.
    Pages: 6,995
    Hits: 7,695
    Bwidth: 38.62 MB
    Verified prolific user
    Pages: 5,232
    Hits: 6,243
    Bwidth: 35.65 MB
    Verified prolific user
    Pages: 4,819
    Hits: 5,203
    Bwidth: 54.49 MB
    Verified prolific user
    Pages: 4,297
    Hits: 4,458
    Bwidth: 21.53 MB
    Verified prolific user
    Pages: 3,987
    Hits: 4,662
    Bwidth: 44.60 MB
    Verified prolific user

    I don't have time tonight to go through the rest -- but if anyone's attacking us, they're not working at it nearly as hard as our busiest real users, who are posting actual, apposite messages in our forums.

    Leave a comment:

  • Wayne Luke
    Settings in vBulletin won't increase the actual traffic of your site or cause IP Addresses to make more connections. A client of some sort has to be doing it.

    You have to actually look at the logs from the web server and determine which IPs are causing the most traffic and where they are originating from. This is not something that can be done reliably within the vBulletin software. Especially since multiple devices can share the same IP address.

    Leave a comment:

  • TimothyJMcGowan
    That's the point: The highest hitters were our own actual users. I know it sounds like DDOS. That's why I started just blocking IPs as a stopgap measure until I could figure out what was going on. And then our own users, who are decidedly not trying to bring us down, especially so many of them, started complaining of a 403 error, if I remember the number correctly. They couldn't get to the site at all. We've blocked a few Russian IPs, but again, they're not the ones flogging the server.

    So I really think something has been misconfigured, either because of our move from Cloud to self-hosted or because we went with defaults that we should have somehow known we should have set. For instance, we're not using memcached, which it seems might help in general, if not for this issue.

    It has been a few days since I last looked at our activity, so I'll go through that all again tonight. But I'll be very careful to assess whether an "attacker" is actually a logged-in, in-good-standing user this time.

    Leave a comment:

  • Wayne Luke
    Sounds like DDOS. Did you piss anyone off lately? People can buy a denial of Service Attack online. What are the user agents and IP addresses of these visitors?

    Leave a comment:

  • TimothyJMcGowan
    started a topic Millions of page hits. Why?

    Millions of page hits. Why?

    I've cribbed this query from a SiteGround support tech who escalated my problem there. I'm asking it here in the hope that this is a common enough problem that you can spot the solution.

    I am having issues with CPU usage on In the AW stats, 4k unique visits but over 1 million page hits. I tried blocking some of the IPs with the greatest number of hits and ended up blocking our own actual users. There is little evidence of bots doing this, and more importantly, no reason our actual users should be hitting the site this hard.

    What might we have misconfigured?

    For what it's worth, we started as a vBulletin Cloud customer and moved to our own server.

    Copied from sticky topic:
    In order to get your questions answered quickly please provide the following information:
    vBulletin Version: 5.3.4
    PHP Version: 7.1.13
    MySQL Version: Unknown (directions to obtain version are out of date)
    Any Addons installed: Only default in our case: vB Cloud Admin and VigLink
    Does the issue occur in a default style? N/A
    Does the issue occur using the English language provided? N/A
    Error message on the screen: N/A
    Browser and Browser version used. N/A
    Did you clear the browser cache and did the error continue? N/A
    A list of steps that can be used to recreate the issue. N/A
    If the issue is an Invalid Server response, the web server and PHP logs that correspond with its timestamp. N/A

    You can find your MySQL version under Maintenance -> Diagnostics -> Server Variables -> MySQL Settings (Perhaps you can. I cannot.)
    You can find your PHP version in the AdminCP under Maintenance -> View PHPInfo
    You can find your addons in the Admin CP under Hooks & Products -> Manage Products.
widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.