RSS issue since upgrade to 6.0

[chrisbeveridge]

Hey gang,

I tried searching but didn't see this issue specifically (and 3-letter searches aren't great!) with RSS. On my cloud forum here: https://fandompost.vbulletin.net/index.php

the RSS buttons all work fine and when you open them you see the XML fine. But when I put them into the feed reader over on the dlvr.it service, it says it can't find a feed. I tried it with this forums RSS here at vbulletin to see if it would read these and it does.

(I basically have new posts sent to social media through this RSS feed)

Is there like a reset that needs to be done? I have XML and RSS syndication enabled but not sure what other options need to be set.

[Ston3y1]

I have also had reported problems with the RSS feeds since the upgrade:

The member has three sections they receive feeds for and all three are reporting the same error. They said:

Where we pick up your RSS feeds on the Dreamwidth side, Dreamwidth is basically telling us that BuffyForums doesn't allow Dreamwidth's built-in feed reader to access the feeds anymore. The error message we get is "(Parse error) Error Message: 403 Forbidden".

I can manually open the feed URLs on the BuffyForums side (the xml files) and look at the RSS feed, but it's not being automatically picked up. I wonder if the update could have changed some permissions or disabled access somehow.

ETA: this is an example of where I see the error:
https://buffyforums-main-feed.dreamwidth.org/profile

They've also said that they have another feed reader, Feedly that can pick up the links from the three feeds fine. So they wondered if there's maybe an allowlist that Feedly is on and Dreamwidth isn't?

[Wayne Luke]

This is most likely a side effect of putting filters into place to prevent bots from eating the bandwidth on cloud customers site. You'll need to ask these services what IP Addresses they use and their user agents. Forward that to us in a Support Ticket. Then we can work on "whitelisting" these services.

[Ston3y1]

Thanks, I have put in a request so hopefully they'll provide these.

[chrisbeveridge]

These kinds of companies like dlvr.it would never respond to an email request like that - if they even see requests.

part of the problem, I suspect, is that even feed validation services don't think these are valid feeds.

https://validator.w3.org/feed/check.cgi

[chrisbeveridge]

Just to follow-up for those who have checked (I just put in a support ticket on this), this was the response I got from dlvr.it

I reviewed your account and settings. It appears your servers are blocking dlvr.it from accessing your RSS feeds.

When our systems / servers request the following feed, for example, your servers block the request:

https://www.barefootstudent.com/index/feeds?category_id=2&miles=75&location=41.8781136,-87.6297982&location_search=Chicago,%20IL,%20USA

That means we cannot access your feeds.

You may need to coordinate with your system admin to ensure dlvr.it and our IP address are whitelisted:

https://support.dlvrit.com/hc/en-us/articles/360009186693-What-IP-address-does-dlvr-it-use-for-requesting-content-

At this time, dlvr.it requests content using the following IP address:
54.213.123.74

[Wayne Luke]

WIll forward the IP address to the cloud team so it can be unblocked. That is why I asked for it originally.

[Wayne Luke]

At this time, dlvr.it requests content using the following IP address:
54.213.123.74

This IP has been whitelisted and should be allowed to access your content now.

[chrisbeveridge]

Yup! Everything is working, thanks. My past dealings with other companies in getting their IP addresses to whitelist them have never gone well so I'm impressed dlvr.it did so!

[Ston3y1]

This is most likely a side effect of putting filters into place to prevent bots from eating the bandwidth on cloud customers site. You'll need to ask these services what IP Addresses they use and their user agents. Forward that to us in a Support Ticket. Then we can work on "whitelisting" these services.

I've finally had a response from Dreamwidth about the feeds with them one member was having problems with. They said, "Unfortunately, the system we use for fetching syndicated feeds runs in ECS containers that don't have dedicated IP addresses and change regularly, so we can't give you a list of IPs to allowlist -- you would need to unblock all of ECS." I have no idea about the tech side of this so I don't know if that is possible, should I put that onto a support ticket?

[Wayne Luke]

I don't know if that can be done as we're not specifically blocking particular IP Addresses. IP addresses should only be blocked if there is an abnormal amount of traffic from them. I suspect that a large number of bots are also using ECS (Part of Amazon's Web Services).

[Ston3y1]

I don't know if that can be done as we're not specifically blocking particular IP Addresses. IP addresses should only be blocked if there is an abnormal amount of traffic from them. I suspect that a large number of bots are also using ECS (Part of Amazon's Web Services).

Okay thank you. I'll ask if there is any chance of them running it differently, as obviously we don't want to give greater access to bots. :s

[Ston3y1]

Wayne Luke Apparently Dreamwidth is no longer being blocked for RSS feed updates, so thank you if this is a result of the questions/query being raised by you.

[Wayne Luke]

Any blocking is all automatic based on the number of requests. I had my IP address blocked from accessing all vB Cloud sites last week and had to have it whitelisted.