< back to Help & Support

The Feed Caching Mechanism

As soon as you create a RSSbox and add a feed, the Rssinclude system fetches this feed, dumps all unneeded ballast, converts RSS or Atom XML into its own format and stores the result in a temporary cache. Then the system updates the RSS feed cache periodically.

Why is this cache necessary?

RSSinclude.com hosts ten thousands of RSSboxes containing hundred thousands of RSS feeds. Each time an RSSbox is shown on your website our server has to deliver a box with the correct content. If our system used live data it would have to load and convert the feed content from an URL, build the html code of the RSSbox and deliver the html code in one single step - every time the RSSbox gets called.

This would take a lot of time and, of course, a lot of resources. We would need lots of servers and high-speed internet connections. In addition, we would get problems with the RSS feed providers who often limit the access to their feeds.

This is why we needed a fast, efficient and reliable caching mechanism.

How long does it take until the cache is cleared?

The feed updating mechanism runs permanently as a background task of our service. RSS feeds from premium users are scheduled to be updated after approximately half an hour. The updating mechanism tries to update RSS feeds of free users every hour. Consequently, the average cache age of all premium users feeds is lower than the average cache age of the free users feeds.

Please have a look at the current average feed age on our status page.

What if a feed is "stuck" and doesn't update?

Yes, indeed, sometimes it happens that a feed is stuck for several hours or even days. Why is that and how can this be? There are several possible reasons.

  • Problem 1: The feed has been valid before but is not valid anymore. So our system can't use it.
    Solution: Please try to validate your feed here. If it validates your feed is fine, try Problem 2,3 and 4 instead. If is doesn't validate, please try to fix the problem in your feed source code if the feed is your own. If it is not your own feed, please tell the publisher.

  • Problem 2: The feed uses a Pubsubhubbub server that doesn't communicate with our server or does not work at all. Our fallback update script only runs once a day for Pubsubhubbub feeds.
    Solution: Please contact us, don't forget to mention your feed url.

  • Problem 3: Our spider tries to reach your feed but it takes too long (timeout).
    Solution: Please contact us, don't forget to mention your feed url.

  • Problem 4: The server which hosts the feed blocks our spider script.
    Solution: Please contact us, don't forget to mention your feed url.

What can we do?
When you report a feed updating problem we often don't know right away what the exact problem is. We have to monitor the updating activity for your feed over a certain period of time. In most cases this takes a few days to get a picture of what's happening. However, once your feed is tagged as "trouble making" in our database the updating activity is logged. When you encounter a problem again after that we should be able to tell you what the problem is right away.

Can the cache be cleared as soon as I change the RSS feed?

You can clear the feed caches manually. Just go to the box administration and select the "Input Feeds" tab. Below to the feed title you will find an "Update Now" link. As soon as you click this link the feed cache will be cleared and the newest version of the feed will be fetched and stored in the cache.