blogspot feeds update issue fixed

2013-02-28 | Category: General (47)

Short version: We fixed a bug on our update servers that slowed down blogspot feed updates.

Long version including technical details for the ones who are interested:

One of our users (thanks Jennifer!) reported that updating blogspot feeds takes longer than it should. As first step we did what we always do in such cases - enabled detailed logging for the feed. That shows us how often it gets updated, if it was successful, if new entries where fetched and which one of our update server actually did the update.

In this particular case we found that our newer update servers that run inside Amazon AWS (a bunch of Spot Instances) where all failing, while the older ones did not. Resulting in on average only half the usual updates.

As it turns out the real problem behind all this is a bug in pecl_http, the php extension we use for getting the feed contents. The response body is not parsed at all if the result contains a If-Last-Modified date equal the requested If-Modified-Since - although the server responded with a 200 OK and not a 304 Not Modified. And this bug does not show in older versions of the software stack like Debian used on our old update servers. (Yeah, never change a running system :D)

And now this particular bug only shows up because of another one - the etag is not quoted correctly when a weak etag is used.

For the second bug we now have a workaround online that should fix all blogspot feed updates.