Update frequency restored

2013-06-14 | Category: General (47)

To have as fast as possible output of our boxes we can't fetch the used feed servers while generating the output. Some feed servers take 10sec+ to respond - and you certainly don't want to wait that long for the box.

So we have a server farm running in background periodically checking feeds for updates. At the moment this causes around 30 updates per second. Some of those updates take up to 30 seconds until they time out. So you can imagine this required quite some resources.

And that's why we have spread that load across multiple servers - including AWS Spot Instances - where you basically bid for server instances. Recently the price got too high for us and so the average feed age went up to 60 minutes.

We now ordered a bunch of ordinary (non "cloud") VPS helping with the updates - and we are finally back to 30 minutes!

Check the status page for the current feed age.