Read Archives

Publishers : how to deal with the Superfeedr polling?

Superfeedr is polling your feeds on behalf of other applications.

We know how hard we're sometimes polling your service and we sincerely apologize if this is causing any issue on your end. We believe the web is currently not efficient in the way information is propagated and we're committed to make this better.

We implement a lot of different techniques to avoid creating issues and we try to back-of when we see HTTP error codes, however there are a few things that you can implement on your side to decrease the load on your server.

Please do not hesitate to get in touch with our team if you have questions or need help.

Setup a hub

This is obviously the best thing you can do. PubSubHubbub is an open protocol implemented by a lot of great feed subscribers, including Google, Netvibes, Facebook (Friendfeed), Twitterfeed... etc.

By setting up a hub, you allow all these subscribers to get your content in realtime, without polling.

Superfeedr host hubs and it's free. You may want to look into that direction.

Use Etag and If-Modified-Headers

Setting up a hub involves pinging us. If you do ping us, we will know about updates and we won't need to poll your feeds anymore. If you don't want to ping us, then, you need to implement a way for us to know that your content hasn't changed.

The HTTP specification has 2 similar mechanisms for this : ETag and If-Modified-Since. In both cases, you will serve us a 304 HTTP status code, which basically means: "this content wasn't modified since you last polled it". You don't have to serve a body along with it.

See Also