Cave is Back

Cave is Back

Last spring, we had our first attempt at releasing a feature that a lot of you asked for : cave. The cave is where Superfeedr stores all the entries that we find in all the feeds that we process. It’s not quite easy with the volume that we’re dealing with, but we finally have found a solution that is working well for now.

Retrieving a feed’s entries.

Luckily, we introduced a feed status retrieval API back in September. We are using the exact same API to return any past entry that we have!


It’s as simple as sending an like the following :

Send this :

The server should then reply with something like this :

Check the XMPP doc for more details.

With PubSubHubbub

Just send a GET request to the hub, with hub.mode=retrieve and hub.topic being the url of the feed you’d like to retrieve. Of course, you will need your superfeedr credentials.

Check the PubSubHubbub doc for more details.

This was a long time feature request and we think it’s overdue, but that should help you build apps that do not need to poll feeds at all, as you can just get the historical content of a feed. Also, if you have a doubt about your endpoint being down for a few minutes, you can just get the status of the feeds and compare with what you got.
We will currently return at most 10 entries for each feed. We may have less as we’re still filling in the cave (it will take a few more weeks to have all feeds completely).

Digest notifications

Also, we’re reviving the digest notifications. These are extremely useful for people who want to guarantee that they haven’t missed any entry on a daily basis, and who want to make sure that they’re subscriptions are still active. You can get them with both PubSubHubbub and XMPP.

In a nutshell, we will send you a daily digest of all the feed entries (up to 10). This way, you’re sure that you haven’t missed anything and that your subscription is still active.

Liked this post? Read the archive or

Previously, on the Superfeedr blog: BlameStella and PubSubHubbub.