JSON Path subscriptions

A bunch of our customers use Superfeedr to poll JSON APIs. Rather than wasting their resources, they use us to subscribe to JSON documents using webhooks. When the document eventually updates, we push them the whole document.

Yet, most of the time, they’re not interested in the whole document, just parts of it, and up until today, there was no way to subscribe to parts of it.

Getting too much information

Let’s assume my awesome application displays user profiles from Github. It does not display the whole profile, but only a few interesting fields, including the login name and avatar.

Github has a nice API to retrieve a user’s profile. (some fields removed)

\$ curl https://api.github.com/users/julien51
{
"login": "julien51",
"avatar_url": "https://avatars.githubusercontent.com/u/17735?v=2",
"public_repos": 68,
"public_gists": 114,
"followers": 156,
"following": 127
}

If you subscribe to https://api.github.com/users/julien51 using Superfeedr, you’ll get a new notification for each update to this user profile. You’ll get a notification when the avatar is updated, but you’ll also get a notification when somebody starts to follow me or when I create a gist… even though my awesome application does not really care about that.

JSONPath

We’ve had HTML fragments for a couple years now, today we’re introducing JSON fragments using the exact same principle: append a JSONPath to the fragment part of URL of a JSON document and you’ll get notifications only when this part if the JSON document changes.

A JSONPath is way to select only parts of a JSON document, using ancestry (I want the third child of the books element), or more complex matchers like I want elements whose value of X is Y.

For example above, if you subscribed to https://api.github.com/users/julien51#%24.avatar_url, you’ll be notified only when the avatar_url changes (%24.avatar_url is $.avatar_url urlencoded).

If you’re looking for more information about JSONPath you should read more on Stefan Goessner site and you can test your expressions using this web application.

Never miss a train!

Another great example comes from Pieter Colpaert. irail.be has the times of all trains in Belgium in all stations. When you pick the page of a station, like the one for Gent-Sint-Pieters, you get the list of trains. By fetching this page with a application/json header, you get a JSON representation of the upcoming trains.

{
"@context": {
"delay": "http://semweb.mmlab.be/ns/rplod/delay",
"platform": "http://semweb.mmlab.be/ns/rplod/platform",
"scheduledDepartureTime": "http://semweb.mmlab.be/ns/rplod/scheduledDepartureTime",
"headsign": "http://vocab.org/transit/terms/headsign",
"routeLabel": "http://semweb.mmlab.be/ns/rplod/routeLabel",
"stop": {
"@id": "http://semweb.mmlab.be/ns/rplod/stop",
"@type": "@id"
}
},
"@graph": [
{
"@id": "http://irail.be/stations/NMBS/008892007/departures/20140926095185d62cd764ea09216c5783586c5b90ba",
"delay": "600",
"platform": "7",
"scheduledDepartureTime": "2014-09-26T09:51:00+02:00",
"stop": "http://irail.be/stations/NMBS/008892007",
"headsign": "De Panne",
"routeLabel": "IR 3631"
},
{
"@id": "http://irail.be/stations/NMBS/008892007/departures/201409261001e93879f9830d734825c817aa195c4141",
"delay": "0",
"platform": "9",
"scheduledDepartureTime": "2014-09-26T10:01:00+02:00",
"stop": "http://irail.be/stations/NMBS/008892007",
"headsign": "Knokke",
"routeLabel": "IC 1530"
},
...
{
"@id": "http://irail.be/stations/NMBS/008892007/departures/20140926110453628a1532e8c3f47417df57e60b0dcd",
"delay": "0",
"platform": "7",
"scheduledDepartureTime": "2014-09-26T11:04:00+02:00",
"stop": "http://irail.be/stations/NMBS/008892007",
"headsign": "Tongeren",
"routeLabel": "IC 1510"
}
]
}

Now, I’m not interested in all trains leaving this station, just the trains to Knokke, because it’s friday and I want to enjoy a weekend by the sea. I can use this JSONPath: $..@graph[?(@.headsign=="Knokke")] to get all the children of the @graph element with a headsign of “Knokke”. This would yield something like this:

[{
"@id": "http://irail.be/stations/NMBS/008892007/departures/201409261001e93879f9830d734825c817aa195c4141",
"delay": "0",
"platform": "9",
"scheduledDepartureTime": "2014-09-26T10:01:00+02:00",
"stop": "http://irail.be/stations/NMBS/008892007",
"headsign": "Knokke",
"routeLabel": "IC 1530"
}]

And there, I have the info I need for the next train. It then becomes incredibly more actionable if I use Superfeedr to subscribe to that specific bit of content using this https://irail.be/stations/NMBS/008892007#%24..%40graph%5B%3F(%40.headsign%3D%3D%22Knokke%22)%5D ¹. I now have a webhook for when a new train from Gent-Sint-Pieters to Knokke is about to leave! Also, the payload of the webhook includes all the info I need: the departure time, the platform and any potential delay.