Playing with ActivityPub


Mastodon

ActivityPub, WebFinger, and Mastodon are getting some attention because of chaos at Twitter.

It’s anyone’s guess how this all shakes out. As an active user of Twitter, it’ll be sad if it goes away. But in the meantime, let’s have some fun with ActivityPub.

ActivityPub

Under the hood, there’s ActivityPub, WebFinger, and a number of other neat standards like JSON-LD, but for most people, they’re using Mastodon, the application. Mastodon is the software that you sign into and use as a Twitter alternative, and it’d built on all of those standards. There are a few other implementations of social networks based on the same standards, like Frendica, and Pixelfed, but right now, Mastodon is where the people are.

Mastodon is decentralized through federation: users can choose a Mastodon server on which to create an account, and they can follow and interact with users on other servers. You’re relying on someone else to host the server and protect your data, but instead of Twitter, you have a choice of servers. If one Mastodon host crashes, those users will lose their accounts, but other hosts will keep going.

So Mastodon doesn’t offer the sort of serverless decentralization you can get with something more radical like Secure Scuttlebutt, but on the other hand, it’s much more user-friendly. Just like Twitter, you log into a server with a username and a password, and you can easily access it on an iPhone and share content on Mastodon with a link.

But anyway, if we’re going to have this federation system, we might as well take it seriously. One of the benefits of Mastodon is that you can run your own instance. The benefit of Mastodon being built on standards like ActivityPub is that you can interact with Mastodon without running the Mastodon application software in particular: you can build your own. So why not: why not make macwright.com an ActivityPub host?

Context

This blog runs on Jekyll, one of the original static site generators. It’s hosted on Netlify, which has branched out to support a bunch of products, but started out as a static site host.

I’m not going to abandon these systems to support ActivityPub. Jekyll works great for me: I’ve been using it for over a decade and have few complaints. There are spectacular examples of what you can do with custom code and indieweb standards, like Aaron’s site, but that’s not for me.

So, ActivityPub needs to be a simple addition on top of this existing site. What’s the absolute least I’ll need to implement?

I started by reading the ActivityPub specification, and then Mastodon’s documentation of ActivityPub. Right off the bat I had a few takeaways:

And a reference arrived, thanks to Darius Kazemi, perhaps the internet’s most famous bot maker and experimenter. He’s been after this for years, writing ActivityPub servers on Glitch, written guides to ActivityPub, the whole thing.

So, the whole time I was doing this I was looking at express-activitypub, one of Darius’s projects. It’s great - simple, but it works. Most of my work here was making it even simpler - removing some of the configurability and hardcoding things like accounts - and porting code that was dependent on Node.js to code that could run in Netlify’s edge functions, which are a whitelabeled layer on top of Deno and thus use standard web APIs instead.

What needs building

After spelunking in the express-activitypub reference implementation, I eventually ended up with the following extremely minimal ActivityPub essentials, listed nearly in order of difficulty:

With all these together, the photos section of this website is a “user” that you can follow from a Mastodon server: @photos@macwright.com.

WebFinger

Step one is WebFinger. Computer history buffs might remember the finger protocol. This is that, for the web, without the infamous security exploits, hopefully. It’s an endpoint that you can hit to get account information. Mine only supports one user:

https://macwright.com/.well-known/webfinger
  ?resource=acct:photos@macwright.com

So, when you search for @photos@macwright.com from a Mastodon host, this endpoint is what it hits: it extracts macwright.com from the username, assumes that .well-known/webfinger is there on the server, and finds the account. Simple as that. Here’s the code - it’s nothing all that interesting.

User endpoint

This, like WebFinger, was easy to implement. It’s just an endpoint that returns some JSON. Here it is.

Inbox

Here’s where things get a lot more complicated. The /api/inbox function needs to:

So, there’s more complexity in the specific code file (which you can see here) as well as in the system. We need persistence to be an ActivityPub host – we’ll need to store a list of all our subscribers, so that we can send them updates.

This is where it sinks in: ActivityPub is totally different from RSS. Of course it is - this is a federated realtime messaging system. But think about it:

RSS (and Atom) might be the most successful “worse is better” standards of all time, up there with Markdown and JSON. Really S-Tier stuff.

Because with ActivityPub:

So, given the requirements of being an participant with ActivityPub, this is the edge function that uses a database. I’m using PlanetScale, because it’s fun and a good learning experience, but anything would work.

Publishing

So, with the Inbox receiving new followers and recording them in a database, when I publish I’ll need to send messages to those followers.

I publish this site by pushing to GitHub: that’s the setup that Netlify gives me, and what I prefer for deploying overall. It’s a nice setup. It also means that, unlike a WordPress site or a hosted service, there’s no “Publish” button.

So, to publish something, I need to devise a trigger and a way for the publishing script to find new content. Here’s the publishing script I cooked up. Connecting this to Netlify’s webhooks did the trick for a trigger: when the site deploys, it hits the publishing script (which is part of the site) and publishes new updates to followers. It pulls the follower list from the database, pulls posts from the RSS feed, and pushes them.

You might notice - this doesn’t check to see what’s new, it just publishes all the RSS items to all the subscribers. This is because I’ve found that publishing, in ActivityPub, is idempotent: each post has an ID, and if you push that post multiple times, Mastodon servers will check that they already have a post with that ID and ignore it.

Architecture

Flow

So, in the whole loop, this website receives follow requests, stores them in a database, and then sends new posts when I publish something to all of the followers.

My site is still deployed as a static website using Jekyll, but the ActivityPub and WebFinger endpoints are served by Netlify Edge Functions. This, to be, is a pretty good setup: I keep the simplicity and efficiency of static content, only layering in server-like dynamic systems where necessary.

The publishing flow - a webhook that triggers an edge function - is a hack, and something I’ll change if I can figure out a better way to do it.

It works, so far, with my photos page.

Fin

So, how does this make you feel? Excited? Overwhelmed? A little of both?

Hacking on ActivityPub was a fun project, but it was chaotic. ActivityPub in practice is a grab-bag of specifications and implementation-specific details. It was hard to find documentation for a lot of things and hard to debug requests that didn’t have their intended effect on Mastodon.

ActivityPub is a distributed architecture, so it’s going to be a lot more complicated than RSS. People smarter than me rightfully wish that ActivityPub was more sophisticated and more on the side of “better” than worse. And the chattiness of the protocol - the fact that if I have thousands of subscribers I’ll have to send out thousands of updates - that comes with the territory. Just look at how much overhead there is in BitTorrent.

What I built isn’t an ActivityPub system as much as a Mastodon-compatible one. I think this is the key contradiction of the ActivityPub system: it’s a specification broad enough to encompass many different services, but ends up being too general to be useful by itself. There are other specifications like this - things like KML which are technically open and specified but practically defined by what Google Earth supports and produces.

With this frame of mind, the question becomes, if ActivityPub probably isn’t going to be a self-contained standard and instead the basis for one or two popular, homogenous implementations, and if federation is probably going to be a secondary property of those implementations, is the specification technically good enough, useful enough, correct enough, that a future Twitter-competitor will use it? I’m not sure.