@@ -0,0 +1,588 @@ | |||
<!doctype html><!-- This is a valid HTML5 document. --> | |||
<!-- Screen readers, SEO, extensions and so on. --> | |||
<html lang="fr"> | |||
<!-- Has to be within the first 1024 bytes, hence before the `title` element | |||
See: https://www.w3.org/TR/2012/CR-html5-20121217/document-metadata.html#charset --> | |||
<meta charset="utf-8"> | |||
<!-- Why no `X-UA-Compatible` meta: https://stackoverflow.com/a/6771584 --> | |||
<!-- The viewport meta is quite crowded and we are responsible for that. | |||
See: https://codepen.io/tigt/post/meta-viewport-for-2015 --> | |||
<meta name="viewport" content="width=device-width,initial-scale=1"> | |||
<!-- Required to make a valid HTML5 document. --> | |||
<title>Adding ActivityPub to your static site (archive) — David Larlet</title> | |||
<meta name="description" content="Publication mise en cache pour en conserver une trace."> | |||
<!-- That good ol' feed, subscribe :). --> | |||
<link rel="alternate" type="application/atom+xml" title="Feed" href="/david/log/"> | |||
<!-- Generated from https://realfavicongenerator.net/ such a mess. --> | |||
<link rel="apple-touch-icon" sizes="180x180" href="/static/david/icons2/apple-touch-icon.png"> | |||
<link rel="icon" type="image/png" sizes="32x32" href="/static/david/icons2/favicon-32x32.png"> | |||
<link rel="icon" type="image/png" sizes="16x16" href="/static/david/icons2/favicon-16x16.png"> | |||
<link rel="manifest" href="/static/david/icons2/site.webmanifest"> | |||
<link rel="mask-icon" href="/static/david/icons2/safari-pinned-tab.svg" color="#07486c"> | |||
<link rel="shortcut icon" href="/static/david/icons2/favicon.ico"> | |||
<meta name="msapplication-TileColor" content="#f7f7f7"> | |||
<meta name="msapplication-config" content="/static/david/icons2/browserconfig.xml"> | |||
<meta name="theme-color" content="#f7f7f7" media="(prefers-color-scheme: light)"> | |||
<meta name="theme-color" content="#272727" media="(prefers-color-scheme: dark)"> | |||
<!-- Documented, feel free to shoot an email. --> | |||
<link rel="stylesheet" href="/static/david/css/style_2021-01-20.css"> | |||
<!-- See https://www.zachleat.com/web/comprehensive-webfonts/ for the trade-off. --> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_regular.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_bold.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_italic.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t3_regular.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t3_bold.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t3_italic.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
<script> | |||
function toggleTheme(themeName) { | |||
document.documentElement.classList.toggle( | |||
'forced-dark', | |||
themeName === 'dark' | |||
) | |||
document.documentElement.classList.toggle( | |||
'forced-light', | |||
themeName === 'light' | |||
) | |||
} | |||
const selectedTheme = localStorage.getItem('theme') | |||
if (selectedTheme !== 'undefined') { | |||
toggleTheme(selectedTheme) | |||
} | |||
</script> | |||
<meta name="robots" content="noindex, nofollow"> | |||
<meta content="origin-when-cross-origin" name="referrer"> | |||
<!-- Canonical URL for SEO purposes --> | |||
<link rel="canonical" href="https://paul.kinlan.me/adding-activity-pub-to-your-static-site/"> | |||
<body class="remarkdown h1-underline h2-underline h3-underline em-underscore hr-center ul-star pre-tick" data-instant-intensity="viewport-all"> | |||
<article> | |||
<header> | |||
<h1>Adding ActivityPub to your static site</h1> | |||
</header> | |||
<nav> | |||
<p class="center"> | |||
<a href="/david/" title="Aller à l’accueil"><svg class="icon icon-home"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-home"></use> | |||
</svg> Accueil</a> • | |||
<a href="https://paul.kinlan.me/adding-activity-pub-to-your-static-site/" title="Lien vers le contenu original">Source originale</a> | |||
</p> | |||
</nav> | |||
<hr> | |||
<p>My blog is built on Hugo and hosted on Vercel. It mostly works well.</p> | |||
<p>I wanted to have my blog automatically publish posts that I create in a way that I didn't need to spin up an instance of Mastodon.</p> | |||
<p>I got a minimal version of it working. You can discover my page, follow my account, and it will post updates when my blog deploys a new page.</p> | |||
<p>The biggest learning that I had was that ActivityPub is a Message protocol. You can't just output a feed of posts and be done (I tried) - so even if you are a statically generated site you need a Server component because you need to POST message replies to people who send a 'Follow' request to your account and POST to the people who follow your account to 'Create' a 'Note'.</p> | |||
<p>To see it all in action, you can subscribe to my blog on any ActivityPub system by following this account: @paul@paul.kinlan.me < try it.</p> | |||
<p>While I had fun, I will say that I found it <strong>very</strong> hard to get started - I found the spec hard to read; testing was almost impossible (there seem to be no easy test harnesses to determine if you are building a compatible client) so I had to test against a live instance; and there is little documentation of what messages should look like; and I hit snags in all places.</p> | |||
<p>Hopefully this post will help you get started if you want to go down a similar path.</p> | |||
<p>My implementation uses Hugo to create my posts and feed data, Vercel Serverless functions to handle in bound messages, and Firebase Firestore to store the data.</p> | |||
<p>This post will assume that you know the terminology of ActivityPub, but I will try and link to the relevant part of the spec. I also made a lot of assumptions that I am a single user host.</p> | |||
<h3 id="discovery">Discovery</h3> | |||
<p><a href="https://docs.joinmastodon.org/spec/webfinger/">Mastodon uses Web Finger</a> to discover where to look for your servers Actor configuration. WebFinger files are served from a <code>./well-known/webfinger</code> file. I created serverless function which returns the required WebFinger configuration. <a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/api/well-known/webfinger.ts">Code</a></p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-typescript" data-lang="typescript"><span><span><span>import</span> <span>type</span> { <span>VercelRequest</span>, <span>VercelResponse</span> } <span>from</span> <span>'@vercel/node'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>export</span> <span>default</span> <span>function</span> (<span>req</span>: <span>VercelRequest</span>, <span>res</span>: <span>VercelResponse</span>) { | |||
</span></span><span><span> <span>res</span>.<span>statusCode</span> <span>=</span> <span>200</span>; | |||
</span></span><span><span> <span>res</span>.<span>setHeader</span>(<span>"Content-Type"</span>, <span>`application/jrd+json`</span>); | |||
</span></span><span><span> <span>res</span>.<span>end</span>(<span>`{ | |||
</span></span></span><span><span><span> "subject": "acct:paul@paul.kinlan.me", | |||
</span></span></span><span><span><span> "aliases": [ | |||
</span></span></span><span><span><span> "https://status.kinlan.me/@paul" | |||
</span></span></span><span><span><span> ], | |||
</span></span></span><span><span><span> "links": [ | |||
</span></span></span><span><span><span> { | |||
</span></span></span><span><span><span> "rel": "self", | |||
</span></span></span><span><span><span> "type": "application/activity+json", | |||
</span></span></span><span><span><span> "href": "https://paul.kinlan.me/paul" | |||
</span></span></span><span><span><span> } | |||
</span></span></span><span><span><span> ] | |||
</span></span></span><span><span><span> }`</span>); | |||
</span></span><span><span>} | |||
</span></span></code></pre></div> | |||
<p>The JSON above describes a number of aliases for my ActivityPub account '@paul@paul.kinlan.me' and it points to where I host my ActivityPub Actor information.</p> | |||
<p><strong>Note</strong>: You need to make sure you are sending the correct MIME types.</p> | |||
<p>Why do I have a serverless function for static content? Vercel... That's why. I couldn't set the <code>Content-Type</code> configuration properly for any static file in the <code>.well-known</code> folder. In the future if I add multiple accounts I will need to parse the query string to be able to target the <code>links</code> and <code>subject</code> fields correctly/</p> | |||
<p>Next you need to create an <a href="https://www.w3.org/TR/activitypub/#actor-objects">Actor</a>. The Actor is a configuration file that tells ActivityPub servers where to find many core functions such as the 'inbox' (which will receive messages from other clients), 'outbox' that contains all the messages that a user has created (like an RSS feed), 'publicKey' for verifying messages, how my face should appear etc.</p> | |||
<p>To serve the actor file, I just send a JSON response from my <a href="https://github.com/PaulKinlan/paul.kinlan.me/tree/main/api"><code>api</code></a><code>/</code><a href="https://github.com/PaulKinlan/paul.kinlan.me/tree/main/api/activitypub"><code>activitypub</code></a><code>/</code><strong><code>actor.ts</code></strong>. You can see the <a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/api/activitypub/actor.ts">code</a> and the <a href="https://paul.kinlan.me/paul">output</a>.</p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-typescript" data-lang="typescript"><span><span><span>import</span> <span>type</span> { <span>VercelRequest</span>, <span>VercelResponse</span> } <span>from</span> <span>'@vercel/node'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>export</span> <span>default</span> <span>function</span> (<span>req</span>: <span>VercelRequest</span>, <span>res</span>: <span>VercelResponse</span>) { | |||
</span></span><span><span> <span>res</span>.<span>statusCode</span> <span>=</span> <span>200</span>; | |||
</span></span><span><span> <span>res</span>.<span>setHeader</span>(<span>"Content-Type"</span>, <span>`application/activity+json`</span>); | |||
</span></span><span><span> <span>res</span>.<span>json</span>({ | |||
</span></span><span><span> <span>"@context"</span><span>:</span> [<span>"https://www.w3.org/ns/activitystreams"</span>, { <span>"@language"</span><span>:</span> <span>"en- GB"</span> }], | |||
</span></span><span><span> <span>"type"</span><span>:</span> <span>"Person"</span>, | |||
</span></span><span><span> <span>"id"</span><span>:</span> <span>"https://paul.kinlan.me/paul"</span>, | |||
</span></span><span><span> <span>"outbox"</span><span>:</span> <span>"https://paul.kinlan.me/outbox"</span>, | |||
</span></span><span><span> <span>"following"</span><span>:</span> <span>"https://paul.kinlan.me/following"</span>, | |||
</span></span><span><span> <span>"followers"</span><span>:</span> <span>"https://paul.kinlan.me/followers"</span>, | |||
</span></span><span><span> <span>"inbox"</span><span>:</span> <span>"https://paul.kinlan.me/inbox"</span>, | |||
</span></span><span><span> <span>"preferredUsername"</span><span>:</span> <span>"paul"</span>, | |||
</span></span><span><span> <span>"name"</span><span>:</span> <span>"Paul Kinlan - Modern Web Development with Chrome"</span>, | |||
</span></span><span><span> <span>"summary"</span><span>:</span> <span>"Paul is a Developer Advocate for Chrome and the Open Web at Google and loves to help make web development easier."</span>, | |||
</span></span><span><span> <span>"icon"</span><span>:</span> [ | |||
</span></span><span><span> <span>"https://paul.kinlan.me/images/me.png"</span> | |||
</span></span><span><span> ], | |||
</span></span><span><span> <span>"publicKey"</span><span>:</span> { | |||
</span></span><span><span> <span>"@context"</span><span>:</span> <span>"https://w3id.org/security/v1"</span>, | |||
</span></span><span><span> <span>"@type"</span><span>:</span> <span>"Key"</span>, | |||
</span></span><span><span> <span>"id"</span><span>:</span> <span>"https://paul.kinlan.me/paul#main-key"</span>, | |||
</span></span><span><span> <span>"owner"</span><span>:</span> <span>"https://paul.kinlan.me/paul"</span>, | |||
</span></span><span><span> <span>"publicKeyPem"</span><span>:</span> <span>process</span>.<span>env</span>.<span>ACTIVITYPUB_PUBLIC_KEY</span> | |||
</span></span><span><span> } | |||
</span></span><span><span> }); | |||
</span></span><span><span>} | |||
</span></span></code></pre></div> | |||
<p>I used a serverless function because for similar reasons to webfinger (setting the correct Content-type) <em>and</em> I wanted to embed a publicKey that I previously generated and store in Vercel's environment variables configuration.</p> | |||
<p>Now that Mastodon can find me and ActivityPub services know where my inboxes are all I needed to do now was to handle what happens when people follow and unfollow me, and what happens when I create a new post.</p> | |||
<h3 id="following">Following</h3> | |||
<p>I found this one particularly hard - it was almost impossible to find an example of what a Follow message looks like, so I ended up spending a lot of time following my account from a Mastodon client and seeing what data was <code>HTTP</code> <code>POST</code>ed; <strong>and</strong> I also need to maintain the state of who followed me (so I can send them messages later). I chose Firebase Firestore to store all follow requests because it's pretty simple, has a good client and can store JSON directly.</p> | |||
<p>ActivityPub clients will send all messages to an <code>Actor</code>'s inbox. My inbox can only handle <code>Follow</code> and <code>Undo</code> a <code>Follow</code> requests. Once a request is sent to me, I store the data in FireStore and send a response back.</p> | |||
<p>The entire flow is very complex so I will try and explain it as best I can.</p> | |||
<p><a href="https://github.com/PaulKinlan/paul.kinlan.me/tree/main/api">api</a>/<a href="https://github.com/PaulKinlan/paul.kinlan.me/tree/main/api/activitypub">activitypub</a>/<strong>inbox.ts</strong></p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-typescript" data-lang="typescript"><span><span><span>import</span> <span>type</span> { <span>VercelRequest</span>, <span>VercelResponse</span> } <span>from</span> <span>'@vercel/node'</span>; | |||
</span></span><span><span><span>import</span> { <span>AP</span> } <span>from</span> <span>'activitypub-core-types'</span>; | |||
</span></span><span><span><span>import</span> <span>type</span> { <span>Readable</span> } <span>from</span> <span>'node:stream'</span>; | |||
</span></span><span><span><span>import</span> <span>*</span> <span>as</span> <span>admin</span> <span>from</span> <span>'firebase-admin'</span>; | |||
</span></span><span><span><span>import</span> { <span>v4</span> <span>as</span> <span>uuid</span> } <span>from</span> <span>'uuid'</span>; | |||
</span></span><span><span><span>import</span> { <span>CoreObject</span>, <span>Entity</span> } <span>from</span> <span>'activitypub-core-types/lib/activitypub/index'</span>; | |||
</span></span><span><span><span>import</span> { <span>sendSignedRequest</span> } <span>from</span> <span>'../../lib/activitypub/sendSignedRequest'</span>; | |||
</span></span><span><span><span>import</span> { <span>parseSignature</span> } <span>from</span> <span>'../../lib/activitypub/utils/parseSignature'</span>; | |||
</span></span><span><span><span>import</span> { <span>fetchActorInformation</span> } <span>from</span> <span>'../../lib/activitypub/utils/fetchActorInformation'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>process</span>.<span>env</span>.<span>NODE_TLS_REJECT_UNAUTHORIZED</span> <span>=</span> <span>'0'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>if</span> (<span>!</span><span>admin</span>.<span>apps</span>.<span>length</span>) { | |||
</span></span><span><span> <span>admin</span>.<span>initializeApp</span>({ | |||
</span></span><span><span> <span>credential</span>: <span>admin.credential.cert</span>({ | |||
</span></span><span><span> <span>projectId</span>: <span>process.env.NEXT_PUBLIC_FIREBASE_PROJECT_ID</span>, | |||
</span></span><span><span> <span>clientEmail</span>: <span>process.env.FIREBASE_CLIENT_EMAIL</span>, | |||
</span></span><span><span> <span>privateKey</span>: <span>process.env.FIREBASE_PRIVATE_KEY.replace</span>(<span>/\\n/g</span>, <span>'\n'</span>) | |||
</span></span><span><span> }) | |||
</span></span><span><span> }); | |||
</span></span><span><span>} | |||
</span></span><span><span> | |||
</span></span><span><span><span>const</span> <span>db</span> <span>=</span> <span>admin</span>.<span>firestore</span>(); | |||
</span></span><span><span> | |||
</span></span><span><span><span>export</span> <span>const</span> <span>config</span> <span>=</span> { | |||
</span></span><span><span> <span>api</span><span>:</span> { | |||
</span></span><span><span> <span>bodyParser</span>: <span>false</span>, | |||
</span></span><span><span> }, | |||
</span></span><span><span>}; | |||
</span></span><span><span> | |||
</span></span><span><span><span>async</span> <span>function</span> <span>buffer</span>(<span>readable</span>: <span>Readable</span>) { | |||
</span></span><span><span> <span>const</span> <span>chunks</span> <span>=</span> []; | |||
</span></span><span><span> <span>for</span> <span>await</span> (<span>const</span> <span>chunk</span> <span>of</span> <span>readable</span>) { | |||
</span></span><span><span> <span>chunks</span>.<span>push</span>(<span>typeof</span> <span>chunk</span> <span>===</span> <span>'string'</span> <span>?</span> <span>Buffer</span>.<span>from</span>(<span>chunk</span>) <span>:</span> <span>chunk</span>); | |||
</span></span><span><span> } | |||
</span></span><span><span> <span>return</span> <span>Buffer</span>.<span>concat</span>(<span>chunks</span>); | |||
</span></span><span><span>} | |||
</span></span><span><span> | |||
</span></span><span><span><span>function</span> <span>verifySignature</span>(<span>signature</span>, <span>publicKeyJson</span>) { | |||
</span></span><span><span> <span>let</span> <span>signatureValid</span>; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>try</span> { | |||
</span></span><span><span> <span>// Verify the signature | |||
</span></span></span><span><span><span></span> <span>signatureValid</span> <span>=</span> <span>signature</span>.<span>verify</span>( | |||
</span></span><span><span> <span>publicKeyJson</span>.<span>publicKeyPem</span>, <span>// The PEM string from the public key object | |||
</span></span></span><span><span><span></span> ); | |||
</span></span><span><span> } <span>catch</span> (<span>error</span>) { | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>"Signature Verification error"</span>, <span>error</span>) | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>return</span> <span>signatureValid</span>; | |||
</span></span><span><span>} | |||
</span></span><span><span> | |||
</span></span><span><span><span>export</span> <span>default</span> <span>async</span> <span>function</span> (<span>req</span>: <span>VercelRequest</span>, <span>res</span>: <span>VercelResponse</span>) { | |||
</span></span><span><span> <span>const</span> { <span>body</span>, <span>query</span>, <span>method</span>, <span>url</span>, <span>headers</span> } <span>=</span> <span>req</span>; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>res</span>.<span>statusCode</span> <span>=</span> <span>200</span>; | |||
</span></span><span><span> <span>res</span>.<span>setHeader</span>(<span>"Content-Type"</span>, <span>`application/activity+json`</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>// Verify the message some how. | |||
</span></span></span><span><span><span></span> <span>const</span> <span>buf</span> <span>=</span> <span>await</span> <span>buffer</span>(<span>req</span>); | |||
</span></span><span><span> <span>const</span> <span>rawBody</span> <span>=</span> <span>buf</span>.<span>toString</span>(<span>'utf8'</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>message</span> <span>=</span> <<span>AP.Activity</span>><span>JSON</span>.<span>parse</span>(<span>rawBody</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>message</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>signature</span> <span>=</span> <span>parseSignature</span>(<span>req</span>); | |||
</span></span><span><span> <span>const</span> <span>actorInformation</span> <span>=</span> <span>await</span> <span>fetchActorInformation</span>(<span>signature</span>.<span>keyId</span>); | |||
</span></span><span><span> <span>const</span> <span>signatureValid</span> <span>=</span> <span>verifySignature</span>(<span>signature</span>, <span>actorInformation</span>.<span>publicKey</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>signatureValid</span> <span>==</span> <span>null</span> <span>||</span> <span>signatureValid</span> <span>==</span> <span>false</span>) { | |||
</span></span><span><span> <span>res</span>.<span>end</span>(<span>'invalid signature'</span>); | |||
</span></span><span><span> <span>return</span>; | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>// We should check the digest. | |||
</span></span></span><span><span><span></span> <span>if</span> (<span>message</span>.<span>type</span> <span>==</span> <span>"Follow"</span>) { | |||
</span></span><span><span> <span>// We are following. | |||
</span></span></span><span><span><span></span> <span>const</span> <span>followMessage</span>: <span>AP.Follow</span> <span>=</span> <<span>AP.Follow</span>><span>message</span>; | |||
</span></span><span><span> <span>if</span> (<span>followMessage</span>.<span>id</span> <span>==</span> <span>null</span>) <span>return</span>; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>collection</span> <span>=</span> <span>db</span>.<span>collection</span>(<span>'followers'</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>actorID</span> <span>=</span> (<<span>URL</span>><span>followMessage</span>.<span>actor</span>).<span>toString</span>(); | |||
</span></span><span><span> <span>const</span> <span>followDocRef</span> <span>=</span> <span>collection</span>.<span>doc</span>(<span>actorID</span>.<span>replace</span>(<span>/\//g</span>, <span>"_"</span>)); | |||
</span></span><span><span> <span>const</span> <span>followDoc</span> <span>=</span> <span>await</span> <span>followDocRef</span>.<span>get</span>(); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>followDoc</span>.<span>exists</span>) { | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>"Already Following"</span>) | |||
</span></span><span><span> <span>return</span> <span>res</span>.<span>end</span>(<span>'already following'</span>); | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>// Create the follow; | |||
</span></span></span><span><span><span></span> <span>await</span> <span>followDocRef</span>.<span>set</span>(<span>followMessage</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>guid</span> <span>=</span> <span>uuid</span>(); | |||
</span></span><span><span> <span>const</span> <span>domain</span> <span>=</span> <span>'paul.kinlan.me'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>acceptRequest</span>: <span>AP.Accept</span> <span>=</span> <<span>AP.Accept</span>>{ | |||
</span></span><span><span> <span>"@context"</span><span>:</span> <span>"https://www.w3.org/ns/activitystreams"</span>, | |||
</span></span><span><span> <span>'id'</span><span>:</span> <span>new</span> <span>URL</span>(<span>`https://</span><span>${</span><span>domain</span><span>}</span><span>/</span><span>${</span><span>guid</span><span>}</span><span>`</span>), | |||
</span></span><span><span> <span>'type'</span><span>:</span> <span>'Accept'</span>, | |||
</span></span><span><span> <span>'actor'</span><span>:</span> <span>"https://paul.kinlan.me/paul"</span>, | |||
</span></span><span><span> <span>'object'</span><span>:</span> <span>followMessage</span> | |||
</span></span><span><span> }; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>actorInbox</span> <span>=</span> <span>new</span> <span>URL</span>(<span>actorInformation</span>.<span>inbox</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>response</span> <span>=</span> <span>await</span> <span>sendSignedRequest</span>(<span>actorInbox</span>, <span>acceptRequest</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>"Following result"</span>, <span>response</span>.<span>status</span>, <span>response</span>.<span>statusText</span>, <span>await</span> <span>response</span>.<span>text</span>()); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>return</span> <span>res</span>.<span>end</span>(<span>"ok"</span>) | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>message</span>.<span>type</span> <span>==</span> <span>"Undo"</span>) { | |||
</span></span><span><span> <span>// Undo a follow. | |||
</span></span></span><span><span><span></span> <span>const</span> <span>undoObject</span>: <span>AP.Undo</span> <span>=</span> <<span>AP.Undo</span>><span>message</span>; | |||
</span></span><span><span> <span>if</span> (<span>undoObject</span> <span>==</span> <span>null</span> <span>||</span> <span>undoObject</span>.<span>id</span> <span>==</span> <span>null</span>) <span>return</span>; | |||
</span></span><span><span> <span>if</span> (<span>undoObject</span>.<span>object</span> <span>==</span> <span>null</span>) <span>return</span>; | |||
</span></span><span><span> <span>if</span> (<span>"actor"</span> <span>in</span> <span>undoObject</span>.<span>object</span> <span>==</span> <span>false</span> <span>&&</span> (<<span>CoreObject</span>><span>undoObject</span>.<span>object</span>).<span>type</span> <span>!=</span> <span>"Follow"</span>) <span>return</span>; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>docId</span> <span>=</span> <span>undoObject</span>.<span>actor</span>.<span>toString</span>().<span>replace</span>(<span>/\//g</span>, <span>"_"</span>); | |||
</span></span><span><span> <span>const</span> <span>res</span> <span>=</span> <span>await</span> <span>db</span>.<span>collection</span>(<span>'followers'</span>).<span>doc</span>(<span>docId</span>).<span>delete</span>(); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>"Deleted"</span>, <span>res</span>) | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>res</span>.<span>end</span>(); | |||
</span></span><span><span>}; | |||
</span></span></code></pre></div> | |||
<ol> | |||
<li>Parse the <code>POST</code> body and cast it to an Activity object.</li> | |||
<li>Parse the signature of the request to verify the message hasn't been tampered with in transit.</li> | |||
<li>From the signature HTTP header get the <code>Actor</code> that wants to follow you and <a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/lib/activitypub/utils/fetchActorInformation.ts">fetch their Public Key </a>(from their Actor file).</li> | |||
<li>Verify the message with their Public Key</li> | |||
</ol> | |||
<p>Now we believe that we have a valid messages.</p> | |||
<p>If the message is a <code>Follow</code> request</p> | |||
<ol> | |||
<li>See if the Actor trying to follow is already in the db, if they are return;</li> | |||
<li>Add the <code>Actor</code> to the <code>followers</code> collection in FireStore</li> | |||
<li><a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/api/activitypub/inbox.ts#L100">Prepare</a> an <code>Accept</code> message to the <code>Actor</code> indicating that the Follow has been accepted and <a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/lib/activitypub/utils/sendSignedRequest.ts">send it</a>.</li> | |||
</ol> | |||
<p>If the message is an <code>Undo</code> for a <code>Follow</code> request.</p> | |||
<ol> | |||
<li>Find the data in the <code>followers</code> collection in FireStore</li> | |||
<li>Delete it.</li> | |||
</ol> | |||
<p><strong>Note</strong>: I found it hard to find much information about sending requests to servers - so after a lot of reading and experimenting I created this <a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/lib/activitypub/utils/sendSignedRequest.ts">routine</a>. It will successfully sign the HTTP request with your configured private key and attach a digest.</p> | |||
<h3 id="posting">Posting</h3> | |||
<p>Like many static sites there is no CMS that knows when new content is posted (it is static after all) so I needed to create a routine that would send my posts to all the people that follow the account.</p> | |||
<p>Firstly I generate the <code>outbox</code> so that people can read all my public posts. I use a hugo template (<a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/config.toml">layouts/index.activity_outbox.ajson</a>) that reads through all my posts and creates a <code>Create</code> object with an embedded <code>Note</code> - this is what Mastodon needs to show a Toot.</p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-go" data-lang="go"><span><span>{{<span>-</span> <span>$</span><span>pctx</span> <span>:=</span> . <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>if</span> .<span>IsHome</span> <span>-</span>}}{{ <span>$</span><span>pctx</span> = .<span>Site</span> }}{{<span>-</span> <span>end</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>$</span><span>pages</span> <span>:=</span> <span>slice</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>if</span> <span>or</span> <span>$</span>.<span>IsHome</span> <span>$</span>.<span>IsSection</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>$</span><span>pages</span> = <span>$</span><span>pctx</span>.<span>RegularPages</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>else</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>$</span><span>pages</span> = <span>$</span><span>pctx</span>.<span>Pages</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>end</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>$</span><span>limit</span> <span>:=</span> .<span>Site</span>.<span>Config</span>.<span>Services</span>.<span>RSS</span>.<span>Limit</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>if</span> <span>ge</span> <span>$</span><span>limit</span> <span>1</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>$</span><span>pages</span> = <span>$</span><span>pages</span> | <span>first</span> <span>$</span><span>limit</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>end</span> <span>-</span>}} | |||
</span></span><span><span>{ | |||
</span></span><span><span> <span>"@context"</span>: <span>"https://www.w3.org/ns/activitystreams"</span>, | |||
</span></span><span><span> <span>"id"</span>: <span>"{{ $.Site.BaseURL }}outbox"</span>, | |||
</span></span><span><span> <span>"summary"</span>: <span>"{{$.Site.Author.name}} - {{$.Site.Title}}"</span>, | |||
</span></span><span><span> <span>"type"</span>: <span>"OrderedCollection"</span>, | |||
</span></span><span><span> {{ <span>$</span><span>notdrafts</span> <span>:=</span> <span>where</span> <span>$</span><span>pages</span> <span>".Draft"</span> <span>"!="</span> <span>true</span> }} | |||
</span></span><span><span> {{ <span>$</span><span>all</span> <span>:=</span> <span>where</span> <span>$</span><span>notdrafts</span> <span>"Type"</span> <span>"in"</span> (<span>slice</span> <span>"journal"</span> <span>"post"</span> <span>"page"</span>)}} | |||
</span></span><span><span> <span>"totalItems"</span>: {{(<span>len</span> <span>$</span><span>all</span>)}}, | |||
</span></span><span><span> <span>"orderedItems"</span>: [ | |||
</span></span><span><span> {{ <span>range</span> <span>$</span><span>index</span>, <span>$</span><span>element</span> <span>:=</span> <span>$</span><span>all</span> }} | |||
</span></span><span><span> {{<span>-</span> <span>if</span> <span>ne</span> <span>$</span><span>index</span> <span>0</span> }}, {{ <span>end</span> }} | |||
</span></span><span><span> { | |||
</span></span><span><span> <span>"@context"</span>: <span>"https://www.w3.org/ns/activitystreams"</span>, | |||
</span></span><span><span> <span>"id"</span>: <span>"{{.Permalink}}-create"</span>, | |||
</span></span><span><span> <span>"type"</span>: <span>"Create"</span>, | |||
</span></span><span><span> <span>"actor"</span>: <span>"https://paul.kinlan.me/paul"</span>, | |||
</span></span><span><span> <span>"object"</span>: { | |||
</span></span><span><span> <span>"id"</span>: <span>"{{ .Permalink }}"</span>, | |||
</span></span><span><span> <span>"type"</span>: <span>"Note"</span>, | |||
</span></span><span><span> <span>"content"</span>: <span>"{{.Title}}<br>{{.Summary}}"</span>, | |||
</span></span><span><span> <span>"url"</span>: <span>"{{.Permalink}}"</span>, | |||
</span></span><span><span> <span>"attributedTo"</span>: <span>"https://paul.kinlan.me/paul"</span>, | |||
</span></span><span><span> <span>"to"</span>: <span>"https://www.w3.org/ns/activitystreams#Public"</span>, | |||
</span></span><span><span> <span>"published"</span>: {{ <span>dateFormat</span> <span>"2006-01-02T15:04:05-07:00"</span> .<span>Date</span> | <span>jsonify</span> }} | |||
</span></span><span><span> } | |||
</span></span><span><span> } | |||
</span></span><span><span> {{<span>end</span>}} | |||
</span></span><span><span> ] | |||
</span></span><span><span>} | |||
</span></span></code></pre></div> | |||
<p>I also set up Hugo to generate this file for the "home" output type as follows</p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-toml" data-lang="toml"><span><span>[<span>mediaTypes</span>] | |||
</span></span><span><span>[<span>mediaTypes</span>.<span>"application/activity+json"</span>] | |||
</span></span><span><span><span>suffixes</span> = [<span>"ajson"</span>] | |||
</span></span><span><span> | |||
</span></span><span><span>[<span>outputFormats</span>] | |||
</span></span><span><span>[<span>outputFormats</span>.<span>ACTIVITY_OUTBOX</span>] | |||
</span></span><span><span><span>mediaType</span> = <span>"application/activity+json"</span> | |||
</span></span><span><span><span>notAlternative</span> = <span>true</span> | |||
</span></span><span><span><span>baseName</span> = <span>"outbox"</span> | |||
</span></span><span><span> | |||
</span></span><span><span>[<span>outputs</span>] | |||
</span></span><span><span><span>home</span> = [<span>"HTML"</span>, <span>"RSS"</span>, <span>"ACTIVITY_OUTBOX"</span>] | |||
</span></span></code></pre></div> | |||
<p>I then serve the file:<a href="https://github.com/PaulKinlan/paul.kinlan.me/tree/main/api/activitypub"> /api/activitypub/outbox.ts</a></p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-typescript" data-lang="typescript"><span><span><span>import</span> <span>type</span> { <span>VercelRequest</span>, <span>VercelResponse</span> } <span>from</span> <span>'@vercel/node'</span>; | |||
</span></span><span><span><span>import</span> { <span>join</span> } <span>from</span> <span>'path'</span>; | |||
</span></span><span><span><span>import</span> { <span>cwd</span> } <span>from</span> <span>'process'</span>; | |||
</span></span><span><span><span>import</span> { <span>readFileSync</span> } <span>from</span> <span>'fs'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>/* | |||
</span></span></span><span><span><span> This returns a list of posts for the single user 'Paul'. | |||
</span></span></span><span><span><span> It's a GET request. This doesn't post it to anyone's timeline. | |||
</span></span></span><span><span><span>*/</span> | |||
</span></span><span><span><span>export</span> <span>default</span> <span>function</span> (<span>req</span>: <span>VercelRequest</span>, <span>res</span>: <span>VercelResponse</span>) { | |||
</span></span><span><span> <span>// All of the outbox data is generated at build time, so just return that static file. | |||
</span></span></span><span><span><span></span> <span>const</span> <span>file</span> <span>=</span> <span>join</span>(<span>cwd</span>(), <span>'public'</span>, <span>'outbox.ajson'</span>); | |||
</span></span><span><span> <span>const</span> <span>stringified</span> <span>=</span> <span>readFileSync</span>(<span>file</span>, <span>'utf8'</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>res</span>.<span>statusCode</span> <span>=</span> <span>200</span>; | |||
</span></span><span><span> <span>res</span>.<span>setHeader</span>(<span>"Content-Type"</span>, <span>`application/activity+json`</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>return</span> <span>res</span>.<span>end</span>(<span>stringified</span>); | |||
</span></span><span><span>}; | |||
</span></span></code></pre></div> | |||
<p>Finally, when my Vercel build completes, I scan the generated <a href="https://paul.kinlan.me/outbox">outbox</a> using my <a href="https://paul.kinlan.me/post-deploy-webhook-for-vercel/">post-deploy Webhook for vercel</a> and calling <a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/api/activitypub/sendNote.ts">api/activitypub/sendNote.ts</a> endpoint to post to all the followers.</p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-typescript" data-lang="typescript"><span><span><span>import</span> <span>type</span> { <span>VercelRequest</span>, <span>VercelResponse</span> } <span>from</span> <span>'@vercel/node'</span>; | |||
</span></span><span><span><span>import</span> { <span>AP</span> } <span>from</span> <span>'activitypub-core-types'</span>; | |||
</span></span><span><span><span>import</span> <span>*</span> <span>as</span> <span>admin</span> <span>from</span> <span>'firebase-admin'</span>; | |||
</span></span><span><span><span>import</span> { <span>OrderedCollection</span> } <span>from</span> <span>'activitypub-core-types/lib/activitypub/index'</span>; | |||
</span></span><span><span><span>import</span> { <span>sendSignedRequest</span> } <span>from</span> <span>'../../lib/activitypub/utils/sendSignedRequest'</span>; | |||
</span></span><span><span><span>import</span> { <span>fetchActorInformation</span> } <span>from</span> <span>'../../lib/activitypub/utils/fetchActorInformation'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>process</span>.<span>env</span>.<span>NODE_TLS_REJECT_UNAUTHORIZED</span> <span>=</span> <span>'0'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>if</span> (<span>!</span><span>admin</span>.<span>apps</span>.<span>length</span>) { | |||
</span></span><span><span> <span>admin</span>.<span>initializeApp</span>({ | |||
</span></span><span><span> <span>credential</span>: <span>admin.credential.cert</span>({ | |||
</span></span><span><span> <span>projectId</span>: <span>process.env.NEXT_PUBLIC_FIREBASE_PROJECT_ID</span>, | |||
</span></span><span><span> <span>clientEmail</span>: <span>process.env.FIREBASE_CLIENT_EMAIL</span>, | |||
</span></span><span><span> <span>privateKey</span>: <span>process.env.FIREBASE_PRIVATE_KEY.replace</span>(<span>/\\n/g</span>, <span>'\n'</span>) | |||
</span></span><span><span> }) | |||
</span></span><span><span> }); | |||
</span></span><span><span>} | |||
</span></span><span><span> | |||
</span></span><span><span><span>const</span> <span>db</span> <span>=</span> <span>admin</span>.<span>firestore</span>(); | |||
</span></span><span><span> | |||
</span></span><span><span><span>export</span> <span>const</span> <span>config</span> <span>=</span> { | |||
</span></span><span><span> <span>api</span><span>:</span> { | |||
</span></span><span><span> <span>bodyParser</span>: <span>false</span> | |||
</span></span><span><span> } | |||
</span></span><span><span>}; | |||
</span></span><span><span> | |||
</span></span><span><span><span>/* | |||
</span></span></span><span><span><span> Sends the latest not that hasn't yet been sent. | |||
</span></span></span><span><span><span>*/</span> | |||
</span></span><span><span><span>export</span> <span>default</span> <span>async</span> <span>function</span> (<span>req</span>: <span>VercelRequest</span>, <span>res</span>: <span>VercelResponse</span>) { | |||
</span></span><span><span> <span>const</span> { <span>body</span>, <span>query</span>, <span>method</span>, <span>url</span>, <span>headers</span> } <span>=</span> <span>req</span>; | |||
</span></span><span><span> <span>const</span> { <span>token</span> } <span>=</span> <span>query</span>; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>method</span> <span>!=</span> <span>"POST"</span>) { | |||
</span></span><span><span> <span>res</span>.<span>status</span>(<span>401</span>).<span>end</span>(<span>"Invalid Method, must be POST"</span>); | |||
</span></span><span><span> <span>return</span>; | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>token</span> <span>!=</span> <span>process</span>.<span>env</span>.<span>ACTIVITYPUB_CREATE_TOKEN</span>) { | |||
</span></span><span><span> <span>res</span>.<span>status</span>(<span>401</span>).<span>end</span>(<span>"Invalid token"</span>); | |||
</span></span><span><span> <span>return</span>; | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>configCollection</span> <span>=</span> <span>db</span>.<span>collection</span>(<span>'config'</span>); | |||
</span></span><span><span> <span>const</span> <span>configRef</span> <span>=</span> <span>configCollection</span>.<span>doc</span>(<span>"config"</span>); | |||
</span></span><span><span> <span>const</span> <span>config</span> <span>=</span> <span>await</span> <span>configRef</span>.<span>get</span>(); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>config</span>.<span>exists</span> <span>==</span> <span>false</span>) { | |||
</span></span><span><span> <span>// Config doesn't exist, make something | |||
</span></span></span><span><span><span></span> <span>configRef</span>.<span>set</span>({ | |||
</span></span><span><span> <span>"lastId"</span><span>:</span> <span>0</span> | |||
</span></span><span><span> }); | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>configData</span> <span>=</span> <span>config</span>.<span>data</span>(); | |||
</span></span><span><span> <span>let</span> <span>lastId</span> <span>=</span> <span>0</span>; | |||
</span></span><span><span> <span>if</span> (<span>configData</span> <span>!=</span> <span>undefined</span>) { | |||
</span></span><span><span> <span>lastId</span> <span>=</span> <span>configData</span>.<span>lastId</span>; | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>// Get my outbox because it contains all my notes. | |||
</span></span></span><span><span><span></span> <span>const</span> <span>outboxResponse</span> <span>=</span> <span>await</span> <span>fetch</span>(<span>'https://paul.kinlan.me/outbox'</span>); | |||
</span></span><span><span> <span>const</span> <span>outbox</span> <span>=</span> <<span>OrderedCollection</span>>(<span>await</span> <span>outboxResponse</span>.<span>json</span>()); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>followersCollection</span> <span>=</span> <span>db</span>.<span>collection</span>(<span>'followers'</span>); | |||
</span></span><span><span> <span>const</span> <span>followersQuerySnapshot</span> <span>=</span> <span>await</span> <span>followersCollection</span>.<span>get</span>(); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>for</span> (<span>const</span> <span>followerDoc</span> <span>of</span> <span>followersQuerySnapshot</span>.<span>docs</span>) { | |||
</span></span><span><span> <span>const</span> <span>follower</span> <span>=</span> <span>followerDoc</span>.<span>data</span>(); | |||
</span></span><span><span> <span>try</span> { | |||
</span></span><span><span> <span>const</span> <span>actorInformation</span> <span>=</span> <span>await</span> <span>fetchActorInformation</span>(<span>follower</span>.<span>actor</span>); | |||
</span></span><span><span> <span>const</span> <span>actorInbox</span> <span>=</span> <span>new</span> <span>URL</span>(<span>actorInformation</span>.<span>inbox</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>for</span> (<span>const</span> <span>iteIdx</span> <span>in</span> (<<span>AP.EntityReference</span><span>[]</span>><span>outbox</span>.<span>orderedItems</span>)) { | |||
</span></span><span><span> <span>// We have to break somewhere... do it after the first. | |||
</span></span></span><span><span><span></span> <span>const</span> <span>item</span> <span>=</span> (<<span>AP.EntityReference</span><span>[]</span>><span>outbox</span>.<span>orderedItems</span>)[<span>iteIdx</span>]; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>item</span>.<span>object</span> <span>!=</span> <span>undefined</span>) { | |||
</span></span><span><span> <span>// We might not need this. | |||
</span></span></span><span><span><span></span> <span>item</span>.<span>object</span>.<span>published</span> <span>=</span> (<span>new</span> Date()).<span>toISOString</span>(); | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>`Sending to </span><span>${</span><span>actorInbox</span><span>}</span><span>`</span>, <span>item</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>// Item will be an entity, i.e, { Create { Note } } | |||
</span></span></span><span><span><span></span> <span>const</span> <span>response</span> <span>=</span> <span>await</span> <span>sendSignedRequest</span>(<span>actorInbox</span>, <<span>AP.Activity</span>> <span>item</span>); | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>"Send result: "</span>, <span>actorInbox</span>, <span>response</span>.<span>status</span>, <span>response</span>.<span>statusText</span>, <span>await</span> <span>response</span>.<span>text</span>()); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>break</span>; | |||
</span></span><span><span> } | |||
</span></span><span><span> } <span>catch</span> (<span>ex</span>) { | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>"Error"</span>, <span>ex</span>, <span>follower</span>); | |||
</span></span><span><span> } | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>res</span>.<span>status</span>(<span>200</span>).<span>end</span>(<span>"ok"</span>); | |||
</span></span><span><span>}; | |||
</span></span></code></pre></div> | |||
<p>The above code is relative long but the summary of it is as follows:</p> | |||
<ol> | |||
<li>Scan the outbox</li> | |||
<li>Pick the first post (I am only sending one note)</li> | |||
<li>For each follower in the <code>followers</code> table | |||
<ol> | |||
<li>Get their actor information (where their inbox is)</li> | |||
<li>Send the <code>Create</code> object from the outbox to them via a signed HTTP request</li> | |||
</ol> | |||
</li> | |||
</ol> | |||
<h3 id="voila">Voila</h3> | |||
<p>Simple... Nah. I think it's pretty complex, but it works.</p> | |||
<p>If you have created something similar, send me a comment. I'd love to improve what I have and share that with more people.</p> | |||
</article> | |||
<hr> | |||
<footer> | |||
<p> | |||
<a href="/david/" title="Aller à l’accueil"><svg class="icon icon-home"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-home"></use> | |||
</svg> Accueil</a> • | |||
<a href="/david/log/" title="Accès au flux RSS"><svg class="icon icon-rss2"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-rss2"></use> | |||
</svg> Suivre</a> • | |||
<a href="http://larlet.com" title="Go to my English profile" data-instant><svg class="icon icon-user-tie"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-user-tie"></use> | |||
</svg> Pro</a> • | |||
<a href="mailto:david%40larlet.fr" title="Envoyer un courriel"><svg class="icon icon-mail"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-mail"></use> | |||
</svg> Email</a> • | |||
<abbr class="nowrap" title="Hébergeur : Alwaysdata, 62 rue Tiquetonne 75002 Paris, +33184162340"><svg class="icon icon-hammer2"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-hammer2"></use> | |||
</svg> Légal</abbr> | |||
</p> | |||
<template id="theme-selector"> | |||
<form> | |||
<fieldset> | |||
<legend><svg class="icon icon-brightness-contrast"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-brightness-contrast"></use> | |||
</svg> Thème</legend> | |||
<label> | |||
<input type="radio" value="auto" name="chosen-color-scheme" checked> Auto | |||
</label> | |||
<label> | |||
<input type="radio" value="dark" name="chosen-color-scheme"> Foncé | |||
</label> | |||
<label> | |||
<input type="radio" value="light" name="chosen-color-scheme"> Clair | |||
</label> | |||
</fieldset> | |||
</form> | |||
</template> | |||
</footer> | |||
<script src="/static/david/js/instantpage-5.1.0.min.js" type="module"></script> | |||
<script> | |||
function loadThemeForm(templateName) { | |||
const themeSelectorTemplate = document.querySelector(templateName) | |||
const form = themeSelectorTemplate.content.firstElementChild | |||
themeSelectorTemplate.replaceWith(form) | |||
form.addEventListener('change', (e) => { | |||
const chosenColorScheme = e.target.value | |||
localStorage.setItem('theme', chosenColorScheme) | |||
toggleTheme(chosenColorScheme) | |||
}) | |||
const selectedTheme = localStorage.getItem('theme') | |||
if (selectedTheme && selectedTheme !== 'undefined') { | |||
form.querySelector(`[value="${selectedTheme}"]`).checked = true | |||
} | |||
} | |||
const prefersColorSchemeDark = '(prefers-color-scheme: dark)' | |||
window.addEventListener('load', () => { | |||
let hasDarkRules = false | |||
for (const styleSheet of Array.from(document.styleSheets)) { | |||
let mediaRules = [] | |||
for (const cssRule of styleSheet.cssRules) { | |||
if (cssRule.type !== CSSRule.MEDIA_RULE) { | |||
continue | |||
} | |||
// WARNING: Safari does not have/supports `conditionText`. | |||
if (cssRule.conditionText) { | |||
if (cssRule.conditionText !== prefersColorSchemeDark) { | |||
continue | |||
} | |||
} else { | |||
if (cssRule.cssText.startsWith(prefersColorSchemeDark)) { | |||
continue | |||
} | |||
} | |||
mediaRules = mediaRules.concat(Array.from(cssRule.cssRules)) | |||
} | |||
// WARNING: do not try to insert a Rule to a styleSheet you are | |||
// currently iterating on, otherwise the browser will be stuck | |||
// in a infinite loop… | |||
for (const mediaRule of mediaRules) { | |||
styleSheet.insertRule(mediaRule.cssText) | |||
hasDarkRules = true | |||
} | |||
} | |||
if (hasDarkRules) { | |||
loadThemeForm('#theme-selector') | |||
} | |||
}) | |||
</script> | |||
</body> | |||
</html> |
@@ -0,0 +1,414 @@ | |||
title: Adding ActivityPub to your static site | |||
url: https://paul.kinlan.me/adding-activity-pub-to-your-static-site/ | |||
hash_url: 1676902071b6e1e7e0d3395bc47956b5 | |||
<p>My blog is built on Hugo and hosted on Vercel. It mostly works well.</p> | |||
<p>I wanted to have my blog automatically publish posts that I create in a way that I didn't need to spin up an instance of Mastodon.</p> | |||
<p>I got a minimal version of it working. You can discover my page, follow my account, and it will post updates when my blog deploys a new page.</p> | |||
<p>The biggest learning that I had was that ActivityPub is a Message protocol. You can't just output a feed of posts and be done (I tried) - so even if you are a statically generated site you need a Server component because you need to POST message replies to people who send a 'Follow' request to your account and POST to the people who follow your account to 'Create' a 'Note'.</p> | |||
<p>To see it all in action, you can subscribe to my blog on any ActivityPub system by following this account: @paul@paul.kinlan.me < try it.</p> | |||
<p>While I had fun, I will say that I found it <strong>very</strong> hard to get started - I found the spec hard to read; testing was almost impossible (there seem to be no easy test harnesses to determine if you are building a compatible client) so I had to test against a live instance; and there is little documentation of what messages should look like; and I hit snags in all places.</p> | |||
<p>Hopefully this post will help you get started if you want to go down a similar path.</p> | |||
<p>My implementation uses Hugo to create my posts and feed data, Vercel Serverless functions to handle in bound messages, and Firebase Firestore to store the data.</p> | |||
<p>This post will assume that you know the terminology of ActivityPub, but I will try and link to the relevant part of the spec. I also made a lot of assumptions that I am a single user host.</p> | |||
<h3 id="discovery">Discovery</h3> | |||
<p><a href="https://docs.joinmastodon.org/spec/webfinger/">Mastodon uses Web Finger</a> to discover where to look for your servers Actor configuration. WebFinger files are served from a <code>./well-known/webfinger</code> file. I created serverless function which returns the required WebFinger configuration. <a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/api/well-known/webfinger.ts">Code</a></p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-typescript" data-lang="typescript"><span><span><span>import</span> <span>type</span> { <span>VercelRequest</span>, <span>VercelResponse</span> } <span>from</span> <span>'@vercel/node'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>export</span> <span>default</span> <span>function</span> (<span>req</span>: <span>VercelRequest</span>, <span>res</span>: <span>VercelResponse</span>) { | |||
</span></span><span><span> <span>res</span>.<span>statusCode</span> <span>=</span> <span>200</span>; | |||
</span></span><span><span> <span>res</span>.<span>setHeader</span>(<span>"Content-Type"</span>, <span>`application/jrd+json`</span>); | |||
</span></span><span><span> <span>res</span>.<span>end</span>(<span>`{ | |||
</span></span></span><span><span><span> "subject": "acct:paul@paul.kinlan.me", | |||
</span></span></span><span><span><span> "aliases": [ | |||
</span></span></span><span><span><span> "https://status.kinlan.me/@paul" | |||
</span></span></span><span><span><span> ], | |||
</span></span></span><span><span><span> "links": [ | |||
</span></span></span><span><span><span> { | |||
</span></span></span><span><span><span> "rel": "self", | |||
</span></span></span><span><span><span> "type": "application/activity+json", | |||
</span></span></span><span><span><span> "href": "https://paul.kinlan.me/paul" | |||
</span></span></span><span><span><span> } | |||
</span></span></span><span><span><span> ] | |||
</span></span></span><span><span><span> }`</span>); | |||
</span></span><span><span>} | |||
</span></span></code></pre></div><p>The JSON above describes a number of aliases for my ActivityPub account '@paul@paul.kinlan.me' and it points to where I host my ActivityPub Actor information.</p> | |||
<p><strong>Note</strong>: You need to make sure you are sending the correct MIME types.</p> | |||
<p>Why do I have a serverless function for static content? Vercel... That's why. I couldn't set the <code>Content-Type</code> configuration properly for any static file in the <code>.well-known</code> folder. In the future if I add multiple accounts I will need to parse the query string to be able to target the <code>links</code> and <code>subject</code> fields correctly/</p> | |||
<p>Next you need to create an <a href="https://www.w3.org/TR/activitypub/#actor-objects">Actor</a>. The Actor is a configuration file that tells ActivityPub servers where to find many core functions such as the 'inbox' (which will receive messages from other clients), 'outbox' that contains all the messages that a user has created (like an RSS feed), 'publicKey' for verifying messages, how my face should appear etc.</p> | |||
<p>To serve the actor file, I just send a JSON response from my <a href="https://github.com/PaulKinlan/paul.kinlan.me/tree/main/api"><code>api</code></a><code>/</code><a href="https://github.com/PaulKinlan/paul.kinlan.me/tree/main/api/activitypub"><code>activitypub</code></a><code>/</code><strong><code>actor.ts</code></strong>. You can see the <a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/api/activitypub/actor.ts">code</a> and the <a href="https://paul.kinlan.me/paul">output</a>.</p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-typescript" data-lang="typescript"><span><span><span>import</span> <span>type</span> { <span>VercelRequest</span>, <span>VercelResponse</span> } <span>from</span> <span>'@vercel/node'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>export</span> <span>default</span> <span>function</span> (<span>req</span>: <span>VercelRequest</span>, <span>res</span>: <span>VercelResponse</span>) { | |||
</span></span><span><span> <span>res</span>.<span>statusCode</span> <span>=</span> <span>200</span>; | |||
</span></span><span><span> <span>res</span>.<span>setHeader</span>(<span>"Content-Type"</span>, <span>`application/activity+json`</span>); | |||
</span></span><span><span> <span>res</span>.<span>json</span>({ | |||
</span></span><span><span> <span>"@context"</span><span>:</span> [<span>"https://www.w3.org/ns/activitystreams"</span>, { <span>"@language"</span><span>:</span> <span>"en- GB"</span> }], | |||
</span></span><span><span> <span>"type"</span><span>:</span> <span>"Person"</span>, | |||
</span></span><span><span> <span>"id"</span><span>:</span> <span>"https://paul.kinlan.me/paul"</span>, | |||
</span></span><span><span> <span>"outbox"</span><span>:</span> <span>"https://paul.kinlan.me/outbox"</span>, | |||
</span></span><span><span> <span>"following"</span><span>:</span> <span>"https://paul.kinlan.me/following"</span>, | |||
</span></span><span><span> <span>"followers"</span><span>:</span> <span>"https://paul.kinlan.me/followers"</span>, | |||
</span></span><span><span> <span>"inbox"</span><span>:</span> <span>"https://paul.kinlan.me/inbox"</span>, | |||
</span></span><span><span> <span>"preferredUsername"</span><span>:</span> <span>"paul"</span>, | |||
</span></span><span><span> <span>"name"</span><span>:</span> <span>"Paul Kinlan - Modern Web Development with Chrome"</span>, | |||
</span></span><span><span> <span>"summary"</span><span>:</span> <span>"Paul is a Developer Advocate for Chrome and the Open Web at Google and loves to help make web development easier."</span>, | |||
</span></span><span><span> <span>"icon"</span><span>:</span> [ | |||
</span></span><span><span> <span>"https://paul.kinlan.me/images/me.png"</span> | |||
</span></span><span><span> ], | |||
</span></span><span><span> <span>"publicKey"</span><span>:</span> { | |||
</span></span><span><span> <span>"@context"</span><span>:</span> <span>"https://w3id.org/security/v1"</span>, | |||
</span></span><span><span> <span>"@type"</span><span>:</span> <span>"Key"</span>, | |||
</span></span><span><span> <span>"id"</span><span>:</span> <span>"https://paul.kinlan.me/paul#main-key"</span>, | |||
</span></span><span><span> <span>"owner"</span><span>:</span> <span>"https://paul.kinlan.me/paul"</span>, | |||
</span></span><span><span> <span>"publicKeyPem"</span><span>:</span> <span>process</span>.<span>env</span>.<span>ACTIVITYPUB_PUBLIC_KEY</span> | |||
</span></span><span><span> } | |||
</span></span><span><span> }); | |||
</span></span><span><span>} | |||
</span></span></code></pre></div><p>I used a serverless function because for similar reasons to webfinger (setting the correct Content-type) <em>and</em> I wanted to embed a publicKey that I previously generated and store in Vercel's environment variables configuration.</p> | |||
<p>Now that Mastodon can find me and ActivityPub services know where my inboxes are all I needed to do now was to handle what happens when people follow and unfollow me, and what happens when I create a new post.</p> | |||
<h3 id="following">Following</h3> | |||
<p>I found this one particularly hard - it was almost impossible to find an example of what a Follow message looks like, so I ended up spending a lot of time following my account from a Mastodon client and seeing what data was <code>HTTP</code> <code>POST</code>ed; <strong>and</strong> I also need to maintain the state of who followed me (so I can send them messages later). I chose Firebase Firestore to store all follow requests because it's pretty simple, has a good client and can store JSON directly.</p> | |||
<p>ActivityPub clients will send all messages to an <code>Actor</code>'s inbox. My inbox can only handle <code>Follow</code> and <code>Undo</code> a <code>Follow</code> requests. Once a request is sent to me, I store the data in FireStore and send a response back.</p> | |||
<p>The entire flow is very complex so I will try and explain it as best I can.</p> | |||
<p><a href="https://github.com/PaulKinlan/paul.kinlan.me/tree/main/api">api</a>/<a href="https://github.com/PaulKinlan/paul.kinlan.me/tree/main/api/activitypub">activitypub</a>/<strong>inbox.ts</strong></p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-typescript" data-lang="typescript"><span><span><span>import</span> <span>type</span> { <span>VercelRequest</span>, <span>VercelResponse</span> } <span>from</span> <span>'@vercel/node'</span>; | |||
</span></span><span><span><span>import</span> { <span>AP</span> } <span>from</span> <span>'activitypub-core-types'</span>; | |||
</span></span><span><span><span>import</span> <span>type</span> { <span>Readable</span> } <span>from</span> <span>'node:stream'</span>; | |||
</span></span><span><span><span>import</span> <span>*</span> <span>as</span> <span>admin</span> <span>from</span> <span>'firebase-admin'</span>; | |||
</span></span><span><span><span>import</span> { <span>v4</span> <span>as</span> <span>uuid</span> } <span>from</span> <span>'uuid'</span>; | |||
</span></span><span><span><span>import</span> { <span>CoreObject</span>, <span>Entity</span> } <span>from</span> <span>'activitypub-core-types/lib/activitypub/index'</span>; | |||
</span></span><span><span><span>import</span> { <span>sendSignedRequest</span> } <span>from</span> <span>'../../lib/activitypub/sendSignedRequest'</span>; | |||
</span></span><span><span><span>import</span> { <span>parseSignature</span> } <span>from</span> <span>'../../lib/activitypub/utils/parseSignature'</span>; | |||
</span></span><span><span><span>import</span> { <span>fetchActorInformation</span> } <span>from</span> <span>'../../lib/activitypub/utils/fetchActorInformation'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>process</span>.<span>env</span>.<span>NODE_TLS_REJECT_UNAUTHORIZED</span> <span>=</span> <span>'0'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>if</span> (<span>!</span><span>admin</span>.<span>apps</span>.<span>length</span>) { | |||
</span></span><span><span> <span>admin</span>.<span>initializeApp</span>({ | |||
</span></span><span><span> <span>credential</span>: <span>admin.credential.cert</span>({ | |||
</span></span><span><span> <span>projectId</span>: <span>process.env.NEXT_PUBLIC_FIREBASE_PROJECT_ID</span>, | |||
</span></span><span><span> <span>clientEmail</span>: <span>process.env.FIREBASE_CLIENT_EMAIL</span>, | |||
</span></span><span><span> <span>privateKey</span>: <span>process.env.FIREBASE_PRIVATE_KEY.replace</span>(<span>/\\n/g</span>, <span>'\n'</span>) | |||
</span></span><span><span> }) | |||
</span></span><span><span> }); | |||
</span></span><span><span>} | |||
</span></span><span><span> | |||
</span></span><span><span><span>const</span> <span>db</span> <span>=</span> <span>admin</span>.<span>firestore</span>(); | |||
</span></span><span><span> | |||
</span></span><span><span><span>export</span> <span>const</span> <span>config</span> <span>=</span> { | |||
</span></span><span><span> <span>api</span><span>:</span> { | |||
</span></span><span><span> <span>bodyParser</span>: <span>false</span>, | |||
</span></span><span><span> }, | |||
</span></span><span><span>}; | |||
</span></span><span><span> | |||
</span></span><span><span><span>async</span> <span>function</span> <span>buffer</span>(<span>readable</span>: <span>Readable</span>) { | |||
</span></span><span><span> <span>const</span> <span>chunks</span> <span>=</span> []; | |||
</span></span><span><span> <span>for</span> <span>await</span> (<span>const</span> <span>chunk</span> <span>of</span> <span>readable</span>) { | |||
</span></span><span><span> <span>chunks</span>.<span>push</span>(<span>typeof</span> <span>chunk</span> <span>===</span> <span>'string'</span> <span>?</span> <span>Buffer</span>.<span>from</span>(<span>chunk</span>) <span>:</span> <span>chunk</span>); | |||
</span></span><span><span> } | |||
</span></span><span><span> <span>return</span> <span>Buffer</span>.<span>concat</span>(<span>chunks</span>); | |||
</span></span><span><span>} | |||
</span></span><span><span> | |||
</span></span><span><span><span>function</span> <span>verifySignature</span>(<span>signature</span>, <span>publicKeyJson</span>) { | |||
</span></span><span><span> <span>let</span> <span>signatureValid</span>; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>try</span> { | |||
</span></span><span><span> <span>// Verify the signature | |||
</span></span></span><span><span><span></span> <span>signatureValid</span> <span>=</span> <span>signature</span>.<span>verify</span>( | |||
</span></span><span><span> <span>publicKeyJson</span>.<span>publicKeyPem</span>, <span>// The PEM string from the public key object | |||
</span></span></span><span><span><span></span> ); | |||
</span></span><span><span> } <span>catch</span> (<span>error</span>) { | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>"Signature Verification error"</span>, <span>error</span>) | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>return</span> <span>signatureValid</span>; | |||
</span></span><span><span>} | |||
</span></span><span><span> | |||
</span></span><span><span><span>export</span> <span>default</span> <span>async</span> <span>function</span> (<span>req</span>: <span>VercelRequest</span>, <span>res</span>: <span>VercelResponse</span>) { | |||
</span></span><span><span> <span>const</span> { <span>body</span>, <span>query</span>, <span>method</span>, <span>url</span>, <span>headers</span> } <span>=</span> <span>req</span>; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>res</span>.<span>statusCode</span> <span>=</span> <span>200</span>; | |||
</span></span><span><span> <span>res</span>.<span>setHeader</span>(<span>"Content-Type"</span>, <span>`application/activity+json`</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>// Verify the message some how. | |||
</span></span></span><span><span><span></span> <span>const</span> <span>buf</span> <span>=</span> <span>await</span> <span>buffer</span>(<span>req</span>); | |||
</span></span><span><span> <span>const</span> <span>rawBody</span> <span>=</span> <span>buf</span>.<span>toString</span>(<span>'utf8'</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>message</span> <span>=</span> <<span>AP.Activity</span>><span>JSON</span>.<span>parse</span>(<span>rawBody</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>message</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>signature</span> <span>=</span> <span>parseSignature</span>(<span>req</span>); | |||
</span></span><span><span> <span>const</span> <span>actorInformation</span> <span>=</span> <span>await</span> <span>fetchActorInformation</span>(<span>signature</span>.<span>keyId</span>); | |||
</span></span><span><span> <span>const</span> <span>signatureValid</span> <span>=</span> <span>verifySignature</span>(<span>signature</span>, <span>actorInformation</span>.<span>publicKey</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>signatureValid</span> <span>==</span> <span>null</span> <span>||</span> <span>signatureValid</span> <span>==</span> <span>false</span>) { | |||
</span></span><span><span> <span>res</span>.<span>end</span>(<span>'invalid signature'</span>); | |||
</span></span><span><span> <span>return</span>; | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>// We should check the digest. | |||
</span></span></span><span><span><span></span> <span>if</span> (<span>message</span>.<span>type</span> <span>==</span> <span>"Follow"</span>) { | |||
</span></span><span><span> <span>// We are following. | |||
</span></span></span><span><span><span></span> <span>const</span> <span>followMessage</span>: <span>AP.Follow</span> <span>=</span> <<span>AP.Follow</span>><span>message</span>; | |||
</span></span><span><span> <span>if</span> (<span>followMessage</span>.<span>id</span> <span>==</span> <span>null</span>) <span>return</span>; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>collection</span> <span>=</span> <span>db</span>.<span>collection</span>(<span>'followers'</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>actorID</span> <span>=</span> (<<span>URL</span>><span>followMessage</span>.<span>actor</span>).<span>toString</span>(); | |||
</span></span><span><span> <span>const</span> <span>followDocRef</span> <span>=</span> <span>collection</span>.<span>doc</span>(<span>actorID</span>.<span>replace</span>(<span>/\//g</span>, <span>"_"</span>)); | |||
</span></span><span><span> <span>const</span> <span>followDoc</span> <span>=</span> <span>await</span> <span>followDocRef</span>.<span>get</span>(); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>followDoc</span>.<span>exists</span>) { | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>"Already Following"</span>) | |||
</span></span><span><span> <span>return</span> <span>res</span>.<span>end</span>(<span>'already following'</span>); | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>// Create the follow; | |||
</span></span></span><span><span><span></span> <span>await</span> <span>followDocRef</span>.<span>set</span>(<span>followMessage</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>guid</span> <span>=</span> <span>uuid</span>(); | |||
</span></span><span><span> <span>const</span> <span>domain</span> <span>=</span> <span>'paul.kinlan.me'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>acceptRequest</span>: <span>AP.Accept</span> <span>=</span> <<span>AP.Accept</span>>{ | |||
</span></span><span><span> <span>"@context"</span><span>:</span> <span>"https://www.w3.org/ns/activitystreams"</span>, | |||
</span></span><span><span> <span>'id'</span><span>:</span> <span>new</span> <span>URL</span>(<span>`https://</span><span>${</span><span>domain</span><span>}</span><span>/</span><span>${</span><span>guid</span><span>}</span><span>`</span>), | |||
</span></span><span><span> <span>'type'</span><span>:</span> <span>'Accept'</span>, | |||
</span></span><span><span> <span>'actor'</span><span>:</span> <span>"https://paul.kinlan.me/paul"</span>, | |||
</span></span><span><span> <span>'object'</span><span>:</span> <span>followMessage</span> | |||
</span></span><span><span> }; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>actorInbox</span> <span>=</span> <span>new</span> <span>URL</span>(<span>actorInformation</span>.<span>inbox</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>response</span> <span>=</span> <span>await</span> <span>sendSignedRequest</span>(<span>actorInbox</span>, <span>acceptRequest</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>"Following result"</span>, <span>response</span>.<span>status</span>, <span>response</span>.<span>statusText</span>, <span>await</span> <span>response</span>.<span>text</span>()); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>return</span> <span>res</span>.<span>end</span>(<span>"ok"</span>) | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>message</span>.<span>type</span> <span>==</span> <span>"Undo"</span>) { | |||
</span></span><span><span> <span>// Undo a follow. | |||
</span></span></span><span><span><span></span> <span>const</span> <span>undoObject</span>: <span>AP.Undo</span> <span>=</span> <<span>AP.Undo</span>><span>message</span>; | |||
</span></span><span><span> <span>if</span> (<span>undoObject</span> <span>==</span> <span>null</span> <span>||</span> <span>undoObject</span>.<span>id</span> <span>==</span> <span>null</span>) <span>return</span>; | |||
</span></span><span><span> <span>if</span> (<span>undoObject</span>.<span>object</span> <span>==</span> <span>null</span>) <span>return</span>; | |||
</span></span><span><span> <span>if</span> (<span>"actor"</span> <span>in</span> <span>undoObject</span>.<span>object</span> <span>==</span> <span>false</span> <span>&&</span> (<<span>CoreObject</span>><span>undoObject</span>.<span>object</span>).<span>type</span> <span>!=</span> <span>"Follow"</span>) <span>return</span>; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>docId</span> <span>=</span> <span>undoObject</span>.<span>actor</span>.<span>toString</span>().<span>replace</span>(<span>/\//g</span>, <span>"_"</span>); | |||
</span></span><span><span> <span>const</span> <span>res</span> <span>=</span> <span>await</span> <span>db</span>.<span>collection</span>(<span>'followers'</span>).<span>doc</span>(<span>docId</span>).<span>delete</span>(); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>"Deleted"</span>, <span>res</span>) | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>res</span>.<span>end</span>(); | |||
</span></span><span><span>}; | |||
</span></span></code></pre></div><ol> | |||
<li>Parse the <code>POST</code> body and cast it to an Activity object.</li> | |||
<li>Parse the signature of the request to verify the message hasn't been tampered with in transit.</li> | |||
<li>From the signature HTTP header get the <code>Actor</code> that wants to follow you and <a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/lib/activitypub/utils/fetchActorInformation.ts">fetch their Public Key </a>(from their Actor file).</li> | |||
<li>Verify the message with their Public Key</li> | |||
</ol> | |||
<p>Now we believe that we have a valid messages.</p> | |||
<p>If the message is a <code>Follow</code> request</p> | |||
<ol> | |||
<li>See if the Actor trying to follow is already in the db, if they are return;</li> | |||
<li>Add the <code>Actor</code> to the <code>followers</code> collection in FireStore</li> | |||
<li><a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/api/activitypub/inbox.ts#L100">Prepare</a> an <code>Accept</code> message to the <code>Actor</code> indicating that the Follow has been accepted and <a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/lib/activitypub/utils/sendSignedRequest.ts">send it</a>.</li> | |||
</ol> | |||
<p>If the message is an <code>Undo</code> for a <code>Follow</code> request.</p> | |||
<ol> | |||
<li>Find the data in the <code>followers</code> collection in FireStore</li> | |||
<li>Delete it.</li> | |||
</ol> | |||
<p><strong>Note</strong>: I found it hard to find much information about sending requests to servers - so after a lot of reading and experimenting I created this <a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/lib/activitypub/utils/sendSignedRequest.ts">routine</a>. It will successfully sign the HTTP request with your configured private key and attach a digest.</p> | |||
<h3 id="posting">Posting</h3> | |||
<p>Like many static sites there is no CMS that knows when new content is posted (it is static after all) so I needed to create a routine that would send my posts to all the people that follow the account.</p> | |||
<p>Firstly I generate the <code>outbox</code> so that people can read all my public posts. I use a hugo template (<a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/config.toml">layouts/index.activity_outbox.ajson</a>) that reads through all my posts and creates a <code>Create</code> object with an embedded <code>Note</code> - this is what Mastodon needs to show a Toot.</p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-go" data-lang="go"><span><span>{{<span>-</span> <span>$</span><span>pctx</span> <span>:=</span> . <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>if</span> .<span>IsHome</span> <span>-</span>}}{{ <span>$</span><span>pctx</span> = .<span>Site</span> }}{{<span>-</span> <span>end</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>$</span><span>pages</span> <span>:=</span> <span>slice</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>if</span> <span>or</span> <span>$</span>.<span>IsHome</span> <span>$</span>.<span>IsSection</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>$</span><span>pages</span> = <span>$</span><span>pctx</span>.<span>RegularPages</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>else</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>$</span><span>pages</span> = <span>$</span><span>pctx</span>.<span>Pages</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>end</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>$</span><span>limit</span> <span>:=</span> .<span>Site</span>.<span>Config</span>.<span>Services</span>.<span>RSS</span>.<span>Limit</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>if</span> <span>ge</span> <span>$</span><span>limit</span> <span>1</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>$</span><span>pages</span> = <span>$</span><span>pages</span> | <span>first</span> <span>$</span><span>limit</span> <span>-</span>}} | |||
</span></span><span><span>{{<span>-</span> <span>end</span> <span>-</span>}} | |||
</span></span><span><span>{ | |||
</span></span><span><span> <span>"@context"</span>: <span>"https://www.w3.org/ns/activitystreams"</span>, | |||
</span></span><span><span> <span>"id"</span>: <span>"{{ $.Site.BaseURL }}outbox"</span>, | |||
</span></span><span><span> <span>"summary"</span>: <span>"{{$.Site.Author.name}} - {{$.Site.Title}}"</span>, | |||
</span></span><span><span> <span>"type"</span>: <span>"OrderedCollection"</span>, | |||
</span></span><span><span> {{ <span>$</span><span>notdrafts</span> <span>:=</span> <span>where</span> <span>$</span><span>pages</span> <span>".Draft"</span> <span>"!="</span> <span>true</span> }} | |||
</span></span><span><span> {{ <span>$</span><span>all</span> <span>:=</span> <span>where</span> <span>$</span><span>notdrafts</span> <span>"Type"</span> <span>"in"</span> (<span>slice</span> <span>"journal"</span> <span>"post"</span> <span>"page"</span>)}} | |||
</span></span><span><span> <span>"totalItems"</span>: {{(<span>len</span> <span>$</span><span>all</span>)}}, | |||
</span></span><span><span> <span>"orderedItems"</span>: [ | |||
</span></span><span><span> {{ <span>range</span> <span>$</span><span>index</span>, <span>$</span><span>element</span> <span>:=</span> <span>$</span><span>all</span> }} | |||
</span></span><span><span> {{<span>-</span> <span>if</span> <span>ne</span> <span>$</span><span>index</span> <span>0</span> }}, {{ <span>end</span> }} | |||
</span></span><span><span> { | |||
</span></span><span><span> <span>"@context"</span>: <span>"https://www.w3.org/ns/activitystreams"</span>, | |||
</span></span><span><span> <span>"id"</span>: <span>"{{.Permalink}}-create"</span>, | |||
</span></span><span><span> <span>"type"</span>: <span>"Create"</span>, | |||
</span></span><span><span> <span>"actor"</span>: <span>"https://paul.kinlan.me/paul"</span>, | |||
</span></span><span><span> <span>"object"</span>: { | |||
</span></span><span><span> <span>"id"</span>: <span>"{{ .Permalink }}"</span>, | |||
</span></span><span><span> <span>"type"</span>: <span>"Note"</span>, | |||
</span></span><span><span> <span>"content"</span>: <span>"{{.Title}}<br>{{.Summary}}"</span>, | |||
</span></span><span><span> <span>"url"</span>: <span>"{{.Permalink}}"</span>, | |||
</span></span><span><span> <span>"attributedTo"</span>: <span>"https://paul.kinlan.me/paul"</span>, | |||
</span></span><span><span> <span>"to"</span>: <span>"https://www.w3.org/ns/activitystreams#Public"</span>, | |||
</span></span><span><span> <span>"published"</span>: {{ <span>dateFormat</span> <span>"2006-01-02T15:04:05-07:00"</span> .<span>Date</span> | <span>jsonify</span> }} | |||
</span></span><span><span> } | |||
</span></span><span><span> } | |||
</span></span><span><span> {{<span>end</span>}} | |||
</span></span><span><span> ] | |||
</span></span><span><span>} | |||
</span></span></code></pre></div><p>I also set up Hugo to generate this file for the "home" output type as follows</p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-toml" data-lang="toml"><span><span>[<span>mediaTypes</span>] | |||
</span></span><span><span>[<span>mediaTypes</span>.<span>"application/activity+json"</span>] | |||
</span></span><span><span><span>suffixes</span> = [<span>"ajson"</span>] | |||
</span></span><span><span> | |||
</span></span><span><span>[<span>outputFormats</span>] | |||
</span></span><span><span>[<span>outputFormats</span>.<span>ACTIVITY_OUTBOX</span>] | |||
</span></span><span><span><span>mediaType</span> = <span>"application/activity+json"</span> | |||
</span></span><span><span><span>notAlternative</span> = <span>true</span> | |||
</span></span><span><span><span>baseName</span> = <span>"outbox"</span> | |||
</span></span><span><span> | |||
</span></span><span><span>[<span>outputs</span>] | |||
</span></span><span><span><span>home</span> = [<span>"HTML"</span>, <span>"RSS"</span>, <span>"ACTIVITY_OUTBOX"</span>] | |||
</span></span></code></pre></div><p>I then serve the file:<a href="https://github.com/PaulKinlan/paul.kinlan.me/tree/main/api/activitypub"> /api/activitypub/outbox.ts</a></p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-typescript" data-lang="typescript"><span><span><span>import</span> <span>type</span> { <span>VercelRequest</span>, <span>VercelResponse</span> } <span>from</span> <span>'@vercel/node'</span>; | |||
</span></span><span><span><span>import</span> { <span>join</span> } <span>from</span> <span>'path'</span>; | |||
</span></span><span><span><span>import</span> { <span>cwd</span> } <span>from</span> <span>'process'</span>; | |||
</span></span><span><span><span>import</span> { <span>readFileSync</span> } <span>from</span> <span>'fs'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>/* | |||
</span></span></span><span><span><span> This returns a list of posts for the single user 'Paul'. | |||
</span></span></span><span><span><span> It's a GET request. This doesn't post it to anyone's timeline. | |||
</span></span></span><span><span><span>*/</span> | |||
</span></span><span><span><span>export</span> <span>default</span> <span>function</span> (<span>req</span>: <span>VercelRequest</span>, <span>res</span>: <span>VercelResponse</span>) { | |||
</span></span><span><span> <span>// All of the outbox data is generated at build time, so just return that static file. | |||
</span></span></span><span><span><span></span> <span>const</span> <span>file</span> <span>=</span> <span>join</span>(<span>cwd</span>(), <span>'public'</span>, <span>'outbox.ajson'</span>); | |||
</span></span><span><span> <span>const</span> <span>stringified</span> <span>=</span> <span>readFileSync</span>(<span>file</span>, <span>'utf8'</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>res</span>.<span>statusCode</span> <span>=</span> <span>200</span>; | |||
</span></span><span><span> <span>res</span>.<span>setHeader</span>(<span>"Content-Type"</span>, <span>`application/activity+json`</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>return</span> <span>res</span>.<span>end</span>(<span>stringified</span>); | |||
</span></span><span><span>}; | |||
</span></span></code></pre></div><p>Finally, when my Vercel build completes, I scan the generated <a href="https://paul.kinlan.me/outbox">outbox</a> using my <a href="https://paul.kinlan.me/post-deploy-webhook-for-vercel/">post-deploy Webhook for vercel</a> and calling <a href="https://github.com/PaulKinlan/paul.kinlan.me/blob/main/api/activitypub/sendNote.ts">api/activitypub/sendNote.ts</a> endpoint to post to all the followers.</p> | |||
<div class="highlight"><pre tabindex="0"><code class="language-typescript" data-lang="typescript"><span><span><span>import</span> <span>type</span> { <span>VercelRequest</span>, <span>VercelResponse</span> } <span>from</span> <span>'@vercel/node'</span>; | |||
</span></span><span><span><span>import</span> { <span>AP</span> } <span>from</span> <span>'activitypub-core-types'</span>; | |||
</span></span><span><span><span>import</span> <span>*</span> <span>as</span> <span>admin</span> <span>from</span> <span>'firebase-admin'</span>; | |||
</span></span><span><span><span>import</span> { <span>OrderedCollection</span> } <span>from</span> <span>'activitypub-core-types/lib/activitypub/index'</span>; | |||
</span></span><span><span><span>import</span> { <span>sendSignedRequest</span> } <span>from</span> <span>'../../lib/activitypub/utils/sendSignedRequest'</span>; | |||
</span></span><span><span><span>import</span> { <span>fetchActorInformation</span> } <span>from</span> <span>'../../lib/activitypub/utils/fetchActorInformation'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>process</span>.<span>env</span>.<span>NODE_TLS_REJECT_UNAUTHORIZED</span> <span>=</span> <span>'0'</span>; | |||
</span></span><span><span> | |||
</span></span><span><span><span>if</span> (<span>!</span><span>admin</span>.<span>apps</span>.<span>length</span>) { | |||
</span></span><span><span> <span>admin</span>.<span>initializeApp</span>({ | |||
</span></span><span><span> <span>credential</span>: <span>admin.credential.cert</span>({ | |||
</span></span><span><span> <span>projectId</span>: <span>process.env.NEXT_PUBLIC_FIREBASE_PROJECT_ID</span>, | |||
</span></span><span><span> <span>clientEmail</span>: <span>process.env.FIREBASE_CLIENT_EMAIL</span>, | |||
</span></span><span><span> <span>privateKey</span>: <span>process.env.FIREBASE_PRIVATE_KEY.replace</span>(<span>/\\n/g</span>, <span>'\n'</span>) | |||
</span></span><span><span> }) | |||
</span></span><span><span> }); | |||
</span></span><span><span>} | |||
</span></span><span><span> | |||
</span></span><span><span><span>const</span> <span>db</span> <span>=</span> <span>admin</span>.<span>firestore</span>(); | |||
</span></span><span><span> | |||
</span></span><span><span><span>export</span> <span>const</span> <span>config</span> <span>=</span> { | |||
</span></span><span><span> <span>api</span><span>:</span> { | |||
</span></span><span><span> <span>bodyParser</span>: <span>false</span> | |||
</span></span><span><span> } | |||
</span></span><span><span>}; | |||
</span></span><span><span> | |||
</span></span><span><span><span>/* | |||
</span></span></span><span><span><span> Sends the latest not that hasn't yet been sent. | |||
</span></span></span><span><span><span>*/</span> | |||
</span></span><span><span><span>export</span> <span>default</span> <span>async</span> <span>function</span> (<span>req</span>: <span>VercelRequest</span>, <span>res</span>: <span>VercelResponse</span>) { | |||
</span></span><span><span> <span>const</span> { <span>body</span>, <span>query</span>, <span>method</span>, <span>url</span>, <span>headers</span> } <span>=</span> <span>req</span>; | |||
</span></span><span><span> <span>const</span> { <span>token</span> } <span>=</span> <span>query</span>; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>method</span> <span>!=</span> <span>"POST"</span>) { | |||
</span></span><span><span> <span>res</span>.<span>status</span>(<span>401</span>).<span>end</span>(<span>"Invalid Method, must be POST"</span>); | |||
</span></span><span><span> <span>return</span>; | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>token</span> <span>!=</span> <span>process</span>.<span>env</span>.<span>ACTIVITYPUB_CREATE_TOKEN</span>) { | |||
</span></span><span><span> <span>res</span>.<span>status</span>(<span>401</span>).<span>end</span>(<span>"Invalid token"</span>); | |||
</span></span><span><span> <span>return</span>; | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>configCollection</span> <span>=</span> <span>db</span>.<span>collection</span>(<span>'config'</span>); | |||
</span></span><span><span> <span>const</span> <span>configRef</span> <span>=</span> <span>configCollection</span>.<span>doc</span>(<span>"config"</span>); | |||
</span></span><span><span> <span>const</span> <span>config</span> <span>=</span> <span>await</span> <span>configRef</span>.<span>get</span>(); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>config</span>.<span>exists</span> <span>==</span> <span>false</span>) { | |||
</span></span><span><span> <span>// Config doesn't exist, make something | |||
</span></span></span><span><span><span></span> <span>configRef</span>.<span>set</span>({ | |||
</span></span><span><span> <span>"lastId"</span><span>:</span> <span>0</span> | |||
</span></span><span><span> }); | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>configData</span> <span>=</span> <span>config</span>.<span>data</span>(); | |||
</span></span><span><span> <span>let</span> <span>lastId</span> <span>=</span> <span>0</span>; | |||
</span></span><span><span> <span>if</span> (<span>configData</span> <span>!=</span> <span>undefined</span>) { | |||
</span></span><span><span> <span>lastId</span> <span>=</span> <span>configData</span>.<span>lastId</span>; | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>// Get my outbox because it contains all my notes. | |||
</span></span></span><span><span><span></span> <span>const</span> <span>outboxResponse</span> <span>=</span> <span>await</span> <span>fetch</span>(<span>'https://paul.kinlan.me/outbox'</span>); | |||
</span></span><span><span> <span>const</span> <span>outbox</span> <span>=</span> <<span>OrderedCollection</span>>(<span>await</span> <span>outboxResponse</span>.<span>json</span>()); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>const</span> <span>followersCollection</span> <span>=</span> <span>db</span>.<span>collection</span>(<span>'followers'</span>); | |||
</span></span><span><span> <span>const</span> <span>followersQuerySnapshot</span> <span>=</span> <span>await</span> <span>followersCollection</span>.<span>get</span>(); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>for</span> (<span>const</span> <span>followerDoc</span> <span>of</span> <span>followersQuerySnapshot</span>.<span>docs</span>) { | |||
</span></span><span><span> <span>const</span> <span>follower</span> <span>=</span> <span>followerDoc</span>.<span>data</span>(); | |||
</span></span><span><span> <span>try</span> { | |||
</span></span><span><span> <span>const</span> <span>actorInformation</span> <span>=</span> <span>await</span> <span>fetchActorInformation</span>(<span>follower</span>.<span>actor</span>); | |||
</span></span><span><span> <span>const</span> <span>actorInbox</span> <span>=</span> <span>new</span> <span>URL</span>(<span>actorInformation</span>.<span>inbox</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>for</span> (<span>const</span> <span>iteIdx</span> <span>in</span> (<<span>AP.EntityReference</span><span>[]</span>><span>outbox</span>.<span>orderedItems</span>)) { | |||
</span></span><span><span> <span>// We have to break somewhere... do it after the first. | |||
</span></span></span><span><span><span></span> <span>const</span> <span>item</span> <span>=</span> (<<span>AP.EntityReference</span><span>[]</span>><span>outbox</span>.<span>orderedItems</span>)[<span>iteIdx</span>]; | |||
</span></span><span><span> | |||
</span></span><span><span> <span>if</span> (<span>item</span>.<span>object</span> <span>!=</span> <span>undefined</span>) { | |||
</span></span><span><span> <span>// We might not need this. | |||
</span></span></span><span><span><span></span> <span>item</span>.<span>object</span>.<span>published</span> <span>=</span> (<span>new</span> Date()).<span>toISOString</span>(); | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>`Sending to </span><span>${</span><span>actorInbox</span><span>}</span><span>`</span>, <span>item</span>); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>// Item will be an entity, i.e, { Create { Note } } | |||
</span></span></span><span><span><span></span> <span>const</span> <span>response</span> <span>=</span> <span>await</span> <span>sendSignedRequest</span>(<span>actorInbox</span>, <<span>AP.Activity</span>> <span>item</span>); | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>"Send result: "</span>, <span>actorInbox</span>, <span>response</span>.<span>status</span>, <span>response</span>.<span>statusText</span>, <span>await</span> <span>response</span>.<span>text</span>()); | |||
</span></span><span><span> | |||
</span></span><span><span> <span>break</span>; | |||
</span></span><span><span> } | |||
</span></span><span><span> } <span>catch</span> (<span>ex</span>) { | |||
</span></span><span><span> <span>console</span>.<span>log</span>(<span>"Error"</span>, <span>ex</span>, <span>follower</span>); | |||
</span></span><span><span> } | |||
</span></span><span><span> } | |||
</span></span><span><span> | |||
</span></span><span><span> <span>res</span>.<span>status</span>(<span>200</span>).<span>end</span>(<span>"ok"</span>); | |||
</span></span><span><span>}; | |||
</span></span></code></pre></div><p>The above code is relative long but the summary of it is as follows:</p> | |||
<ol> | |||
<li>Scan the outbox</li> | |||
<li>Pick the first post (I am only sending one note)</li> | |||
<li>For each follower in the <code>followers</code> table | |||
<ol> | |||
<li>Get their actor information (where their inbox is)</li> | |||
<li>Send the <code>Create</code> object from the outbox to them via a signed HTTP request</li> | |||
</ol> | |||
</li> | |||
</ol> | |||
<h3 id="voila">Voila</h3> | |||
<p>Simple... Nah. I think it's pretty complex, but it works.</p> | |||
<p>If you have created something similar, send me a comment. I'd love to improve what I have and share that with more people.</p> |
@@ -0,0 +1,335 @@ | |||
<!doctype html><!-- This is a valid HTML5 document. --> | |||
<!-- Screen readers, SEO, extensions and so on. --> | |||
<html lang="fr"> | |||
<!-- Has to be within the first 1024 bytes, hence before the `title` element | |||
See: https://www.w3.org/TR/2012/CR-html5-20121217/document-metadata.html#charset --> | |||
<meta charset="utf-8"> | |||
<!-- Why no `X-UA-Compatible` meta: https://stackoverflow.com/a/6771584 --> | |||
<!-- The viewport meta is quite crowded and we are responsible for that. | |||
See: https://codepen.io/tigt/post/meta-viewport-for-2015 --> | |||
<meta name="viewport" content="width=device-width,initial-scale=1"> | |||
<!-- Required to make a valid HTML5 document. --> | |||
<title>Understanding A Protocol (archive) — David Larlet</title> | |||
<meta name="description" content="Publication mise en cache pour en conserver une trace."> | |||
<!-- That good ol' feed, subscribe :). --> | |||
<link rel="alternate" type="application/atom+xml" title="Feed" href="/david/log/"> | |||
<!-- Generated from https://realfavicongenerator.net/ such a mess. --> | |||
<link rel="apple-touch-icon" sizes="180x180" href="/static/david/icons2/apple-touch-icon.png"> | |||
<link rel="icon" type="image/png" sizes="32x32" href="/static/david/icons2/favicon-32x32.png"> | |||
<link rel="icon" type="image/png" sizes="16x16" href="/static/david/icons2/favicon-16x16.png"> | |||
<link rel="manifest" href="/static/david/icons2/site.webmanifest"> | |||
<link rel="mask-icon" href="/static/david/icons2/safari-pinned-tab.svg" color="#07486c"> | |||
<link rel="shortcut icon" href="/static/david/icons2/favicon.ico"> | |||
<meta name="msapplication-TileColor" content="#f7f7f7"> | |||
<meta name="msapplication-config" content="/static/david/icons2/browserconfig.xml"> | |||
<meta name="theme-color" content="#f7f7f7" media="(prefers-color-scheme: light)"> | |||
<meta name="theme-color" content="#272727" media="(prefers-color-scheme: dark)"> | |||
<!-- Documented, feel free to shoot an email. --> | |||
<link rel="stylesheet" href="/static/david/css/style_2021-01-20.css"> | |||
<!-- See https://www.zachleat.com/web/comprehensive-webfonts/ for the trade-off. --> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_regular.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_bold.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_italic.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t3_regular.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t3_bold.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t3_italic.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
<script> | |||
function toggleTheme(themeName) { | |||
document.documentElement.classList.toggle( | |||
'forced-dark', | |||
themeName === 'dark' | |||
) | |||
document.documentElement.classList.toggle( | |||
'forced-light', | |||
themeName === 'light' | |||
) | |||
} | |||
const selectedTheme = localStorage.getItem('theme') | |||
if (selectedTheme !== 'undefined') { | |||
toggleTheme(selectedTheme) | |||
} | |||
</script> | |||
<meta name="robots" content="noindex, nofollow"> | |||
<meta content="origin-when-cross-origin" name="referrer"> | |||
<!-- Canonical URL for SEO purposes --> | |||
<link rel="canonical" href="https://aeracode.org/2022/12/05/understanding-a-protocol/"> | |||
<body class="remarkdown h1-underline h2-underline h3-underline em-underscore hr-center ul-star pre-tick" data-instant-intensity="viewport-all"> | |||
<article> | |||
<header> | |||
<h1>Understanding A Protocol</h1> | |||
</header> | |||
<nav> | |||
<p class="center"> | |||
<a href="/david/" title="Aller à l’accueil"><svg class="icon icon-home"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-home"></use> | |||
</svg> Accueil</a> • | |||
<a href="https://aeracode.org/2022/12/05/understanding-a-protocol/" title="Lien vers le contenu original">Source originale</a> | |||
</p> | |||
</nav> | |||
<hr> | |||
<p>Yesterday I pushed out the <a href="https://docs.jointakahe.org/en/latest/releases/0.5/">0.5.0 release of Takahē</a>, | |||
and while there's plenty left to do, this release is somewhat of a milestone | |||
in its own right, as it essentially marks the point where I've implemented | |||
enough of ActivityPub to shift focus.</p> | |||
<p>With the implementation of image posting in this release, there are now only | |||
a few things left at a <em>protocol</em> level that I know I'm missing:</p> | |||
<ul> | |||
<li> | |||
<p>Custom emoji (these are custom per-server and a mapping of name-to-image | |||
comes with each post)</p> | |||
</li> | |||
<li> | |||
<p>Reply fan-out to the original author's followers</p> | |||
</li> | |||
<li> | |||
<p>Pinned posts on profiles (and collections in general)</p> | |||
</li> | |||
<li> | |||
<p>Shared inbox delivery (to reduce fan-out requests)</p> | |||
</li> | |||
</ul> | |||
<p>My current aim is to get Takahē to a point where a few small communities can | |||
run on it (including takahe.social), and while these are nice, | |||
they are not critical for that. The reply fan-out is probably most important, | |||
but is also the easiest given what we have written already.</p> | |||
<p>Instead, it's now time to shift and focus on stability and efficiency. My | |||
general tactic for big new projects like this is an initial "spike" period, | |||
where I am more focused on pushing out code with a roughly correct architecture | |||
rather than focusing on query efficiency, caching or the like, and to then | |||
shift gears into more of a "polish" period.</p> | |||
<p>Takahē is actually pretty useable for me as a daily driver for | |||
the <a href="https://takahe.social/@takahe@jointakahe.org/">@takahe@jointakahe.org</a> | |||
account - sure, I find a few bugs here or there, but it's honestly not bad. | |||
That means, to me, it's time to shift focus a bit more towards polishing.</p> | |||
<p>The other big missing feature for a community at this point is probably having | |||
mobile app support (which I plan to do by implementing a Mastodon-compatible | |||
client API) and better moderation features (reporting and user blocking, | |||
in addition to the existing server blocking).</p> | |||
<p>So, I'm going to focus on adding those, polishing, and improving efficiency; there's now | |||
quite a few other contributors to the project who have been helping out | |||
with bugfixes, efficiency, and plenty more, which is helping a great deal.</p> | |||
<p>I'll also be sending out a few | |||
invitations to <a href="https://takahe.social">takahe.social</a> to use that as a testbed | |||
as the first small community; nothing like dogfooding your own software to see | |||
what it needs (as well as asking some existing Mastodon admins for their | |||
thoughts, if they are gracious enough to lend me some of their time).</p> | |||
<p>Still, though, getting to this point is quite a big deal - I feel like I've | |||
learned a lot about ActivityPub and its related specifications by implementing | |||
them. So let's talk about it a little bit.</p> | |||
<h2>Fan-Out</h2> | |||
<p>ActivityPub is all about "fan-out" - the process of getting posts from their | |||
authors to their followers. At the basic level, this means one HTTP request | |||
per follower to deliver it to their inbox - but there's some efficiency gains | |||
to be made with "shared inboxes", where you can push things on a per-server | |||
basis rather than per-user.</p> | |||
<p>Obviously, doing this is noisy and takes a lot of requests, and has to be done | |||
as background workers - especially as the server on the other end might be | |||
down when you try and send the message over, and you need to retry.</p> | |||
<p>Plus, whenever you reply to someone, that reply is then sent to every one of | |||
their followers so that it can appear in reply threads. This means there's an | |||
increasing amplification effect as you get more and more followers, and your | |||
server spends a lot of its life just sending request and getting requests from | |||
other servers.</p> | |||
<p>There's other aspects to fan-out, though; there was an | |||
<a href="https://ar.al/2022/11/09/is-the-fediverse-about-to-get-fryed-or-why-every-toot-is-also-a-potential-denial-of-service-attack/">excellent blog post</a> | |||
about that last month that outlines the problems with link previews. See, when | |||
you post a message to a server's inbox, Mastodon (at least) goes and fetches | |||
any image attachments, and tries to generate web previews for any links. If | |||
people have mobile clients, some of those will <em>also</em> try to fetch previews. | |||
This does not end well for unprepared servers - and for those links, they | |||
could just be someone's random blog.</p> | |||
<p>Takahē does not do this prefetching yet - we'll likely never do it for the | |||
link previews (but some clients connected to us will, once there's a client API). | |||
Post attachments and profile images are a different story - we need to at | |||
least proxy those for user privacy, but we can hopefully make it a caching | |||
proxy. If users have their timelines open, there's not a big difference between | |||
a caching proxy and prefetching for the source server, either.</p> | |||
<p>How do we solve this? Well, bundling some of this data into the original post | |||
is one idea; having shared caching proxies split between multiple servers is | |||
another potential one as well.</p> | |||
<p>That brings us, though, to the push-pull of scaling that's at the heart of | |||
ActivityPub.</p> | |||
<h2>Two Axes Of Scaling</h2> | |||
<p>In a previous post about ActivityPub and Takahē, I referred to the fact that | |||
there are two "axes" of scaling available in the protocol:</p> | |||
<ul> | |||
<li>Having more people per server/instance</li> | |||
<li>Running more servers/instances</li> | |||
</ul> | |||
<p>Both of these have their pros and cons, and it's hard to go all in on one of | |||
them - having a million people on one server is difficult to scale for that | |||
individual server (you have to start building it as its own distributed | |||
system), but having a million servers makes the fan-out problem even worse | |||
(say hello to massive prefetching loads and shared inboxes being not very | |||
useful).</p> | |||
<p>It seems to me like a bit of a separation between domain, moderator, and | |||
caching store is needed - Takahē already lets multiple domains be on a single | |||
server, but the server moderation and caching are scoped just to that one server.</p> | |||
<p>I do believe that sharing moderation across domains is a very important scaling | |||
step; this doesn't have to purely be "multiple domains on the same server", | |||
either - I think there's scope for a moderation API where you can have a | |||
team of professional (volunteer or paid) moderators look after multiple | |||
servers.</p> | |||
<p>Sharing caching and previews is also important, though; if there was just ten | |||
or so link preview caches around, and all servers used one of them, then we | |||
still avoid centralisation while massively lowering the load on the target | |||
of links.</p> | |||
<h2>That Transport Layer</h2> | |||
<p>I both love that ActivityPub is all over HTTP, and hate it.</p> | |||
<p>On the plus side, it means there's all manner of pre-existing load balancers, | |||
gateways, frameworks and more at our disposal. Plus, every programming language | |||
on Earth has some way of slinging JSON over HTTP.</p> | |||
<p>On the negative side, it's wildly inefficient. There's a lot of overhead for | |||
each individual call, <code>Accept</code> headers have to be bandied around everywhere, | |||
and there's a lot of HTTP implementation variation that has to be accounted for.</p> | |||
<p>If, magically, I could change it - would I go to something like SMTP, with its | |||
own port and protocol? I'm not entirely sure, to be honest - I do like the ease | |||
of entry with HTTP, and it does mean there's a lot of framing and encoding | |||
already agreed. Maybe HTTP as a base protocol with an optional TCP alternate | |||
for high-traffic servers to talk to each other over.</p> | |||
<p>The one thing I would get rid of, though, is JSON-LD. If you're not aware, | |||
ActivityPub is not just JSON - it's JSON-LD, which has schemas, namespaces, | |||
expanded and compressed forms, transforms, and all manner of other stuff. | |||
You need to transform each message to a canonical form before you parse it!</p> | |||
<p>I get the idea, but I was never an RDF fan (it's just JSON RDF, basically) and | |||
it just makes everything so much more complex. A plain | |||
JSON specification with known keys would have been better, I think, though | |||
I was not there when the spec was written, so I'm sure there's more context | |||
I lack.</p> | |||
<p>I do want to stress, though, that while I am not a huge fan of the transport | |||
layer, I think the object model is quite decent. If we could get | |||
<code>preferredDomain</code> in there along with some proper multi-size image support, | |||
I would be even happier than my usual buoyant self.</p> | |||
<h2>Difference Is Strength</h2> | |||
<p>A virtual Mastodon monopoly is not good for almost anyone, I think - I'm | |||
actually quite excited for Tumblr to implement ActivityPub, because it stands | |||
a chance of forcing protocol changes and improvements to be discussed, | |||
rather than directed almost entirely by one project.</p> | |||
<p>If we can get Takahē to even 5% of active users on the Fediverse, that would | |||
be a significant impact, too. I'm not sure we'll get there, but I do at least | |||
hope the attempt will also place its own bit of pressure on the protocol in | |||
terms of evolving and trying to fix some of the scaling issues we're all | |||
sailing directly towards.</p> | |||
<p>How to do that responsibly is another question - I would ideally like to make | |||
sure we have a server that is designed to easily handle things like DMCA | |||
requests, GDPR requests, and the awful spectre of terrorist content and CSAM. | |||
I'm looking into starting a fund to pay for some legal and compliance | |||
consultations on this front; it's the sort of work that | |||
every admin should not have to do themselves, and I'd love us to have a server | |||
designed to handle the requirements easily, and written guides as to how to do it.</p> | |||
<p>Still, for me, the focus right now is on growing Takahē and hopefully fostering some | |||
communities under its wing, and that means getting stability, efficiency, | |||
and working closely with people who want to use us to run communities. It's | |||
also about slowly fostering a set of people who look after it with sensible | |||
governance, so I'm not needed as a decision-making leader forever.</p> | |||
<p>I'm not yet focused on people migrating servers from Mastodon - supporting | |||
that is eventually on the roadmap, and I've reserved the appropriate URL | |||
patterns so actor/object URLs can move over seamlessly, but it's still a lot | |||
of work, and we're not ready for that quite yet.</p> | |||
<p>If you're interested in helping out with Takahē, do pop over to our | |||
<a href="https://discord.gg/qvQ39tAMvf">Discord</a> or email me at andrew@aeracode.org | |||
and mention what you'd like to help out with - there's a large | |||
<a href="https://docs.jointakahe.org/en/latest/contributing/">number of areas</a> we need | |||
help with, not just coding!</p> | |||
</article> | |||
<hr> | |||
<footer> | |||
<p> | |||
<a href="/david/" title="Aller à l’accueil"><svg class="icon icon-home"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-home"></use> | |||
</svg> Accueil</a> • | |||
<a href="/david/log/" title="Accès au flux RSS"><svg class="icon icon-rss2"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-rss2"></use> | |||
</svg> Suivre</a> • | |||
<a href="http://larlet.com" title="Go to my English profile" data-instant><svg class="icon icon-user-tie"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-user-tie"></use> | |||
</svg> Pro</a> • | |||
<a href="mailto:david%40larlet.fr" title="Envoyer un courriel"><svg class="icon icon-mail"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-mail"></use> | |||
</svg> Email</a> • | |||
<abbr class="nowrap" title="Hébergeur : Alwaysdata, 62 rue Tiquetonne 75002 Paris, +33184162340"><svg class="icon icon-hammer2"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-hammer2"></use> | |||
</svg> Légal</abbr> | |||
</p> | |||
<template id="theme-selector"> | |||
<form> | |||
<fieldset> | |||
<legend><svg class="icon icon-brightness-contrast"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-brightness-contrast"></use> | |||
</svg> Thème</legend> | |||
<label> | |||
<input type="radio" value="auto" name="chosen-color-scheme" checked> Auto | |||
</label> | |||
<label> | |||
<input type="radio" value="dark" name="chosen-color-scheme"> Foncé | |||
</label> | |||
<label> | |||
<input type="radio" value="light" name="chosen-color-scheme"> Clair | |||
</label> | |||
</fieldset> | |||
</form> | |||
</template> | |||
</footer> | |||
<script src="/static/david/js/instantpage-5.1.0.min.js" type="module"></script> | |||
<script> | |||
function loadThemeForm(templateName) { | |||
const themeSelectorTemplate = document.querySelector(templateName) | |||
const form = themeSelectorTemplate.content.firstElementChild | |||
themeSelectorTemplate.replaceWith(form) | |||
form.addEventListener('change', (e) => { | |||
const chosenColorScheme = e.target.value | |||
localStorage.setItem('theme', chosenColorScheme) | |||
toggleTheme(chosenColorScheme) | |||
}) | |||
const selectedTheme = localStorage.getItem('theme') | |||
if (selectedTheme && selectedTheme !== 'undefined') { | |||
form.querySelector(`[value="${selectedTheme}"]`).checked = true | |||
} | |||
} | |||
const prefersColorSchemeDark = '(prefers-color-scheme: dark)' | |||
window.addEventListener('load', () => { | |||
let hasDarkRules = false | |||
for (const styleSheet of Array.from(document.styleSheets)) { | |||
let mediaRules = [] | |||
for (const cssRule of styleSheet.cssRules) { | |||
if (cssRule.type !== CSSRule.MEDIA_RULE) { | |||
continue | |||
} | |||
// WARNING: Safari does not have/supports `conditionText`. | |||
if (cssRule.conditionText) { | |||
if (cssRule.conditionText !== prefersColorSchemeDark) { | |||
continue | |||
} | |||
} else { | |||
if (cssRule.cssText.startsWith(prefersColorSchemeDark)) { | |||
continue | |||
} | |||
} | |||
mediaRules = mediaRules.concat(Array.from(cssRule.cssRules)) | |||
} | |||
// WARNING: do not try to insert a Rule to a styleSheet you are | |||
// currently iterating on, otherwise the browser will be stuck | |||
// in a infinite loop… | |||
for (const mediaRule of mediaRules) { | |||
styleSheet.insertRule(mediaRule.cssText) | |||
hasDarkRules = true | |||
} | |||
} | |||
if (hasDarkRules) { | |||
loadThemeForm('#theme-selector') | |||
} | |||
}) | |||
</script> | |||
</body> | |||
</html> |
@@ -0,0 +1,168 @@ | |||
title: Understanding A Protocol | |||
url: https://aeracode.org/2022/12/05/understanding-a-protocol/ | |||
hash_url: 4b5bae499ad13fe0f5413d8c7b77c09a | |||
<p>Yesterday I pushed out the <a href="https://docs.jointakahe.org/en/latest/releases/0.5/">0.5.0 release of Takahē</a>, | |||
and while there's plenty left to do, this release is somewhat of a milestone | |||
in its own right, as it essentially marks the point where I've implemented | |||
enough of ActivityPub to shift focus.</p> | |||
<p>With the implementation of image posting in this release, there are now only | |||
a few things left at a <em>protocol</em> level that I know I'm missing:</p> | |||
<ul> | |||
<li> | |||
<p>Custom emoji (these are custom per-server and a mapping of name-to-image | |||
comes with each post)</p> | |||
</li> | |||
<li> | |||
<p>Reply fan-out to the original author's followers</p> | |||
</li> | |||
<li> | |||
<p>Pinned posts on profiles (and collections in general)</p> | |||
</li> | |||
<li> | |||
<p>Shared inbox delivery (to reduce fan-out requests)</p> | |||
</li> | |||
</ul> | |||
<p>My current aim is to get Takahē to a point where a few small communities can | |||
run on it (including takahe.social), and while these are nice, | |||
they are not critical for that. The reply fan-out is probably most important, | |||
but is also the easiest given what we have written already.</p> | |||
<p>Instead, it's now time to shift and focus on stability and efficiency. My | |||
general tactic for big new projects like this is an initial "spike" period, | |||
where I am more focused on pushing out code with a roughly correct architecture | |||
rather than focusing on query efficiency, caching or the like, and to then | |||
shift gears into more of a "polish" period.</p> | |||
<p>Takahē is actually pretty useable for me as a daily driver for | |||
the <a href="https://takahe.social/@takahe@jointakahe.org/">@takahe@jointakahe.org</a> | |||
account - sure, I find a few bugs here or there, but it's honestly not bad. | |||
That means, to me, it's time to shift focus a bit more towards polishing.</p> | |||
<p>The other big missing feature for a community at this point is probably having | |||
mobile app support (which I plan to do by implementing a Mastodon-compatible | |||
client API) and better moderation features (reporting and user blocking, | |||
in addition to the existing server blocking).</p> | |||
<p>So, I'm going to focus on adding those, polishing, and improving efficiency; there's now | |||
quite a few other contributors to the project who have been helping out | |||
with bugfixes, efficiency, and plenty more, which is helping a great deal.</p> | |||
<p>I'll also be sending out a few | |||
invitations to <a href="https://takahe.social">takahe.social</a> to use that as a testbed | |||
as the first small community; nothing like dogfooding your own software to see | |||
what it needs (as well as asking some existing Mastodon admins for their | |||
thoughts, if they are gracious enough to lend me some of their time).</p> | |||
<p>Still, though, getting to this point is quite a big deal - I feel like I've | |||
learned a lot about ActivityPub and its related specifications by implementing | |||
them. So let's talk about it a little bit.</p> | |||
<h2>Fan-Out</h2> | |||
<p>ActivityPub is all about "fan-out" - the process of getting posts from their | |||
authors to their followers. At the basic level, this means one HTTP request | |||
per follower to deliver it to their inbox - but there's some efficiency gains | |||
to be made with "shared inboxes", where you can push things on a per-server | |||
basis rather than per-user.</p> | |||
<p>Obviously, doing this is noisy and takes a lot of requests, and has to be done | |||
as background workers - especially as the server on the other end might be | |||
down when you try and send the message over, and you need to retry.</p> | |||
<p>Plus, whenever you reply to someone, that reply is then sent to every one of | |||
their followers so that it can appear in reply threads. This means there's an | |||
increasing amplification effect as you get more and more followers, and your | |||
server spends a lot of its life just sending request and getting requests from | |||
other servers.</p> | |||
<p>There's other aspects to fan-out, though; there was an | |||
<a href="https://ar.al/2022/11/09/is-the-fediverse-about-to-get-fryed-or-why-every-toot-is-also-a-potential-denial-of-service-attack/">excellent blog post</a> | |||
about that last month that outlines the problems with link previews. See, when | |||
you post a message to a server's inbox, Mastodon (at least) goes and fetches | |||
any image attachments, and tries to generate web previews for any links. If | |||
people have mobile clients, some of those will <em>also</em> try to fetch previews. | |||
This does not end well for unprepared servers - and for those links, they | |||
could just be someone's random blog.</p> | |||
<p>Takahē does not do this prefetching yet - we'll likely never do it for the | |||
link previews (but some clients connected to us will, once there's a client API). | |||
Post attachments and profile images are a different story - we need to at | |||
least proxy those for user privacy, but we can hopefully make it a caching | |||
proxy. If users have their timelines open, there's not a big difference between | |||
a caching proxy and prefetching for the source server, either.</p> | |||
<p>How do we solve this? Well, bundling some of this data into the original post | |||
is one idea; having shared caching proxies split between multiple servers is | |||
another potential one as well.</p> | |||
<p>That brings us, though, to the push-pull of scaling that's at the heart of | |||
ActivityPub.</p> | |||
<h2>Two Axes Of Scaling</h2> | |||
<p>In a previous post about ActivityPub and Takahē, I referred to the fact that | |||
there are two "axes" of scaling available in the protocol:</p> | |||
<ul> | |||
<li>Having more people per server/instance</li> | |||
<li>Running more servers/instances</li> | |||
</ul> | |||
<p>Both of these have their pros and cons, and it's hard to go all in on one of | |||
them - having a million people on one server is difficult to scale for that | |||
individual server (you have to start building it as its own distributed | |||
system), but having a million servers makes the fan-out problem even worse | |||
(say hello to massive prefetching loads and shared inboxes being not very | |||
useful).</p> | |||
<p>It seems to me like a bit of a separation between domain, moderator, and | |||
caching store is needed - Takahē already lets multiple domains be on a single | |||
server, but the server moderation and caching are scoped just to that one server.</p> | |||
<p>I do believe that sharing moderation across domains is a very important scaling | |||
step; this doesn't have to purely be "multiple domains on the same server", | |||
either - I think there's scope for a moderation API where you can have a | |||
team of professional (volunteer or paid) moderators look after multiple | |||
servers.</p> | |||
<p>Sharing caching and previews is also important, though; if there was just ten | |||
or so link preview caches around, and all servers used one of them, then we | |||
still avoid centralisation while massively lowering the load on the target | |||
of links.</p> | |||
<h2>That Transport Layer</h2> | |||
<p>I both love that ActivityPub is all over HTTP, and hate it.</p> | |||
<p>On the plus side, it means there's all manner of pre-existing load balancers, | |||
gateways, frameworks and more at our disposal. Plus, every programming language | |||
on Earth has some way of slinging JSON over HTTP.</p> | |||
<p>On the negative side, it's wildly inefficient. There's a lot of overhead for | |||
each individual call, <code>Accept</code> headers have to be bandied around everywhere, | |||
and there's a lot of HTTP implementation variation that has to be accounted for.</p> | |||
<p>If, magically, I could change it - would I go to something like SMTP, with its | |||
own port and protocol? I'm not entirely sure, to be honest - I do like the ease | |||
of entry with HTTP, and it does mean there's a lot of framing and encoding | |||
already agreed. Maybe HTTP as a base protocol with an optional TCP alternate | |||
for high-traffic servers to talk to each other over.</p> | |||
<p>The one thing I would get rid of, though, is JSON-LD. If you're not aware, | |||
ActivityPub is not just JSON - it's JSON-LD, which has schemas, namespaces, | |||
expanded and compressed forms, transforms, and all manner of other stuff. | |||
You need to transform each message to a canonical form before you parse it!</p> | |||
<p>I get the idea, but I was never an RDF fan (it's just JSON RDF, basically) and | |||
it just makes everything so much more complex. A plain | |||
JSON specification with known keys would have been better, I think, though | |||
I was not there when the spec was written, so I'm sure there's more context | |||
I lack.</p> | |||
<p>I do want to stress, though, that while I am not a huge fan of the transport | |||
layer, I think the object model is quite decent. If we could get | |||
<code>preferredDomain</code> in there along with some proper multi-size image support, | |||
I would be even happier than my usual buoyant self.</p> | |||
<h2>Difference Is Strength</h2> | |||
<p>A virtual Mastodon monopoly is not good for almost anyone, I think - I'm | |||
actually quite excited for Tumblr to implement ActivityPub, because it stands | |||
a chance of forcing protocol changes and improvements to be discussed, | |||
rather than directed almost entirely by one project.</p> | |||
<p>If we can get Takahē to even 5% of active users on the Fediverse, that would | |||
be a significant impact, too. I'm not sure we'll get there, but I do at least | |||
hope the attempt will also place its own bit of pressure on the protocol in | |||
terms of evolving and trying to fix some of the scaling issues we're all | |||
sailing directly towards.</p> | |||
<p>How to do that responsibly is another question - I would ideally like to make | |||
sure we have a server that is designed to easily handle things like DMCA | |||
requests, GDPR requests, and the awful spectre of terrorist content and CSAM. | |||
I'm looking into starting a fund to pay for some legal and compliance | |||
consultations on this front; it's the sort of work that | |||
every admin should not have to do themselves, and I'd love us to have a server | |||
designed to handle the requirements easily, and written guides as to how to do it.</p> | |||
<p>Still, for me, the focus right now is on growing Takahē and hopefully fostering some | |||
communities under its wing, and that means getting stability, efficiency, | |||
and working closely with people who want to use us to run communities. It's | |||
also about slowly fostering a set of people who look after it with sensible | |||
governance, so I'm not needed as a decision-making leader forever.</p> | |||
<p>I'm not yet focused on people migrating servers from Mastodon - supporting | |||
that is eventually on the roadmap, and I've reserved the appropriate URL | |||
patterns so actor/object URLs can move over seamlessly, but it's still a lot | |||
of work, and we're not ready for that quite yet.</p> | |||
<p>If you're interested in helping out with Takahē, do pop over to our | |||
<a href="https://discord.gg/qvQ39tAMvf">Discord</a> or email me at andrew@aeracode.org | |||
and mention what you'd like to help out with - there's a large | |||
<a href="https://docs.jointakahe.org/en/latest/contributing/">number of areas</a> we need | |||
help with, not just coding!</p> |
@@ -0,0 +1,226 @@ | |||
<!doctype html><!-- This is a valid HTML5 document. --> | |||
<!-- Screen readers, SEO, extensions and so on. --> | |||
<html lang="fr"> | |||
<!-- Has to be within the first 1024 bytes, hence before the `title` element | |||
See: https://www.w3.org/TR/2012/CR-html5-20121217/document-metadata.html#charset --> | |||
<meta charset="utf-8"> | |||
<!-- Why no `X-UA-Compatible` meta: https://stackoverflow.com/a/6771584 --> | |||
<!-- The viewport meta is quite crowded and we are responsible for that. | |||
See: https://codepen.io/tigt/post/meta-viewport-for-2015 --> | |||
<meta name="viewport" content="width=device-width,initial-scale=1"> | |||
<!-- Required to make a valid HTML5 document. --> | |||
<title>Playing with ActivityPub (archive) — David Larlet</title> | |||
<meta name="description" content="Publication mise en cache pour en conserver une trace."> | |||
<!-- That good ol' feed, subscribe :). --> | |||
<link rel="alternate" type="application/atom+xml" title="Feed" href="/david/log/"> | |||
<!-- Generated from https://realfavicongenerator.net/ such a mess. --> | |||
<link rel="apple-touch-icon" sizes="180x180" href="/static/david/icons2/apple-touch-icon.png"> | |||
<link rel="icon" type="image/png" sizes="32x32" href="/static/david/icons2/favicon-32x32.png"> | |||
<link rel="icon" type="image/png" sizes="16x16" href="/static/david/icons2/favicon-16x16.png"> | |||
<link rel="manifest" href="/static/david/icons2/site.webmanifest"> | |||
<link rel="mask-icon" href="/static/david/icons2/safari-pinned-tab.svg" color="#07486c"> | |||
<link rel="shortcut icon" href="/static/david/icons2/favicon.ico"> | |||
<meta name="msapplication-TileColor" content="#f7f7f7"> | |||
<meta name="msapplication-config" content="/static/david/icons2/browserconfig.xml"> | |||
<meta name="theme-color" content="#f7f7f7" media="(prefers-color-scheme: light)"> | |||
<meta name="theme-color" content="#272727" media="(prefers-color-scheme: dark)"> | |||
<!-- Documented, feel free to shoot an email. --> | |||
<link rel="stylesheet" href="/static/david/css/style_2021-01-20.css"> | |||
<!-- See https://www.zachleat.com/web/comprehensive-webfonts/ for the trade-off. --> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_regular.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_bold.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_italic.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t3_regular.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t3_bold.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t3_italic.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
<script> | |||
function toggleTheme(themeName) { | |||
document.documentElement.classList.toggle( | |||
'forced-dark', | |||
themeName === 'dark' | |||
) | |||
document.documentElement.classList.toggle( | |||
'forced-light', | |||
themeName === 'light' | |||
) | |||
} | |||
const selectedTheme = localStorage.getItem('theme') | |||
if (selectedTheme !== 'undefined') { | |||
toggleTheme(selectedTheme) | |||
} | |||
</script> | |||
<meta name="robots" content="noindex, nofollow"> | |||
<meta content="origin-when-cross-origin" name="referrer"> | |||
<!-- Canonical URL for SEO purposes --> | |||
<link rel="canonical" href="https://macwright.com/2022/12/09/activitypub.html"> | |||
<body class="remarkdown h1-underline h2-underline h3-underline em-underscore hr-center ul-star pre-tick" data-instant-intensity="viewport-all"> | |||
<article> | |||
<header> | |||
<h1>Playing with ActivityPub</h1> | |||
</header> | |||
<nav> | |||
<p class="center"> | |||
<a href="/david/" title="Aller à l’accueil"><svg class="icon icon-home"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-home"></use> | |||
</svg> Accueil</a> • | |||
<a href="https://macwright.com/2022/12/09/activitypub.html" title="Lien vers le contenu original">Source originale</a> | |||
</p> | |||
</nav> | |||
<hr> | |||
<p><picture><source srcset="https://macwright.com/images/2022-12-09-activitypub-mastodon.webp" type="image/webp"><img alt="Mastodon" src="https://macwright.com/images/2022-12-09-activitypub-mastodon.jpg"></source></picture></p> | |||
<p><a href="https://activitypub.rocks/">ActivityPub</a>, <a href="https://webfinger.net/">WebFinger</a>, and <a href="https://en.wikipedia.org/wiki/Mastodon_(social_network)">Mastodon</a> are getting some attention because of <a href="https://www.nytimes.com/2022/12/07/technology/twitter-rivals-alternative-platforms.html">chaos at Twitter</a>.</p> | |||
<p>It’s anyone’s guess how this all shakes out. As an active user of Twitter, it’ll be sad if it goes away. But in the meantime, let’s have some fun with ActivityPub.</p> | |||
<h2 id="activitypub">ActivityPub</h2> | |||
<p>Under the hood, there’s ActivityPub, WebFinger, and a number of other neat standards like JSON-LD, but for most people, they’re using Mastodon, the application. Mastodon is the software that you sign into and use as a Twitter alternative, and it’d built on all of those standards. There are a few other implementations of social networks based on the same standards, like <a href="https://en.wikipedia.org/wiki/Friendica">Frendica</a>, and <a href="https://pixelfed.org/">Pixelfed</a>, but right now, Mastodon is where the people are.</p> | |||
<p>Mastodon is decentralized through <em>federation</em>: users can choose a Mastodon server on which to create an account, and they can follow and interact with users on other servers. You’re relying on someone else to host the server and <a href="https://github.com/mastodon/mastodon/issues/18079">protect your data</a>, but instead of Twitter, you have a choice of servers. If one Mastodon host crashes, those users will lose their accounts, but other hosts will keep going.</p> | |||
<p>So Mastodon doesn’t offer the sort of serverless decentralization you can get with something more radical like <a href="https://scuttlebutt.nz/">Secure Scuttlebutt</a>, but on the other hand, it’s much more user-friendly. Just like Twitter, you log into a server with a username and a password, and you can easily access it on an iPhone and share content on Mastodon with a link.</p> | |||
<p>But anyway, if we’re going to have this federation system, we might as well take it seriously. One of the benefits of Mastodon is that you can run your own instance. The benefit of Mastodon being built on standards like ActivityPub is that you can interact with Mastodon without running the Mastodon application software in particular: you can build your own. So why not: why not make macwright.com an ActivityPub host?</p> | |||
<h2 id="context">Context</h2> | |||
<p><a href="https://macwright.com/2016/05/03/the-featherweight-website.html">This blog</a> runs on <a href="https://jekyllrb.com/">Jekyll</a>, one of the original static site generators. It’s hosted on <a href="https://www.netlify.com/">Netlify</a>, which has branched out to support a bunch of products, but started out as a static site host.</p> | |||
<p>I’m not going to abandon these systems to support ActivityPub. Jekyll works great for me: I’ve been using it for over a decade and have few complaints. There are spectacular examples of what you can do with custom code and indieweb standards, like <a href="https://aaronparecki.com/">Aaron’s site</a>, but that’s not for me.</p> | |||
<p>So, ActivityPub needs to be a simple addition on top of this existing site. What’s the absolute least I’ll need to implement?</p> | |||
<p>I started by reading the <a href="https://www.w3.org/TR/activitypub/">ActivityPub specification</a>, and then <a href="https://docs.joinmastodon.org/spec/activitypub/">Mastodon’s documentation of ActivityPub</a>. Right off the bat I had a few takeaways:</p> | |||
<ul><li>It isn’t possible to implement ActivityPub without a server and a database. You can’t do it with just a static site.</li><li>ActivityPub is the kind of specification that’s so generic that everything implemented on top of it is a particular “flavor” of the specification. There’s an opinionated kind of ActivityPub that Mastodon speaks, which is different from <a href="https://bookwyrm.social/">bookwyrm</a> or <a href="https://pixelfed.org/">pixelfed</a>.</li><li>The documentation for all of this is sort of spread out - to implement something compatible with Mastodon, you’ll need both WebFinger and ActivityPub support, and make sure that you’re making compatible decisions. Plus do some specialized cryptography to do HTTP signatures - something that the ActivityPub spec doesn’t specify. It’s good that we’re reusing existing specifications instead of inventing a whole new thing, but it fragments documentation and makes it a lot harder to get to a working implementation. So for the intent of getting something done, it’ll be better for me to just find a reference.</li><li>There are still things, like unfollowing, that aren’t implemented in the reference implementation, and aren’t well-documented anywhere.</li></ul> | |||
<p>And a reference arrived, thanks to <a href="https://tinysubversions.com/">Darius Kazemi</a>, perhaps the internet’s most famous bot maker and experimenter. He’s been after this for years, writing ActivityPub servers <a href="https://tinysubversions.com/notes/activitypub-tool/">on Glitch</a>, written <a href="https://tinysubversions.com/notes/reading-activitypub/">guides to ActivityPub</a>, the whole thing.</p> | |||
<p>So, the whole time I was doing this I was looking at <a href="https://github.com/dariusk/express-activitypub">express-activitypub</a>, one of Darius’s projects. It’s great - simple, but it works. Most of my work here was making it even simpler - removing some of the configurability and hardcoding things like accounts - and porting code that was dependent on Node.js to code that could run in Netlify’s edge functions, which are a whitelabeled layer on top of <a href="https://deno.land/">Deno</a> and thus use standard web APIs instead.</p> | |||
<h2 id="what-needs-building">What needs building</h2> | |||
<p>After spelunking in the express-activitypub reference implementation, I eventually ended up with the following <em>extremely minimal ActivityPub essentials</em>, listed nearly in order of difficulty:</p> | |||
<ul><li>A <a href="https://webfinger.net/">WebFinger</a> endpoint that returns account information.</li><li>A user endpoint (<code class="language-plaintext highlighter-rouge">https://macwright.com/u/photos</code>) that returns more account information if you use an <code class="language-plaintext highlighter-rouge">Accept: application/json</code> header.</li><li>An inbox (<code class="language-plaintext highlighter-rouge">https://macwright.com/api/inbox</code>) that receives follow requests.</li><li>A process to post new photos when I publish them.</li></ul> | |||
<p>With all these together, the <a href="https://macwright.com/photos/">photos section</a> of this website is a “user” that you can follow from a Mastodon server: <code class="language-plaintext highlighter-rouge">@photos@macwright.com</code>.</p> | |||
<h2 id="webfinger">WebFinger</h2> | |||
<p>Step one is WebFinger. Computer history buffs might remember the <a href="https://en.wikipedia.org/wiki/Finger_(protocol)">finger protocol</a>. This is that, for the web, without the infamous security exploits, hopefully. It’s an endpoint that you can hit to get account information. Mine only supports one user:</p> | |||
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>https://macwright.com/.well-known/webfinger | |||
?resource=acct:photos@macwright.com | |||
</code></pre></div></div> | |||
<p>So, when you search for <code class="language-plaintext highlighter-rouge">@photos@macwright.com</code> from a Mastodon host, this endpoint is what it hits: it extracts <code class="language-plaintext highlighter-rouge">macwright.com</code> from the username, assumes that <code class="language-plaintext highlighter-rouge">.well-known/webfinger</code> is there on the server, and finds the account. Simple as that. <a href="https://gist.github.com/tmcw/a9a359744693861fa6ec2887a6b01715">Here’s the code - it’s nothing all that interesting.</a></p> | |||
<h2 id="user-endpoint">User endpoint</h2> | |||
<p>This, like WebFinger, was easy to implement. It’s just an endpoint that returns some JSON. <a href="https://gist.github.com/tmcw/4ba9dfcf06c98c0d0da932b83519c662">Here it is</a>.</p> | |||
<h2 id="inbox">Inbox</h2> | |||
<p>Here’s where things get a lot more complicated. The <code class="language-plaintext highlighter-rouge">/api/inbox</code> function needs to:</p> | |||
<ul><li>Implement some HTTP signatures cryptography, which is, as far as I can tell, <a href="https://oauth.net/http-signatures/">still a work-in-progress specification</a> and isn’t very well described anywhere.</li><li>Store follow requests, and respond to them with a signed message.</li></ul> | |||
<p>So, there’s more complexity in the specific code file (which you can <a href="https://gist.github.com/tmcw/7394bc8588a63399bea23d15a34fa2fa">see here</a>) as well as in the system. We need <em>persistence</em> to be an ActivityPub host – we’ll need to store a list of all our subscribers, so that we can send them updates.</p> | |||
<p>This is where it sinks in: ActivityPub is <em>totally different from RSS</em>. Of course it is - this is a federated realtime messaging system. But think about it:</p> | |||
<ul><li>You can implement an RSS feed with basically any system. A static site generated by a static site generator like Jekyll? Sure! You can even write an RSS feed by hand and upload it with FTP if you want.</li><li>Your RSS feed doesn’t know who’s reading it. If you have 1 million people subscribed, sure, that’s fine. At most you’ll need to use caching or a CDN to help the server serve those requests, but they’re just GET requests, the simplest possible kind of internet.</li><li>RSS has obvious points of optimization. If 10,000 people subscribe to my RSS feed but 5,000 of them are using Feedbin, those 5,000 can share the same GET request that Feedbin makes to pull the latest posts.</li><li>An RSS feed reader only needs a list of feed URLs and an XML parser. It doesn’t need to have its own domain name or identity in the system. A feed reader can be a command-line script or a desktop application.</li></ul> | |||
<p>RSS (and Atom) might be the most successful “worse is better” standards of all time, up there with Markdown and JSON. Really S-Tier stuff.</p> | |||
<p>Because with ActivityPub:</p> | |||
<ul><li>If 10,000 people follow my blog, I have a database with 10,000 entries in it.</li><li>Every time I publish something, I send an update to every subscriber. If this blog gets popular, it’ll send an enormous amount of updates. Maybe there’s a more efficient way to get this done, but I couldn’t find it.</li><li>There are many Mastodon hosts and they don’t share any kind of cache so popular posts themselves <a href="https://www.jwz.org/blog/2022/11/mastodon-stampede/">have been known to DDoS websites</a>.</li><li>There’s nothing like a “feed reader” in the world of ActivityPub. If you want to subscribe to someone’s content, you need an account and to send and receive messages. You need to be addressable on the internet.</li></ul> | |||
<p>So, given the requirements of being an participant with ActivityPub, this is the edge function that uses a database. I’m using <a href="https://planetscale.com/">PlanetScale</a>, because it’s fun and a good learning experience, but anything would work.</p> | |||
<h2 id="publishing">Publishing</h2> | |||
<p>So, with the Inbox receiving new followers and recording them in a database, when I publish I’ll need to send messages to those followers.</p> | |||
<p>I publish this site by pushing to GitHub: that’s the setup that Netlify gives me, and what I prefer for deploying overall. It’s a nice setup. It also means that, unlike a WordPress site or a hosted service, there’s no “Publish” button.</p> | |||
<p>So, to publish something, I need to devise a <em>trigger</em> and a way for the publishing script to find new content. Here’s the <a href="https://gist.github.com/tmcw/e4410c0255be738379e6dbbefed3f149">publishing script</a> I cooked up. Connecting this to Netlify’s <a href="https://docs.netlify.com/site-deploys/notifications/">webhooks</a> did the trick for a trigger: when the site deploys, it hits the publishing script (which is part of the site) and publishes new updates to followers. It pulls the follower list from the database, pulls posts from the RSS feed, and pushes them.</p> | |||
<p>You might notice - this doesn’t check to see what’s new, it just publishes all the RSS items to all the subscribers. This is because I’ve found that publishing, in ActivityPub, is idempotent: each post has an ID, and if you push that post multiple times, Mastodon servers will check that they already have a post with that ID and ignore it.</p> | |||
<h2 id="architecture">Architecture</h2> | |||
<p><img alt="Flow" src="https://macwright.com/images/2022-12-09-activitypub-flow.png"></p> | |||
<p>So, in the whole loop, this website receives follow requests, stores them in a database, and then sends new posts when I publish something to all of the followers.</p> | |||
<p>My site is still deployed as a static website using Jekyll, but the ActivityPub and WebFinger endpoints are served by <a href="https://docs.netlify.com/edge-functions/overview/">Netlify Edge Functions</a>. This, to be, is a pretty good setup: I keep the simplicity and efficiency of static content, only layering in server-like dynamic systems where necessary.</p> | |||
<p>The publishing flow - a webhook that triggers an edge function - is a hack, and something I’ll change if I can figure out a better way to do it.</p> | |||
<p>It works, so far, with my photos page.</p> | |||
<h2 id="fin">Fin</h2> | |||
<p>So, how does this make you feel? Excited? Overwhelmed? A little of both?</p> | |||
<p>Hacking on ActivityPub was a fun project, but it was chaotic. ActivityPub in practice is a grab-bag of specifications and implementation-specific details. It was hard to find documentation for a lot of things and hard to debug requests that didn’t have their intended effect on Mastodon.</p> | |||
<p>ActivityPub is a distributed architecture, so it’s going to be a lot more complicated than RSS. People smarter than me rightfully wish that <a href="https://ariadne.space/2019/01/07/activitypub-the-worse-is-better-approach-to-federated-social-networking/">ActivityPub was more sophisticated</a> and more on the side of “better” than worse. And the chattiness of the protocol - the fact that if I have thousands of subscribers I’ll have to send out thousands of updates - that comes with the territory. Just look at how much overhead there is in <a href="https://en.wikipedia.org/wiki/BitTorrent">BitTorrent</a>.</p> | |||
<p>What I built isn’t an ActivityPub system as much as a Mastodon-compatible one. I think this is the key contradiction of the ActivityPub system: it’s a specification broad enough to encompass many different services, but ends up being too general to be useful by itself. There are other specifications like this - things like <a href="https://developers.google.com/kml/documentation/kml_tut">KML</a> which are technically open and specified but practically defined by what Google Earth supports and produces.</p> | |||
<p>With this frame of mind, the question becomes, if ActivityPub probably isn’t going to be a self-contained standard and instead the basis for one or two popular, homogenous implementations, and if federation is probably going to be a secondary property of those implementations, is the specification technically good enough, useful enough, correct enough, that a future Twitter-competitor will use it? I’m not sure.</p> | |||
</article> | |||
<hr> | |||
<footer> | |||
<p> | |||
<a href="/david/" title="Aller à l’accueil"><svg class="icon icon-home"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-home"></use> | |||
</svg> Accueil</a> • | |||
<a href="/david/log/" title="Accès au flux RSS"><svg class="icon icon-rss2"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-rss2"></use> | |||
</svg> Suivre</a> • | |||
<a href="http://larlet.com" title="Go to my English profile" data-instant><svg class="icon icon-user-tie"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-user-tie"></use> | |||
</svg> Pro</a> • | |||
<a href="mailto:david%40larlet.fr" title="Envoyer un courriel"><svg class="icon icon-mail"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-mail"></use> | |||
</svg> Email</a> • | |||
<abbr class="nowrap" title="Hébergeur : Alwaysdata, 62 rue Tiquetonne 75002 Paris, +33184162340"><svg class="icon icon-hammer2"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-hammer2"></use> | |||
</svg> Légal</abbr> | |||
</p> | |||
<template id="theme-selector"> | |||
<form> | |||
<fieldset> | |||
<legend><svg class="icon icon-brightness-contrast"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-brightness-contrast"></use> | |||
</svg> Thème</legend> | |||
<label> | |||
<input type="radio" value="auto" name="chosen-color-scheme" checked> Auto | |||
</label> | |||
<label> | |||
<input type="radio" value="dark" name="chosen-color-scheme"> Foncé | |||
</label> | |||
<label> | |||
<input type="radio" value="light" name="chosen-color-scheme"> Clair | |||
</label> | |||
</fieldset> | |||
</form> | |||
</template> | |||
</footer> | |||
<script src="/static/david/js/instantpage-5.1.0.min.js" type="module"></script> | |||
<script> | |||
function loadThemeForm(templateName) { | |||
const themeSelectorTemplate = document.querySelector(templateName) | |||
const form = themeSelectorTemplate.content.firstElementChild | |||
themeSelectorTemplate.replaceWith(form) | |||
form.addEventListener('change', (e) => { | |||
const chosenColorScheme = e.target.value | |||
localStorage.setItem('theme', chosenColorScheme) | |||
toggleTheme(chosenColorScheme) | |||
}) | |||
const selectedTheme = localStorage.getItem('theme') | |||
if (selectedTheme && selectedTheme !== 'undefined') { | |||
form.querySelector(`[value="${selectedTheme}"]`).checked = true | |||
} | |||
} | |||
const prefersColorSchemeDark = '(prefers-color-scheme: dark)' | |||
window.addEventListener('load', () => { | |||
let hasDarkRules = false | |||
for (const styleSheet of Array.from(document.styleSheets)) { | |||
let mediaRules = [] | |||
for (const cssRule of styleSheet.cssRules) { | |||
if (cssRule.type !== CSSRule.MEDIA_RULE) { | |||
continue | |||
} | |||
// WARNING: Safari does not have/supports `conditionText`. | |||
if (cssRule.conditionText) { | |||
if (cssRule.conditionText !== prefersColorSchemeDark) { | |||
continue | |||
} | |||
} else { | |||
if (cssRule.cssText.startsWith(prefersColorSchemeDark)) { | |||
continue | |||
} | |||
} | |||
mediaRules = mediaRules.concat(Array.from(cssRule.cssRules)) | |||
} | |||
// WARNING: do not try to insert a Rule to a styleSheet you are | |||
// currently iterating on, otherwise the browser will be stuck | |||
// in a infinite loop… | |||
for (const mediaRule of mediaRules) { | |||
styleSheet.insertRule(mediaRule.cssText) | |||
hasDarkRules = true | |||
} | |||
} | |||
if (hasDarkRules) { | |||
loadThemeForm('#theme-selector') | |||
} | |||
}) | |||
</script> | |||
</body> | |||
</html> |
@@ -0,0 +1,263 @@ | |||
<!doctype html><!-- This is a valid HTML5 document. --> | |||
<!-- Screen readers, SEO, extensions and so on. --> | |||
<html lang="fr"> | |||
<!-- Has to be within the first 1024 bytes, hence before the `title` element | |||
See: https://www.w3.org/TR/2012/CR-html5-20121217/document-metadata.html#charset --> | |||
<meta charset="utf-8"> | |||
<!-- Why no `X-UA-Compatible` meta: https://stackoverflow.com/a/6771584 --> | |||
<!-- The viewport meta is quite crowded and we are responsible for that. | |||
See: https://codepen.io/tigt/post/meta-viewport-for-2015 --> | |||
<meta name="viewport" content="width=device-width,initial-scale=1"> | |||
<!-- Required to make a valid HTML5 document. --> | |||
<title>Fast Path to a Great UX - Increased Exposure Hours (archive) — David Larlet</title> | |||
<meta name="description" content="Publication mise en cache pour en conserver une trace."> | |||
<!-- That good ol' feed, subscribe :). --> | |||
<link rel="alternate" type="application/atom+xml" title="Feed" href="/david/log/"> | |||
<!-- Generated from https://realfavicongenerator.net/ such a mess. --> | |||
<link rel="apple-touch-icon" sizes="180x180" href="/static/david/icons2/apple-touch-icon.png"> | |||
<link rel="icon" type="image/png" sizes="32x32" href="/static/david/icons2/favicon-32x32.png"> | |||
<link rel="icon" type="image/png" sizes="16x16" href="/static/david/icons2/favicon-16x16.png"> | |||
<link rel="manifest" href="/static/david/icons2/site.webmanifest"> | |||
<link rel="mask-icon" href="/static/david/icons2/safari-pinned-tab.svg" color="#07486c"> | |||
<link rel="shortcut icon" href="/static/david/icons2/favicon.ico"> | |||
<meta name="msapplication-TileColor" content="#f7f7f7"> | |||
<meta name="msapplication-config" content="/static/david/icons2/browserconfig.xml"> | |||
<meta name="theme-color" content="#f7f7f7" media="(prefers-color-scheme: light)"> | |||
<meta name="theme-color" content="#272727" media="(prefers-color-scheme: dark)"> | |||
<!-- Documented, feel free to shoot an email. --> | |||
<link rel="stylesheet" href="/static/david/css/style_2021-01-20.css"> | |||
<!-- See https://www.zachleat.com/web/comprehensive-webfonts/ for the trade-off. --> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_regular.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_bold.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_italic.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t3_regular.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t3_bold.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
<link rel="preload" href="/static/david/css/fonts/triplicate_t3_italic.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
<script> | |||
function toggleTheme(themeName) { | |||
document.documentElement.classList.toggle( | |||
'forced-dark', | |||
themeName === 'dark' | |||
) | |||
document.documentElement.classList.toggle( | |||
'forced-light', | |||
themeName === 'light' | |||
) | |||
} | |||
const selectedTheme = localStorage.getItem('theme') | |||
if (selectedTheme !== 'undefined') { | |||
toggleTheme(selectedTheme) | |||
} | |||
</script> | |||
<meta name="robots" content="noindex, nofollow"> | |||
<meta content="origin-when-cross-origin" name="referrer"> | |||
<!-- Canonical URL for SEO purposes --> | |||
<link rel="canonical" href="https://articles.uie.com/user_exposure_hours/"> | |||
<body class="remarkdown h1-underline h2-underline h3-underline em-underscore hr-center ul-star pre-tick" data-instant-intensity="viewport-all"> | |||
<article> | |||
<header> | |||
<h1>Fast Path to a Great UX - Increased Exposure Hours</h1> | |||
</header> | |||
<nav> | |||
<p class="center"> | |||
<a href="/david/" title="Aller à l’accueil"><svg class="icon icon-home"> | |||
<use xlink:href="/static/david/icons2/symbol-defs-2021-12.svg#icon-home"></use> | |||
</svg> Accueil</a> • | |||
<a href="https://articles.uie.com/user_exposure_hours/" title="Lien vers le contenu original">Source originale</a> | |||
</p> | |||
</nav> | |||
<hr> | |||
<p> | |||
As we’ve been researching what design teams need to do to create great user experiences, we’ve stumbled across an interesting finding. It’s the closest thing we’ve found to a silver bullet when it comes to reliably improving the designs teams produce. This solution is so simple that we didn’t believe it at first. After all, if it was this easy, why isn’t everyone already doing it? | |||
</p> | |||
<p> | |||
To make sure, we’ve spent the last few years working directly with teams, showing them what we found and helping them do it themselves. By golly, it actually worked. We were stunned. | |||
</p> | |||
<p> | |||
The solution? Exposure hours. The number of hours each team member is exposed directly to real users interacting with the team’s designs or the team’s competitor’s designs. There is a direct correlation between this exposure and the improvements we see in the designs that team produces. | |||
</p> | |||
<h2> | |||
It Makes Perfect Sense: Watch Your Users<br> | |||
</h2> | |||
<p> | |||
For more than 20 years, we’ve known that teams spending time watching users, can see improvements. Yet we still see many teams with regular user research programs that produce complicated, unusable products. We couldn’t understand why, until now. | |||
</p> | |||
<p> | |||
Each team member has to be exposed directly to the users themselves. Teams that have dedicated user research professionals, who watch the users, then in turn, report the results through documents or videos, don’t deliver the same benefits. It’s from the direct exposure to the users that we see the improvements in the design. | |||
</p> | |||
<p> | |||
Over the years, there has been plenty of debate over how many participants are enough for a study. It turns out we were looking in the wrong direction. When you focus on the hours of exposure, the number of participants disappears as an important discussion. We found 2 hours of direct exposure with one participant could be as valuable (if not more valuable) than eight participants at 15-minutes each. The two hours with that one participant, seeing the detailed subtleties and nuances of their interactions with the design, can drive a tremendous amount of actionable value to the team, when done well. | |||
</p> | |||
<h2> | |||
First Forays: Field Visits<br> | |||
</h2> | |||
<p> | |||