A place to cache linked articles (think custom and personal wayback machine)
Du kannst nicht mehr als 25 Themen auswählen Themen müssen mit entweder einem Buchstaben oder einer Ziffer beginnen. Sie können Bindestriche („-“) enthalten und bis zu 35 Zeichen lang sein.

vor 1 Jahr
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146
  1. title: Getting Takahē to run on Piku
  2. url: https://taoofmac.com/space/blog/2022/12/21/0900
  3. hash_url: a889fa6d4e07bdc390d44461ed6dce21
  4. <p class="lead">Last night after work I decided to see how easy it would be to run a <a href="https://jointakahe.org" rel="external">Takahē</a> <a href="/space/protocols/activitypub" rel="next">ActivityPub</a> instance under <a href="https://github.com/piku/piku" rel="external">Piku</a>, my tiny <a href="/space/dev/python" rel="next">Python</a>-oriented PaaS.</p>
  5. <p>Self-hosting <a href="https://joinmastodon.org" rel="external">Mastodon</a> is all the rage, but having to deal with a full-blown installation of <a href="/space/dev/ruby" rel="next">Ruby</a> (which is always a pain to install properly, even if you use <code>rbenv</code>), plus the abomination that is <a href="https://sidekiq.org" rel="external">Sidekiq</a> and the overall Rube Goldberg-esque architectural approach that is almost mandatory to deal with the complexities of <a href="/space/protocols/activitypub" rel="next">ActivityPub</a> is just something I don’t want to maintain. Ever. Even inside Docker.</p>
  6. <p>Which is why I have been developing my own <a href="/space/protocols/activitypub" rel="next">ActivityPub</a> server using <a href="https://sanic.dev" rel="external">Sanic</a> and a very lightweight <code>asyncio</code>-based approach at handling all the transactional aspects of <a href="/space/protocols/activitypub" rel="next">ActivityPub</a> atop <a href="/space/db/sqlite" rel="next">SQLite</a>. And let me tell you, I honestly wish the protocol was less about doing what boils down to P2P webhooks with PEM signatures embedded in requests.</p>
  7. <a id="anchor-enter-takahe" class="anchor" href="/space/blog/2022/12/21/0900#enter-takahe" rel="anchor"><h2 id="enter-takahe">Enter Takahē</h2></a><figure>
  8. <img src="/media/blog/2022/12/21/0900/bqAzQDKitQgUZ3Uu9MByVxvt8r4=/takahe.png" title="logo">
  9. <figcaption>Blue birds are cool. Well, at least flightless ones.</figcaption>
  10. </figure>
  11. <p>But <a href="https://jointakahe.org" rel="external">Takahē</a> is now aiming to support client apps as of version <code>0.6</code>, is built on <a href="dev/python/django" rel="nofollow">Django</a> (which I have always loved as a framework), and it saves me from the trouble of building <em>everything</em> from scratch, so… I had to try it out.</p>
  12. <p>More to the point, <a href="dev/python/django" rel="nofollow">Django</a> is <em>exactly</em> what <a href="https://github.com/piku/piku" rel="external">Piku</a> was originally designed to run.</p>
  13. <p>Besides running as a <code>WSGI</code> app, <a href="https://jointakahe.org" rel="external">Takahē</a> uses an async stator to handle all the background tasks (which is also <em>exactly</em> the pattern I aim for and designed <a href="https://github.com/piku/piku" rel="external">Piku</a> to support), so I just had to see how easy it was to get it running under <a href="https://github.com/piku/piku" rel="external">Piku</a> on very low-end hardware.</p>
  14. <a id="anchor-the-hardware" class="anchor" href="/space/blog/2022/12/21/0900#the-hardware" rel="anchor"><h2 id="the-hardware">The Hardware</h2></a><p>I have a 4GB <a href="/space/hw/raspberry_pi" rel="next">Raspberry Pi</a> 4s set up as an SSD-backed <a href="https://github.com/pimox/pimox7" rel="external">Proxomox</a> server, hosting several different <code>arm64</code> LXC containers I use for developing stuff. I love it because I can use LXC CPU allocations to throttle things and make sure they run fast enough on very low-end hardware, plus I can just snapshot, mess up and restore entire environments.</p>
  15. <p>So I set up an Ubuntu 22.04 container with 1GB of RAM and access to 2 CPU cores, capped to 50% overall usage–which is <em>roughly</em> the performance of a <a href="/space/hw/raspberry_pi" rel="next">Raspberry Pi</a> 2 give or take, albeit with a fully 64-bit CPU.</p>
  16. <p>I deployed <a href="https://github.com/piku/piku" rel="external">Piku</a>, set up a CloudFlare tunnel, and then went to town.</p>
  17. <a id="anchor-zero-code-changes-required" class="anchor" href="/space/blog/2022/12/21/0900#zero-code-changes-required" rel="anchor"><h2 id="zero-code-changes-required">Zero Code Changes Required</h2></a><p>In short, what I needed to get <a href="https://jointakahe.org" rel="external">Takahē</a> up and running under <a href="https://github.com/piku/piku" rel="external">Piku</a> was to:</p>
  18. <ol>
  19. <li>Clone the repository.</li>
  20. <li>Create a <code>production</code> remote pointing to <a href="https://github.com/piku/piku" rel="external">Piku</a>.</li>
  21. <li>Edit the supplied <code>ENV</code> and <code>Procfile</code>.</li>
  22. <li>Do a <code>git push production main</code>.</li>
  23. </ol>
  24. <p>It was <em>that simple</em>.</p>
  25. <p>Here’s the configuration I used, annotated. First the <code>ENV</code> file:</p>
  26. <div class="highlight"><pre><span></span><code><span class="c1"># Yes, I went and got it to use SQLite, and it nearly worked 100%</span><span class="w"></span>
  27. <span class="na">TAKAHE_DATABASE_SERVER</span><span class="o">=</span><span class="s">sqlite:////home/piku/takahe.db</span><span class="w"></span>
  28. <span class="c1"># This is what I eventually migrated to (more below)</span><span class="w"></span>
  29. <span class="c1"># TAKAHE_DATABASE_SERVER=postgres://piku:&lt;password&gt;@localhost/takahe</span><span class="w"></span>
  30. <span class="c1"># I actually love Django debugging, and with it on I can see the inner workings</span><span class="w"></span>
  31. <span class="na">TAKAHE_DEBUG</span><span class="o">=</span><span class="s">true</span><span class="w"></span>
  32. <span class="c1"># You know who uses this password, don't you? </span><span class="w"></span>
  33. <span class="na">TAKAHE_SECRET_KEY</span><span class="o">=</span><span class="s">pepsicola</span><span class="w"></span>
  34. <span class="c1"># No, it's not the one I'm actually using.</span><span class="w"></span>
  35. <span class="c1"># Anyway, this next one breaks a little on Piku, so I need to revise parsing for this case.</span><span class="w"></span>
  36. <span class="na">TAKAHE_CSRF_TRUSTED_ORIGINS</span><span class="o">=</span><span class="s">["http://127.0.0.1:8000", "https://127.0.0.1:8000"]</span><span class="w"></span>
  37. <span class="na">TAKAHE_USE_PROXY_HEADERS</span><span class="o">=</span><span class="s">true</span><span class="w"></span>
  38. <span class="na">TAKAHE_EMAIL_SERVER</span><span class="o">=</span><span class="s">console://console</span><span class="w"></span>
  39. <span class="na">TAKAHE_MAIN_DOMAIN</span><span class="o">=</span><span class="s">insightful.systems</span><span class="w"></span>
  40. <span class="na">TAKAHE_ENVIRONMENT</span><span class="o">=</span><span class="s">development</span><span class="w"></span>
  41. <span class="na">TAKAHE_MEDIA_BACKEND</span><span class="o">=</span><span class="s">local://</span><span class="w"></span>
  42. <span class="na">TAKAHE_MEDIA_ROOT</span><span class="o">=</span><span class="s">/home/piku/media</span><span class="w"></span>
  43. <span class="na">TAKAHE_MEDIA_URL</span><span class="o">=</span><span class="s">https://insightful.systems/media/</span><span class="w"></span>
  44. <span class="na">TAKAHE_AUTO_ADMIN_EMAIL</span><span class="o">=</span><span class="s">&lt;my e-mail&gt;</span><span class="w"></span>
  45. <span class="na">SERVER_NAME</span><span class="o">=</span><span class="s">insightful.systems</span><span class="w"></span>
  46. <span class="c1"># This is all Piku config from here on down</span><span class="w"></span>
  47. <span class="c1"># I need IPv6 off for sanity inside Proxmox</span><span class="w"></span>
  48. <span class="na">DISABLE_IPV6</span><span class="o">=</span><span class="s">true</span><span class="w"></span>
  49. <span class="na">LC_ALL</span><span class="o">=</span><span class="s">en_US.UTF-8</span><span class="w"></span>
  50. <span class="na">LANG</span><span class="o">=</span><span class="s">$LC_ALL</span><span class="w"></span>
  51. <span class="c1"># This ensures nginx only accepts requests from CloudFlare, plus a few extra tweaks</span><span class="w"></span>
  52. <span class="na">NGINX_CLOUDFLARE_ACL</span><span class="o">=</span><span class="s">True</span><span class="w"></span>
  53. <span class="na">NGINX_SERVER_NAME</span><span class="o">=</span><span class="s">$SERVER_NAME</span><span class="w"></span>
  54. <span class="c1"># These are caching settings for my dev branch of Piku</span><span class="w"></span>
  55. <span class="na">NGINX_CACHE_SIZE</span><span class="o">=</span><span class="s">2</span><span class="w"></span>
  56. <span class="na">NGINX_CACHE_TIME</span><span class="o">=</span><span class="s">28800</span><span class="w"></span>
  57. <span class="na">NGINX_CACHE_DAYS</span><span class="o">=</span><span class="s">12</span><span class="w"></span>
  58. <span class="c1"># This has nginx cache these prefixes</span><span class="w"></span>
  59. <span class="na">NGINX_CACHE_PREFIXES</span><span class="o">=</span><span class="s">/media,/proxy </span><span class="w"></span>
  60. <span class="c1"># This maps static user media directly to an nginx route</span><span class="w"></span>
  61. <span class="na">NGINX_STATIC_PATHS</span><span class="o">=</span><span class="s">/media:/home/piku/media,/static:static,/robots.txt:static/robots.txt</span><span class="w"></span>
  62. <span class="na">PORT</span><span class="o">=</span><span class="s">8000</span><span class="w"></span>
  63. <span class="c1"># You want to set these, trust me. I should make them defaults in Piku.</span><span class="w"></span>
  64. <span class="na">PYTHONIOENCODING</span><span class="o">=</span><span class="s">UTF_8:replace</span><span class="w"></span>
  65. <span class="na">PYTHONUNBUFFERED</span><span class="o">=</span><span class="s">1</span><span class="w"></span>
  66. <span class="na">TZ</span><span class="o">=</span><span class="s">Europe/Lisbon</span><span class="w"></span>
  67. <span class="c1"># This tells uWSGI to shut down idle HTTP workers</span><span class="w"></span>
  68. <span class="c1"># Saves RAM, but startup from idle is a bit more expensive CPU-wise</span><span class="w"></span>
  69. <span class="na">UWSGI_IDLE</span><span class="o">=</span><span class="s">60</span><span class="w"></span>
  70. <span class="c1"># We need to run at least 2 uWSGI workers for Takahe</span><span class="w"></span>
  71. <span class="na">UWSGI_PROCESSES</span><span class="o">=</span><span class="s">2</span><span class="w"></span>
  72. <span class="c1"># Each worker will have this many threads </span><span class="w"></span>
  73. <span class="c1"># (even though I'm only giving this 2 cores)</span><span class="w"></span>
  74. <span class="c1"># to match the original gunicorn config.</span><span class="w"></span>
  75. <span class="na">UWSGI_THREADS</span><span class="o">=</span><span class="s">4</span><span class="w"></span>
  76. </code></pre></div>
  77. <p>…and only very minor changes to the <code>Procfile</code>:</p>
  78. <div class="highlight"><pre><span></span><code><span class="nl">wsgi</span><span class="p">:</span><span class="w"> </span>takahe.wsgi<span class="err">:</span>application<span class="w"></span>
  79. <span class="nl">worker</span><span class="p">:</span><span class="w"> </span>python<span class="w"> </span>manage.py<span class="w"> </span>runstator<span class="w"></span>
  80. <span class="nl">release</span><span class="p">:</span><span class="w"> </span>python<span class="w"> </span>manage.py<span class="w"> </span>migrate<span class="w"></span>
  81. </code></pre></div>
  82. <p>In essence, I removed <code>gunicorn</code> (which I could use anyway) to let <code>uWSGI</code> handle HTTP requests and scale down to <em>zero</em> (saving RAM). And yes, <a href="https://github.com/piku/piku" rel="external">Piku</a> also supports <code>release</code> activities, thanks to <a href="https://github.com/chr15m" rel="external">Chris McCormick</a>.</p>
  83. <p>And that was it. <em>Zero</em> code changes. None. <em>Nada</em>. And I can use exactly the same setup on <em>any</em> VPS on the planet, thanks to <a href="https://github.com/piku/piku" rel="external">Piku</a>.</p>
  84. <p>After a little faffing about with the media storage settings (which I got wrong the first time around, since <a href="https://jointakahe.org" rel="external">Takahē</a> also uses <code>/static</code> for its own assets), I had a fully working <a href="/space/protocols/activitypub" rel="next">ActivityPub</a> instance, and, well… John Mastodon just happened to sign up:</p>
  85. <figure>
  86. <img src="/media/blog/2022/12/21/0900/4XIkhj5AApzxqSDDOgaKutrv51k=/johnmastodon.jpg" title="John Mastodon, with fail whale by Matthew Inman" rel="hero">
  87. <figcaption>Every ActivityPub server ought to make this their demo account.</figcaption>
  88. </figure>
  89. <a id="anchor-teething-issues" class="anchor" href="/space/blog/2022/12/21/0900#teething-issues" rel="anchor"><h2 id="teething-issues">Teething Issues</h2></a><p><a href="https://jointakahe.org" rel="external">Takahē</a> <em>nearly</em> works with <a href="/space/db/sqlite" rel="next">SQLite</a>, but sadly it relies on <code>JSON_CONTAINS</code>, which is an unsupported feature in <a href="/space/db/sqlite" rel="next">SQLite</a> (but one which <a href="/space/db/postgresql" rel="next">PostgreSQL</a> excels at).</p>
  90. <p>The upshot of this was that the stator <code>worker</code> was very sad and bombed out when trying to handle hashtags–but all critical stuff worked, so there might well be a workaroud.</p>
  91. <p>But I took some time after breakfast to migrate the database, and since my <a href="dev/python/django" rel="nofollow">Django</a> skills are rusty, here are my notes:</p>
  92. <div class="highlight"><pre><span></span><code><span class="c1"># Open a shell to Piku</span>
  93. ssh -t <a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="daaab3b1af9abbb9aeb3acb3aea3aaafb8f4b6bbb4">[email protected]</a> run takahe bash
  94. sudo apt install postgresql
  95. python manage.py dumpdata &gt; /tmp/dump.json
  96. sudo su - postgres
  97. psql
  98. </code></pre></div>
  99. <div class="highlight"><pre><span></span><code><span class="c1">-- Set up the database</span>
  100. <span class="k">create</span><span class="w"> </span><span class="k">user</span><span class="w"> </span><span class="n">piku</span><span class="p">;</span><span class="w"></span>
  101. <span class="k">create</span><span class="w"> </span><span class="k">database</span><span class="w"> </span><span class="n">takahe</span><span class="p">;</span><span class="w"></span>
  102. <span class="k">alter</span><span class="w"> </span><span class="k">role</span><span class="w"> </span><span class="n">piku</span><span class="w"> </span><span class="k">with</span><span class="w"> </span><span class="n">password</span><span class="w"> </span><span class="s1">'&lt;mysecret&gt;'</span><span class="p">;</span><span class="w"></span>
  103. <span class="k">grant</span><span class="w"> </span><span class="k">all</span><span class="w"> </span><span class="k">privileges</span><span class="w"> </span><span class="k">on</span><span class="w"> </span><span class="k">database</span><span class="w"> </span><span class="n">takahe</span><span class="w"> </span><span class="k">to</span><span class="w"> </span><span class="n">piku</span><span class="p">;</span><span class="w"></span>
  104. <span class="k">alter</span><span class="w"> </span><span class="k">database</span><span class="w"> </span><span class="n">takahe</span><span class="w"> </span><span class="k">owner</span><span class="w"> </span><span class="k">to</span><span class="w"> </span><span class="n">piku</span><span class="p">;</span><span class="w"></span>
  105. </code></pre></div>
  106. <div class="highlight"><pre><span></span><code><span class="c1"># Reset all the migrations, just in case</span>
  107. find . -path “*/migrations/*.py” -not -name “__init__.py” -delete
  108. find . -path “*/migrations/*.pyc” -delete
  109. <span class="c1"># Reapply them</span>
  110. python manage.py makemigrations
  111. python manage.py migrate
  112. <span class="c1"># Wipe all default entities</span>
  113. python manage.py shell
  114. from django.contrib.contenttypes.models import ContentType
  115. ContentType.objects.all<span class="o">()</span>.delete<span class="o">()</span>
  116. <span class="c1"># Load everything back</span>
  117. python manage.py loaddata /tmp/dump.json
  118. </code></pre></div>
  119. <a id="anchor-performance" class="anchor" href="/space/blog/2022/12/21/0900#performance" rel="anchor"><h2 id="performance">Performance</h2></a><p>Overall, I’m quite impressed with the whole thing. Even with such measly resources and Linux’s tendency to take up RAM with buffers, <a href="https://jointakahe.org" rel="external">Takahē</a> under <a href="https://github.com/piku/piku" rel="external">Piku</a> is taking up around 100MB per active worker (2 web handlers, plus the stator worker), plus less than 50MB for <a href="/space/db/postgresql" rel="next">PostgreSQL</a> and <code>nginx</code> <em>together</em>.</p>
  120. <p>So I’m seeing <em>less than 512MB of RAM</em> in actual use, and a steady &lt;10% CPU load inside the container as the stator keeps picking up inbound updates, handling them (including any outbound requests) and doing all the messy housekeeping associated with <a href="/space/protocols/activitypub" rel="next">ActivityPub</a>:</p>
  121. <figure>
  122. <img src="/media/blog/2022/12/21/0900/-ppthry9RSX9xU_aab-SgYe8sok=/pimox.jpg" title="My Pimox console, showing CPU and RAM" rel="hero">
  123. <figcaption>These are the stats a few hours later in the day, after publishing this post.</figcaption>
  124. </figure>
  125. <p>But here’s the kicker: Since this is being capped inside LXC, that is actually around 5% <em>overall</em> CPU load on the hardware–which should translate to something like 2% of CPU usage on any kind of “real” hardware. </p>
  126. <p>With only one active user for now (but following a few accounts already), this is very, very promising.</p>
  127. <p>I have no real plans to leave <code>mastodon.social</code> for my own domain, but using <a href="https://jointakahe.org" rel="external">Takahē</a> to host a small group of people (or a company) with nothing more than a tiny VPS seems entirely feasible, and is certainly in my future.</p>
  128. <a id="anchor-next-steps" class="anchor" href="/space/blog/2022/12/21/0900#next-steps" rel="anchor"><h2 id="next-steps">Next Steps</h2></a><p>Right now, I’m going to try to contribute by testing various iOS clients (I will be using the <a href="https://jointakahe.org" rel="external">Takahē</a> public test instance as well) and do some minor tweaks to my install, namely:</p>
  129. <ul>
  130. <li>Setting up <code>nginx</code> caching. Cloudflare is already caching one third of the data, but I want to bulk up this setup so that I can eventually move it to Azure, and I’ve been meaning to add that to <a href="https://github.com/piku/piku" rel="external">Piku</a> anyway.</li>
  131. <li>Fine-tuning the stator to see how it scales up or down (I might want to try to scale it down further).</li>
  132. <li>Trying <code>gunicorn</code> to see if it makes any difference in overall RAM and CPU.</li>
  133. <li>Seeing if I can get it to work on Azure Functions (that is sure to be fun, although the current SDK failed to install on my M1 and I haven’t tried since).</li>
  134. <li>Look at how media assets are handled and see if I can add a patch to support Azure Storage via <a href="https://github.com/rcarmo/aioazstorage" rel="external">my own <code>aioazstorage</code> library</a>.</li>
  135. <li>Deploy on my <a href="https://github.com/rcarmo/azure-k3s-cluster" rel="external">k3s cluster</a>, to get a feel for how much it would cost to run on spot instances.</li>
  136. </ul>
  137. <p>There goes my holiday break, I guess…</p>
  138. <h3 id="update-a-few-days-later"><strong>Update:</strong> A Few Days Later</h3>
  139. <p>I’ve since sorted out <code>nginx</code> caching in <a href="https://github.com/piku/piku" rel="external">Piku</a> (and will soon be merging it to <code>main</code>), which makes things significantly snappier. I’ve also filed <a href="https://github.com/jointakahe/takahe/issues/287" rel="external">#287</a> to improve caching via Cloudflare and <a href="https://github.com/jointakahe/takahe/pull/288" rel="external">#288</a> to have <code>nginx</code> immediately cache assets (which works for me, at least).</p>
  140. <p>Before that, I had some fun tuning stator pauses and filed <a href="https://github.com/jointakahe/takahe/issues/232" rel="external">#232</a>, which resulted in a tweak that lowered idle CPU consumption to a pretty amazing 3% in my test instance.</p>
  141. <p>With the caching tweaks, <code>gunicorn</code> doesn’t have any real advantage against <code>uWSGI</code> workers, although I suspect that may be different in higher-load instances.</p>
  142. <p>I’ve also tossed the source tree into an Azure Function and got it to “work”, but not fully. Right now I’m not sure that is worth pursuing given I still need an external database, but I’m really curious to try again in a few months’ time.</p>