larlet-fr-david-cache/cache/2020/57341e992c1478f0d90d61df71426372/index.md

title: Why I Built My Own Shitty Static Site Generator
url: https://erikwinter.nl/articles/2020/why-i-built-my-own-shitty-static-site-generator/
hash_url: 57341e992c1478f0d90d61df71426372

<time datetime="2020-11-09 00:00:00">November 9, 2020</time>
<p>On the internet, there is no shortage of good quality <a href="https://jamstack.org/generators/">static site generators</a> (SSG’s) that you can download for free. <a href="https://gohugo.io/">Hugo</a>, <a href="https://jekyllrb.com/">Jekyll</a>, and hundreds of others are readily available. And they work. You can build all kinds of sites with them. I know that, because I’ve used some of them. Hugo was the driving force behind this website until very recently. Despite that, when I tried to add a new section a while ago, I got rather frustrated with it and decided to build my own generator. It turned out to be a very pleasant experience and not just because I like to program things.</p>
<p>While working on it, I discovered some of the deeper motivations that drove me to undertake this project. On the surface it would seem an odd thing to do, because it takes a lot of time and it appears to offer little benefit. I did not create any new or spectacular functionality. If you click around this site and think: Hey, but I could totally make this with &lt;SSG of choice&gt;, then you would probably be right. But that is not the point. There are certain advantages to making it all yourself and I suspect that these advantages trancend the subject of SSG’s and programming. My speculation is that exploring this direction might also be interesting for people who do not like to program, or maybe even for those who don’t like computers that much at all.</p>
<p>So, why choose this project out so many others that all sound so much more interesting? It is easy to summarize, but without some context it may sound a bit abstract. The real reason for all this work is that I think that a personal site should be personal and to make it personal, one should solely be guided by one’s intuition and not by the mental models of available tools and the restrictions they impose on your thoughts.</p>
<p>That probably sounded vague and perhaps a little far fetched. After all, as long as you can write the words you want to write, draw the lines you want to draw, you are not limited in your creativity, right? Maybe you are. To make the point more tangible, let me expand on my own situation for a bit. I wil focus on Hugo, since that is the SSG I know best. But the same principles hold for other generators. All other tools even, I believe.</p>
<h2>Metadata and Organising Thoughts</h2>
<p>As said, the list of available static site generators is endless. But somehow they all seem to focus on <a href="https://en.wikipedia.org/wiki/Markdown">Markdown</a> as markup language to write your posts in. Markdown is very easy to learn and that is probably the reason why it is so popular. Unfortunately, it is a pretty bad markup language for this use case, as it is very incomplete. It is not really a language. Markdown is better seen as a bunch of shortcuts to simplify writing a few common HTML tags. Out of the box you can only sort of markup the body of a document with it. Titles, paragraphs, lists, etc. But not more than that. As we are dealing with a website, shortcuts for HTML tags can be useful, but we need more. For instance, one also needs metadata, like tags, publishing dates, etc. You do want the latest post to be at the top of the newsfeed, right? Then we must find a way to indicate the time when a post was published.</p>
<p>In Hugo this is solved with the awkward concept of <a href="https://gohugo.io/content-management/front-matter/">frontmatter</a>. At the top of each Markdown file, one needs to add a block of text that is not Markdown, but another format. You can pick either YAML, TOML, or JSON. In that block you can specify the things I mentioned. Publishing date, author, category, etc. It is all very flexible, you can even define your own metadata. So you can definitely make a nice looking site out of this.</p>
<p>But the downside is that any blog or article you write is now tied to the system that you use to publish it. Frontmatter is not part of Markdown. It is part of Hugo. So other Markdown tools will not be able to extract that info. Only Hugo can. So if you, like me, like to write all kinds of things, and you want to have some of your writings to end up on your site and others not, a decision that is perhaps made even only after you’ve done the writing, then you have just created a problem. Because now you have to make two piles of writing. One pile for texts that will be published on your site, and another for text that you want to export to, say, PDF. But wait, maybe you <em>also</em> want to compile a selection of your stories and make it into an ebook. And maybe you are really proud of a piece and you want to publish it all three ways at once. It becomes a bit weird, and also very impractical, to store your work then. Are you going to change the document format each time you want to make an export? Are you going to make multiple copies? What happens if you find a spelling error later? This all quickly becomes a big mess.</p>
<p>Ideally, I want to have one folder for storing all my writing. I want to organize that folder the way I think fits best with the content. Then I want to point the site generator to that folder and it should figure out the rest by itself. What needs to be published on my site? Where should it end up? The metadata should be all that is needed to figure it out. And I want the same thing to be true for all other publishers, generators, indexers, etc. that I use, or may want to use in the future. The only solution is then to store your texts in open, tool agnostic document format that can hold all the relevant info. Preferably a plain text format too. Because that part of Markdown I do like. Using simple text editors, Git version control, yes, give me that.</p>
<p>Enter <a href="https://asciidoc.org/">Asciidoc</a>. A Markdown so structured and complete that you can write a whole book in it. Yet is has the same simple way of adding markup and it looks very similar. I wrote <a href="/notes/2020/a-tiny-subset-of-asciidoc-for-blogging/">another post</a> on how I used a subset of Asciidoc to make my generator. The point I want to make here is that a simple, in my opinion very reasonable requirement to not want to be forced to reorganise and duplicate my files in an illogical way, already rules out 90% of the available tools. And, conversely, that merely by adopting one of those existing tools, you have suddenly become a bit restricted in the way you can think about your creative work.</p>
<p>Think about it. The moment you start anything, the moment where the ideas in your head are not more than undefined glimpses of images or feelings. The moment you have to concentrate really hard to not let the fleeting, still wordless impressions slip. Blink your eyes one time too many and they will be lost, floated away. On <em>that</em> very moment, you get bothered by the question “Where should this end up, once it is finished? Is it a note for my diary, or a book?”  That is totally backwards. That should not be the first question about your work, but the last. Not everyone is the same, but for me this upfront question is limiting. Thoughts that are not mature enough to be categorized, are forced to materialize in anyway, so you can put them in the right bucket. And then they slip away.</p>
<h2>Blank Page Instead of Puzzles and Pieces</h2>
<p>By now, some pepole might be thinking: yes, that is all fine, but some generators, like Hugo <em>can</em> use Asciidoc as input and you <em>can</em> set an alternative content path. Surely you can work something out here and configure things the way it suits you?</p>
<p>Well, yes, from the face of it, it might be possible to cobble something up. But that is not going to end up well. Going that route, you will get dragged down in an endless cycle of figuring out options and options of options and in the end, you will be happy to get anything on the screen at all.</p>
<p>Let’s start simple. Some generators say they support Asciidoc, but they don’t do that nativly. At least the ones I’ve seen. That is, you have to install another piece of software to get the functionality. In this case <a href="https://asciidoctor.org/">Asciidoctor</a>. (And Asciidocter in turn requires Ruby, but I think it ends there.) Then the two pieces must be configured to work together and this must be done on overy device you want to use. This is what developers call a dependency and they are to be avoided wherever possible, as they require you to do work, just to keep things running as they are. At the time of writing you can read <a href="https://gohugo.io/content-management/formats/#additional-formats-through-external-helpers">this</a> in the Hugo documentation on how to configure Asciidoc:</p>
<p><em>“AsciiDoc implementation EOLs in Jan 2020 and is no longer supported. AsciiDoc development is being continued under Asciidoctor. The format AsciiDoc remains of course. Please continue with the implementation Asciidoctor.”</em></p>
<p>So what this says is that the people behind Hugo saw that a piece of software they relied on, AsciiDoc, was going to be outdated so they switched to another piece of external software, Asciidoctor, to prevent things from breaking. A sensible move. But for you, the user, things are now already broken, because now you have remove the first piece of software, install the second, and configure things again to make them work together. And again, this must be done on all your devices. You get to choose between options, but the options are not stable and require work by you, the user. Sometimes the options have some dependencies themselves, repeating the problem. Not a fun way to spend an afternoon.</p>
<p>But enough about Asciidoc. Let’s talk about templates.</p>
<p>It is wellknown that Hugo has a difficult templating language. This is because Hugo is built in Go and leverages the <a href="https://golang.org/pkg/html/template/">template functionality</a> of the Go standard library. That functionality is fast and elegant, but very much geared to programmers. I knew this beforehand, but since I am a Go developer, I figured this would be an advantage, not a disadvantage. I do this Go stuff all day. It has payed for the laptop I am writing on right now and the chair I am sitting in. Surely this will be easy?</p>
<p>Not so much. Writing Hugo templates turned out to be a moderately frustrating experience. It was an endless game of guessing variables, types and trying to combine them into something useful by chaining functions. There is documentation, every variable and function is listed, but a crucial thing is missing: the big picture.</p>
<p>The act of templating consists of two parts: data, and what to do with the data. Let’s say for instance we want to render a title. In the template we indicate that titles have a certain color, size and font. For each page we get a new title, the data, but we render them all the same way, like we specified. That is the job of a template. Telling how we want data to be rendered.</p>
<p>However, a page contains a lot more data than just the title. In fact, in Hugo you get access to everything. Everything that is content. Including all metadata and, crucially, how it all relates together in the big abstract, imagined structure that makes up a site. This mental picture is not shown in the documentation. And worse, it is a <em>generalized</em> mental picture of a site. Because Hugo is a universal SSG and you must have the option to build any type of site. Count the levels of unwinding you have to do before you can translate this to your actual site: generalized (one), abstract (two), mental (three) picture.</p>
<p>Of course, making use of a template already imposes some abstractions. They can be very helpful, I ended up using a few too. But sometimes it is better for all parties involved (i.e. you and your tools) to just hardcode some things.</p>
<p>These two issues, the templating and the external dependencies, have something in common. To solve them, your mind must switch to an analysing mode. There is a black box with buttons on it. If you figure out how it works inside, you can push the buttons in the right order and make it do the thing you want. If the box contains a lot of complex gears and levers, this can be a hard riddle to solve and you need to spend more effort. You will start to ask yourself questions along the lines of: What did the makers of the box think when they designed it? What was it designed for exactly? What problem does it solve and what would seem logical to them? They probably catered to the most common needs as they saw them.</p>
<p>If you want solve this riddle, you have to leave your own framing of the the problem aside for a moment and adopt theirs. You have to step out of your own thinking and into theirs.</p>
<p>At one point, if you are succesful, you’ve grasped it and then you want to get back to your own, original frame. See how you can connect the two. But more often than not this is hard, or even impossible. By making their way of doing things your own, you have overwritten your original perspective. At least in part. This is not always a bad thing, but it is important to realize that it happens. That you might not want that. Compare this with starting from scratch. No solutions to other peoples problems, just your own. This means creating a solution all by yourself, which is hard. But you can al least be sure that it fits <em>your</em> problem.</p>
<h2>The Spectrum of Software Tools</h2>
<p>So, should everyone and their mother start programming everything from scratch, even if they have no interest in making software whatsoever? That would be impractical. And probably bad for their motivation. Not to mention that for a lot of people, programming feels exactly like that magical black box with buttons and complicated machinery inside, so that would be counterproductive. Nevertheless, I think there are some general lessons to draw from this.</p>
<p>All software tools make some kind of trade-off between flexibility and ease of use. Some make a better compromise than others, but a compromise it allways will be. The easiest tool to use is the one that has only one button. Push it and you get a complete result. But in order to do that, the tool, or actually the creators of the tool, have to make all kinds of decisions for you, both big and small. If you want more control over the outcome, that is possible, but by definition that means that you have to give more input. More buttons that need to be pushed, more dials to adjust. The level of control you have will match the level of input you have to give. If you extend this far enough, add every control imaginable, you end up with the very intricate and elaborate tool that we call a programming language. In a programming language, every little detail of the end result is yours to dictate. But on the flip side, it requires a lot of input and effort to get something moving.</p>
<p>Site generators can be anywhere on this scale. One could argue that services like Facebook and Twitter are the ultimate “require only the push of one button” versions in this space. Thanks to them, anyone can publish without having to invest time and effort. Write your text, push the button and it is there for everyone to see. Design, structure, notifying readers, it is al magically there.</p>
<p>But remember, if you don’t make the decisions, someone else has do it for you. It might be a good feeling to outsource all these difficult problems. Maybe you assume that it is for the better, because you think that other person knows more about the necessary mechanics. They probably do. But on the other hand, that other person does not know what is inside <em>your</em> head.</p>
<p>If Twitter is the only publishing platform you’ll ever use, then, without trying, you will naturally start to write texts that are 280 characters or less. That is just how most people work. But maybe this limitation irritates you often enough that you start to look for a way around it. You search online and you find apps like <a href="https://threadreaderapp.com/">Threadreader</a>, that lets users string multiple tweets into one document as if they were a single text. This is a solution to the problem you had, but if you read your new posts carefully, you will notice that they don’t  “feel” right. The limitation of 280 characters is still there, but it is hidden. One tweet becomes one paragraph, so you are bound very short paragraphs and as a result the flow of your text is still very.. <em>staccato</em>. Even though your texts can now be much longer, you still can’t write the way you want. Not to mention the clumsy process of composing the multiple tweets in the right order.</p>
<p>In a situation like this, you would have been much better off with starting a <a href="https://wordpress.com/">Wordpress</a> blog. One step on the scale of tools, a little more work to do, but now you are able to write exactly the way you want. No programming required. If you want to have more control, you have to give more input. But there is a major difference between using one tool with two buttons, versus using two tools with one button.</p>
<p>So, my advise is to be aware of the restrictions and the hidden models of the tools you use as much as possible. Maybe it is not necessary to become a programmer. But imagine for a moment that you are one. Let you mind wander and see what comes up. What would you build? How would it work? And if you’ve thought of something, take as many steps on the scale as you’re comfortable with and see if you can make it work. Trust me, it will feel liberating.</p>
<h2>My Shitty SSG</h2>
<p>In the title, I mention that my generator is “shitty” and it is. It does not have many features. It is riddled with bugs and edge cases that it can’t handle. But that is not important. It works for my problem. If I don’t like something, I can fix it. If bug doesn’t bother me, I’ll let it be. Like all creative endevours, it is important to just start and get it out. You can always improve it later.</p>
<p>In a few weeks time, I will put the source online. Not for people to blindly copy and run (why would you?), but to give some inspiration for people who are still on the fence. To show them that shitty does not have to be hard and that it can be good enough, as long as it is the right kind of shitty. <em>Your</em> kind of shitty.</p>