title: How to Design Perfect (Software) Products
url: http://hintjens.com/blog:19
hash_url: 2bfa30e8d911bac28aa4844302437c6c

<p>My tweet <a href="https://twitter.com/#!/hintjens/status/162619976451244032">"Still amazed by the power of engineers to over-design. Complexity is easy, folks, it's simplicity that is hard"</a> got over 50 retweets. Clearly I touched a nerve in a world swimming in hopeless complexity. But talk is easy. How do we design for simplicity? Well, I've got a process, which I will explain. I call this process "Simplicity Oriented Design", or SOD.</p>
<p>Before we get to SOD, let's look at two other classic design processes. These don't work, yet are firmly applied by a majority of engineers and designers, especially in software where it's possible to construct byzantine complexity. They are slow-motion tragedies but can be fun to watch, from a safe distance.</p>
<h2 id="toc0"><span>Trash-Oriented Design</span></h2>
<p>The most popular design process in large businesses seems to be "Trash Oriented Design", or TOD. TOD feeds off the belief that all we need to make money are great ideas. It's tenacious nonsense but a powerful crutch for people who lack imagination. The theory goes that ideas are rare, so the trick is to capture them. It's like non-musicians being awed by a guitar player, not realizing that great talent is so cheap it literally plays on the streets for coins.</p>
<p>The main output of TODs are expensive "ideations", concepts, design documents, and finally products that go straight into the trash can. It works as follows:</p>
<ul>
<li>The Creative People come up with long lists of "we could do X and Y". I've seen endlessly detailed lists of everything amazing a product could do. Once the creative work of idea generation has happened, it's just a matter of execution, of course.</li>
</ul>
<ul>
<li>So the managers and their consultants pass their brilliant, world shattering ideas to "user experience" designers. These talented designers take the tens of ideas the managers came up with, and turn them into hundreds of amazing, world-changing designs.</li>
</ul>
<ul>
<li>These bountiful and detailed designs get passed to engineers, who scratch their heads and wonder who the heck came up with such stupid nonsense. They start to argue back but the designs come from up high, and really, it's not up to engineers to argue with creative people and expensive consultants.</li>
</ul>
<ul>
<li>So the engineers creep back to their cubicles, humiliated and threatened into building the gigantic but oh so elegant pile of crap. It is bone-breakingly hard work since the designs take no account of practical costs. Minor whims might take weeks of work to build. As the project gets delayed, the managers bully the engineers into giving up their evenings and weekends.</li>
</ul>
<ul>
<li>Eventually, something resembling a working product makes it out of the door. It's creaky and fragile, complex and ugly. The designers curse the engineers for their incompetence and pay more consultants to put lipstick onto the pig, and slowly the product starts to look a little nicer.</li>
</ul>
<ul>
<li>By this time, the managers have started to try to sell the product and they find, shockingly, that no-one wants it. Undaunted and courageously they build million-dollar web sites and ad campaigns to explain to the public why they absolutely need this product. They do deals with other businesses to force the product on the lazy, stupid and ungrateful market.</li>
</ul>
<ul>
<li>After twelve months of intense marketing, the product still isn't making profits. Worse, it suffers dramatic failures and gets branded in the press as a disaster. The company quietly shelves it, fires the consultants, buys a competing product from a small start-up and rebrands that as its own Version 2. Hundreds of millions of dollars end up in the trash.</li>
</ul>
<ul>
<li>Meanwhile, another visionary manager, somewhere in the organization, drinks a little too much tequila with some marketing people and has a Brilliant Idea.</li>
</ul>
<p>TOD would be a caricature if it wasn't so common. Something like 19 out of 20 market-ready products built by large firms are failures. The remaining 1 in 20 probably only succeeds because the competitors are so bad.</p>
<p>The main lessons of TOD are quite straight-forward but hard to swallow. They are:</p>
<ul>
<li>Ideas are cheap. No exceptions. There are no brilliant ideas. Anyone who tries to start a discussion with "oooh, we can do this too!" should be beaten down with all the passion one reserves for traveling musicians. It is like sitting in a cafe at the foot of a mountain, drinking a hot chocolate and telling others, "hey, I have a great idea, we can climb that mountain! And build a chalet on top! With two saunas! And a garden! Hey, and we can make it solar powered! Dude, that's awesome! What color should we paint it? Green! No, blue! OK, go and make it, I'll stay here and make spreadsheets and graphics!"</li>
</ul>
<ul>
<li>The starting point for a good design process is to collect problems that confront people. The second step is to evaluate these problems with the basic question, "how much is it worth to solve this problem?" Having done that, we can collect a set of problems that are worth solving.</li>
</ul>
<ul>
<li>Good solutions to real problems will succeed as products. Their success will depend on how good and cheap the solution is, and how important the problem is. But their success will also depend on how much they demand in effort to use, in other words how simple they are.</li>
</ul>
<p>Hence after slaying the dragon of utter irrelevance, we attack the demon of complexity.</p>
<h2 id="toc1"><span>Complexity-Oriented Design</span></h2>
<p>Really good engineering teams and small firms can usually build good products. But the vast majority of products still end up being too complex and less successful than they might be. This is because specialist teams, even the best, often stubbornly apply a process I call "Complexity-Oriented Design", or COD, which works as follows:</p>
<ul>
<li>Management correctly identifies some interesting and difficult problem with economic value. In doing so they already leapfrog over any TOD team.</li>
</ul>
<ul>
<li>The team with enthusiasm start to build prototypes and core layers. These work as designed and thus encouraged, the team go off into intense design and architecture discussions, coming up with elegant schemas that look beautiful and solid.</li>
</ul>
<ul>
<li>Management comes back and challenges team with yet more difficult problems. We tend to equate value with cost, so the harder the problem, and more expensive to solve, the more the solution should be worth, in their minds.</li>
</ul>
<ul>
<li>The team, being engineers and thus loving to build stuff, build stuff. They build and build and build and end up with massive, perfectly designed complexity.</li>
</ul>
<ul>
<li>The products go to market, and the market scratches its head and asks, "seriously, is this the best you can do?" People do use the products, especially if they aren't spending their own money in climbing the learning curve.</li>
</ul>
<ul>
<li>Management gets positive feedback from its larger customers, who share the same idea that high cost (in training and use) means high value. and so continues to push the process.</li>
</ul>
<ul>
<li>Meanwhile somewhere across the world, a small team is solving the same problem using SOD, and a year later smashes the market to little pieces.</li>
</ul>
<p>COD is characterized by a team obsessively solving the wrong problems to the point of ridiculousness. COD products tend to be large, ambitious, complex, and unpopular. Much open source software is the output of COD processes. It is insanely hard for engineers to <strong>stop</strong> extending a design to cover more potential problems. They argue, "what if someone wants to do X?" but never ask themselves, "what is the real value of solving X?"</p>
<p>A good example of COD in practice is Bluetooth, a complex, over-designed set of protocols that users hate. It continues to exist only because there are no alternatives. Bluetooth is perfectly secure, which is close to useless for a proximity protocol. At the same time it lacks a standard API for developers, meaning it's really costly to use Bluetooth in applications.</p>
<p>On the #zeromq IRC channel, Wintre wrote of how enraged he was many years ago when he "<em>found that XMMS 2 had a working plugin system but could not actually play music.</em>"</p>
<p>COD is a form of large-scale "rabbit holing", in which designers and engineers cannot distance themselves from the technical details of their work. They add more and more features, utterly misreading the economics of their work.</p>
<p>The main lessons of COD are also simple but hard for experts to swallow. They are:</p>
<ul>
<li>Making stuff that you don't immediately have a need for is pointless. Doesn't matter how talented or brilliant you are, if you just sit down and make stuff, you are most likely wasting your time.</li>
</ul>
<ul>
<li>Problems are not equal. Some are simple, and some are complex. Ironically, solving the simpler problems often has more value to more people than solving the really hard ones. So if you allow engineers to just work on random things, they'll most focus on the most interesting but least worthwhile things.</li>
</ul>
<ul>
<li>Engineers and designers love to make stuff and decoration, and this inevitably leads to complexity. It is crucial to have a "stop mechanism", a way to set short, hard deadlines that force people to make smaller, simpler answers to just the most crucial problems.</li>
</ul>
<h2 id="toc2"><span>Simplicity-Oriented Design</span></h2>
<p>Simplicity-Oriented Design starts with a realization: we do not know what we have to make until after we start making it. Coming up with ideas, or large-scale designs isn't just wasteful, it's directly toxic to designing the truly accurate solutions. The really juicy problems are hidden like far valleys, and any activity except active scouting creates a fog that hides those distant valleys. You need to keep mobile, pack light, and move fast.</p>
<p>SOD works as follows:</p>
<ul>
<li>We collect a set of interesting problems (by looking at how people use technology or other products) and we line these up from simple to complex, looking for and identifying patterns of use.</li>
</ul>
<ul>
<li>We take the simplest, most dramatic problem and we solve this with a minimal plausible solution, or "patch". Each patch solves exactly a genuine and agreed problem in a brutally minimal fashion.</li>
</ul>
<ul>
<li>We apply one measure of quality to patches, namely "can this be done any simpler while still solving the stated problem?" We can measure complexity in terms of concepts and models that the user has to learn or guess in order to use the patch. The fewer, the better. A perfect patch solves a problem with zero learning required by the user.</li>
</ul>
<ul>
<li>Our product development consists of a patch that solves the problem "we need a proof of concept" and then evolves in an unbroken line to a mature series of products, through hundreds or thousands of patches piled on top of each other.</li>
</ul>
<ul>
<li>We do not do <em>anything</em> that is not a patch. We enforce this rule with formal processes that demand that every activity or task is tied to a genuine and agreed problem, explicitly enunciated and documented.</li>
</ul>
<ul>
<li>We build our projects into a supply chain where each project can provide problems to its "suppliers" and receive patches in return. The supply chain creates the "stop mechanism" since when people are impatiently waiting for an answer, we necessarily cut our work short.</li>
</ul>
<ul>
<li>Individuals are free to work on any projects, and provide patches at any place they feel it's worthwhile. No individuals "own" any project, except to enforce the formal processes. A single project can have many variations, each a collection of different, competing patches.</li>
</ul>
<ul>
<li>Projects export formal and documented interfaces so that upstream (client) projects are unaware of change happening in supplier projects. Thus multiple supplier projects can compete for client projects, in effect creating a free and competitive market.</li>
</ul>
<ul>
<li>We tie our supply chain to real users and external clients and we drive the whole process by rapid cycles so that a problem received from outside users can be analyzed, evaluated, and solved with a patch in a few hours.</li>
</ul>
<ul>
<li>At every moment from the very first patch, our product is shippable. This is essential, because a large proportion of patches will be wrong (10-30%) and only by giving the product to users can we know which patches have become problems and themselves need solving.</li>
</ul>
<p>SOD is a form of "hill climbing algorithm", a reliable way of finding optimal solutions to the most significant problems in an unknown landscape. You don't need to be a genius to use SOD successfully, you just need to be able to see the difference between the fog of activity and the progress towards new real problems.</p>
<p>A really good designer with a good team can use SOD to build world-class products, rapidly and accurately. To get the most out of SOD, the designer has to use the product continuously, from day 1, and develop his or her ability to smell out problems such as inconsistency, surprising behavior, and other forms of friction. We naturally overlook many annoyances but a good designer picks these up, and thinks about how to patch them. Design is about removing friction in the use of a product.</p>
<h2 id="toc3"><span>Conclusions</span></h2>
<p>There are many aspects to getting product-building teams and organizations to think wisely. You need diversity, freedom, challenge, resources, and so on. I discuss these in detail in my forthcoming book, Culture and Empire. However, even if you have all the right ingredients, the default processes that skilled engineers and designers develop will result in complex, hard-to-use products.</p>
<p>The classic errors are to focus on ideas, not problems; to focus on the wrong problems; to misjudge the value of solving problems; not using ones' own work; and in many other ways to misjudge the real market.</p>
<p>Simplicity Oriented Design is a reliable, repeatable way of developing world-class products that delight users with their simplicity and elegance. This process organizes people into flexible supply chains that are able to navigate a problem landscape rapidly and cheaply. They do this by building, testing, and keeping or discarding minimal plausible solutions, called "patches". Living products consist of long series of patches, applied one atop the other.</p>
<p>SOD works best for software design, and some open source projects already work in this way. However many or most still fall into classic traps such as over-engineering and "what if" design. Wikipedia is a good example of SOD applied to the non-software domain.</p>
<p>I use SOD daily to build, and help others build, world-class commercial products. If you find SOD interesting and useful, read Culture and Empire when it comes out, particularly chapter two, where I explain the science of Social Architecture.</p>
<p><em>This article is an extract from Chapter 6 of the ZeroMQ book, on community building. If you like this, buy the book :-)</em></p>