5 changed files with 1498 additions and 0 deletions
+ 362
- 0
cache/2021/6514143dca5d96bf9e751236b800fba5/index.html
View File
| @@ -0,0 +1,362 @@ | |||
| <!doctype html><!-- This is a valid HTML5 document. --> | |||
| <!-- Screen readers, SEO, extensions and so on. --> | |||
| <html lang="fr"> | |||
| <!-- Has to be within the first 1024 bytes, hence before the <title> | |||
| See: https://www.w3.org/TR/2012/CR-html5-20121217/document-metadata.html#charset --> | |||
| <meta charset="utf-8"> | |||
| <!-- Why no `X-UA-Compatible` meta: https://stackoverflow.com/a/6771584 --> | |||
| <!-- The viewport meta is quite crowded and we are responsible for that. | |||
| See: https://codepen.io/tigt/post/meta-viewport-for-2015 --> | |||
| <meta name="viewport" content="width=device-width,initial-scale=1"> | |||
| <!-- Required to make a valid HTML5 document. --> | |||
| <title>Are We Really Engineers? (archive) — David Larlet</title> | |||
| <meta name="description" content="Publication mise en cache pour en conserver une trace."> | |||
| <!-- That good ol' feed, subscribe :). --> | |||
| <link rel="alternate" type="application/atom+xml" title="Feed" href="/david/log/"> | |||
| <!-- Generated from https://realfavicongenerator.net/ such a mess. --> | |||
| <link rel="apple-touch-icon" sizes="180x180" href="/static/david/icons2/apple-touch-icon.png"> | |||
| <link rel="icon" type="image/png" sizes="32x32" href="/static/david/icons2/favicon-32x32.png"> | |||
| <link rel="icon" type="image/png" sizes="16x16" href="/static/david/icons2/favicon-16x16.png"> | |||
| <link rel="manifest" href="/static/david/icons2/site.webmanifest"> | |||
| <link rel="mask-icon" href="/static/david/icons2/safari-pinned-tab.svg" color="#07486c"> | |||
| <link rel="shortcut icon" href="/static/david/icons2/favicon.ico"> | |||
| <meta name="msapplication-TileColor" content="#f0f0ea"> | |||
| <meta name="msapplication-config" content="/static/david/icons2/browserconfig.xml"> | |||
| <meta name="theme-color" content="#f0f0ea"> | |||
| <!-- Documented, feel free to shoot an email. --> | |||
| <link rel="stylesheet" href="/static/david/css/style_2021-01-20.css"> | |||
| <!-- See https://www.zachleat.com/web/comprehensive-webfonts/ for the trade-off. --> | |||
| <link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_regular.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
| <link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_bold.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
| <link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_italic.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
| <link rel="preload" href="/static/david/css/fonts/triplicate_t3_regular.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
| <link rel="preload" href="/static/david/css/fonts/triplicate_t3_bold.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
| <link rel="preload" href="/static/david/css/fonts/triplicate_t3_italic.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
| <script> | |||
| function toggleTheme(themeName) { | |||
| document.documentElement.classList.toggle( | |||
| 'forced-dark', | |||
| themeName === 'dark' | |||
| ) | |||
| document.documentElement.classList.toggle( | |||
| 'forced-light', | |||
| themeName === 'light' | |||
| ) | |||
| } | |||
| const selectedTheme = localStorage.getItem('theme') | |||
| if (selectedTheme !== 'undefined') { | |||
| toggleTheme(selectedTheme) | |||
| } | |||
| </script> | |||
| <meta name="robots" content="noindex, nofollow"> | |||
| <meta content="origin-when-cross-origin" name="referrer"> | |||
| <!-- Canonical URL for SEO purposes --> | |||
| <link rel="canonical" href="https://www.hillelwayne.com/post/crossover-project/are-we-really-engineers/"> | |||
| <body class="remarkdown h1-underline h2-underline h3-underline em-underscore hr-center ul-star pre-tick"> | |||
| <article> | |||
| <header> | |||
| <h1>Are We Really Engineers?</h1> | |||
| </header> | |||
| <nav> | |||
| <p class="center"> | |||
| <a href="/david/" title="Aller à l’accueil"><svg class="icon icon-home"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-home"></use> | |||
| </svg> Accueil</a> • | |||
| <a href="https://www.hillelwayne.com/post/crossover-project/are-we-really-engineers/" title="Lien vers le contenu original">Source originale</a> | |||
| </p> | |||
| </nav> | |||
| <hr> | |||
| <p>I sat in front of Mat, idly chatting about tech and cuisine. Before now, I had known him mostly for his cooking pictures on Twitter, the kind that made me envious of suburbanites and their 75,000 BTU woks. But now he was the test subject for my new project, to see if it was going to be fruitful or a waste of time.</p> | |||
| <p>“What’s your job?”</p> | |||
| <p>“Right now I’m working on microservices for a social media management platform.”</p> | |||
| <p>“And before that?”</p> | |||
| <p>“Geological engineering. A lot of open pit mining, some amount of underground tunnel work. Hydropower work. Earth embankment dams because they come along with mines.”</p> | |||
| <p>He told me a story about his old job. His firm was hired to analyze a block cave in British Columbia. Block caves are a kind of mining project where you dig tunnels underneath the deposit to destabilize it. The deposit slowly collapses and leaks material into the tunnels, and then “you just print money”, as Mat called it. The big problem here? The block cave was a quarter mile under a rival company’s toxic waste dump. “In the event of an earthquake, could the waste flood the mine and kill everyone?” He had to prove it was safe. A different kind of work than what he was doing now.</p> | |||
| <p>As for another difference? “My personal blog has better security than some $100 million mining projects.”</p> | |||
| <p>This wasn’t going to be a waste of time, after all.</p> | |||
| <hr> | |||
| <p>Is software engineering “really” engineering? A lot of us call ourselves software engineers. Do we deserve that title? Are we mere pretenders to the idea of engineering? This is an important question, and like all important questions, it regularly sparks arguments online. On one hand you have the people who say we are not engineers because we do not live up to “engineering standards”. These people point to things like <a href="https://www.theatlantic.com/technology/archive/2017/09/saving-the-world-from-code/540393/">The Coming Software Apocalypse</a> as evidence that we don’t have it together. They say that we need things like certification, licensing, and rigorous design if we want to earn the title engineer.</p> | |||
| <p>On the other end of the horseshoe, we have people like Pete McBreen and Paul Graham who say that we are not engineers because engineering cannot apply to our domain. Engineers work on predictable projects with a lot of upfront planning and rigorous requirements. Software is dynamic, constantly changing, unpredictable. If we try to apply engineering practice to software then software would be 10 times as expensive and stuck in 1970.</p> | |||
| <p>For a long time I refused to call myself a software engineer and thought of other people with that title as poseurs. It was reading a short story against engineering that made me question my core assumptions. It was <a href="http://thecodelesscode.com/case/154">A Bridge to Nowhere</a>, one of the <a href="http://thecodelesscode.com/">Codeless Code stories</a>. The author argues that the techniques of bridge building don’t apply to software, since software clients can change their requirements and software gravity can sometimes reverse itself.</p> | |||
| <blockquote> | |||
| <p>The predictability of a true engineer’s world is an enviable thing. But ours is a world always in flux, where the laws of physics change weekly. If we did not quickly adapt to the unforeseen, the only foreseeable event would be our own destruction.</p> | |||
| </blockquote> | |||
| <p>That’s when I realized something about everybody involved in all of these arguments.</p> | |||
| <p>They’ve never built a bridge.</p> | |||
| <p>Nobody I read in these arguments, not one single person, ever worked as a “real” engineer. At best they had some classical training in the classroom, but we all know that looks nothing like reality. Nobody in this debate had anything more than stereotypes to work with. The difference between the engineering in our heads and in reality has been noticed by others before, most visibly by <a href="https://www.youtube.com/watch?v=RhdlBHHimeM">Glenn Vanderburg</a>. He read books on engineering to figure out the difference. But I wanted to go further.</p> | |||
| <p>If I wanted to know how software development compares and contrasts with “real” engineering, then I’d have to talk to “real” engineers. But thinking about that, I realized another problem: while “real” engineers could tell me what they did, they couldn’t tell me how their work <em>differed</em> from mine. Just as we make sweeping statements about real engineering without any evidence, they could make sweeping statements about software without knowing anything about it either. Even if they use a bit of software in the day-to-day work, that’s not the same as developing software as a job. Only a person who’s done <em>both</em> software development and “real” engineering can truthfully speak to the differences between them.</p> | |||
| <p>So that’s what I set out to find: people who used to be professional engineers and then became professional software developers. I call these people <strong>crossovers</strong>, hybrids between the two worlds. I interviewed 17 crossovers on common software misconceptions, how the two worlds relate to each other, whether we can truthfully call what we do engineering, and what the different fields can teach and learn from each other.</p> | |||
| <p>There’s a lot I want to talk about here, much more than can comfortably fit in one blog post. I divided the write up into three parts, dealing with my three core topics. Part one is about the term “engineering”. Is what we do engineering, and can we honestly call ourselves engineers?</p> | |||
| <h2 id="why-is-this-a-question">Why is this a Question?</h2> | |||
| <p>So first some ground rules: “software engineering” is a real field. Most people would agree that the software running on spacecraft counts as “real engineering”. The debate is about rest of the field. Are the people who build websites engineers? What about embedded hardware? I found in my research that people defending the title “engineer” rarely coherently define what an engineer <em>is</em>, usually boiling it down to “engineers solve problems”. The arguments against the title tend to be a little more developed, as are the arguments about transcending it.</p> | |||
| <p>The backlash against “software engineering” comes from two places. First, there are the gatekeepers. While at first we called ourselves programmers and “developers”, after 1985 more and more people started using the title “software engineer”. This “title bloat” is one of the things that led to the backlash: people using it as a title without trying to live up to some form of standards.</p> | |||
| <p>The other side of the backlash comes from the new software movements of the 90s and early 00s. Movements like Extreme Programming and Agile were rejections of applying standard project management techniques to software development. To them, engineering represented the old way of doing things, incompatible with the new way of software development. Software engineering was the past, a failed dead end, and artisanal software was the future.</p> | |||
| <p>Both of these backlashes come from the connotations of engineering. In the case of gatekeepers, it’s the positive connotations they believe we have not earned. In the case of artisanal craftship, it’s the negative baggage they believe we don’t deserve. Both of these are agenda-driven viewpoints, arguments based on how they want software to be. This is good for advocacy but doesn’t help us figure out where software is right now. And their agendas are based on a reference point they don’t understand themselves. None of the people arguing for or against software engineering as engineering have worked as engineers.</p> | |||
| <h2 id="common-misconceptions">Common Misconceptions</h2> | |||
| <p>Let’s start with common arguments that people make about whether to include software in engineering. Most people have an informal concept of what engineering is but not a strict definition. Like pornography, people know what engineering is when they see it. When I asked people to explicitly list the qualities of what makes something engineering, here are the most common responses:</p> | |||
| <ol> | |||
| <li>Making something complex as part of a team.</li> | |||
| <li>Making something physical.</li> | |||
| <li>Making something according to a rigorous spec with strict principles.</li> | |||
| <li>Using mathematical principles in their design.</li> | |||
| <li>Working on situations with high consequences, like loss of life.</li> | |||
| <li>Doing work with a certification and license.</li> | |||
| </ol> | |||
| <p>The first quality is too broad: almost all human professions involve making something complex in a team. It doesn’t capture “engineering”. The second quality is too restrictive: not all forms of engineering result in physical processes. In particular, industrial engineering rarely does. The third quality will be discussed in the next essay.</p> | |||
| <p>That leaves three claims to discuss here. All three are used to say software can’t be engineering. “We don’t use any math, unlike <em>real</em> engineers. Software doesn’t matter, unlike <em>real</em> engineers. We aren’t licensed, unlike <em>real</em> engineers.” As we’ll see, none of these quite hold up.</p> | |||
| <h3 id="engineering-is-mathematical">Engineering is mathematical</h3> | |||
| <p>This one I can address directly. The claim is that engineering involves a lot of hard math, while software involves very little math. The confusion here comes from our misunderstanding of mathematics. Much of the math that mechanical engineers use is <strong>continuous math</strong>. This is where we work over a continuous domain, like real numbers. Things like calculus, trigonometry, and differential equations are in this category. This is what most people in the US learn in high school, codifying it as what they think of as “math”.</p> | |||
| <p>In software, we don’t use these things, leading to the conception that we don’t use math. But we actually use <strong>discrete math</strong>, where we deal exclusively with non-continuous numbers. This includes things like graph theory, logic, and combinatorics. You might not realize that you are using these, but you do. They’re just so internalized in software that we don’t see them as math! In fact most of computer science is viewable as a branch of mathematics. Every time you simplify a conditional or work through the performance complexity of an algorithm, you are using math. Just because there are no integrals doesn’t mean we are mathless.</p> | |||
| <p>This falls in line with the rest of engineering. Different branches use different kinds of math in different ways. Industrial engineers are concerned with very different things than mechanical engineers are. Just because we use a different branch of math doesn’t mean we’re not doing engineering.</p> | |||
| <h3 id="engineering-is-high-consequence">Engineering is high-consequence</h3> | |||
| <p>I’m not sure what first disabused me of this notion. It might have been the conversation with Mike, an ex-mechanical engineer who now designs glaciology sensors. One of his last projects before the crossover was as part of a team working on a classified medical device. Most of it went smoothly, but then came the big blocker:</p> | |||
| <blockquote> | |||
| <p>The one the thing that gave us the most grief was the handle, which was a bent bit of wire with some plastic molded around it. Yeah, getting the wires to bend correctly and the plastic onto this shiny piece of anodized aluminium wire. […] That was the thing that took three months in the project.</p> | |||
| </blockquote> | |||
| <p>Three months of the project dedicated to getting the handle working right.</p> | |||
| <p>This is not to say the work wasn’t important. It does mean that the engineers spent a lot of time on something that’s low-consequence or non-safety critical. In retrospect, this shouldn’t have surprised me. The world is vast and our industry is huge. Someone needs to design the bridges, yes, but someone also needs to design all of the little things we use in our day-to-day life. Much of the engineering there is low-stakes, low-consequence, just like much software is.</p> | |||
| <blockquote> | |||
| <p>There’s a huge difference between if you’re making a cabinet for Bluetooth speakers, like ones that are on my desk, or you’re making an assembly for the landing gear of 737. You’re using some of the same tools, but your approach is wildly different. <em>- Nathan (mechanical)</em></p> | |||
| </blockquote> | |||
| <p>«But there are still <em>some</em> things that lead to loss of life!» Yes, and the same is true of software. An integer overflow <a href="https://www.theverge.com/2014/10/20/7014705/coding-error-911-fcc-washington">in the Intrado software</a> lead to a several hour 911 outage for millions of people. A biased algorithm <a href="https://www.technologyreview.com/2019/01/21/137783/algorithms-criminal-justice-ai/">unfairly sent people to jail</a>. Just like with “real” engineering, there’s also a wide swath of software that won’t kill people but will lead to millions of dollars in damages. This can be everything from Amazon going down to a game being unplayable.</p> | |||
| <h3 id="engineers-are-licensed">Engineers are licensed</h3> | |||
| <p>This is the claim I’ve heard most: licensure. There’s a difference between doing engineering and being an engineer, just as people practice medicine at home without being doctors. Maybe software engineering is possible, but without the licenses we are not software <em>engineers</em>. In Canada you can’t even call yourself a “Software Engineer” unless you’re accredited!</p> | |||
| <p>At least that’s what people in the United States say. Several Europeans I spoke to said the exact same thing about US system. Everybody seems to think the worst about their own system and the best about others. And about half of the engineers I spoke to weren’t licensed but were still considered professional engineers by their peers. They didn’t have the “certification”, but they had the <em>skills</em>.</p> | |||
| <p>In the US, you don’t need a license to practice any kind of engineering. You need a license to be a “principal engineer”, a.k.a. the person who formally signs off on plans as valid. But the engineers working under the principal engineer don’t need to be accredited and often are not. In fact many of them don’t even have formal training as engineers.</p> | |||
| <p>Here’s the problem with deciding engineering based on licenses: licenses are a political and social construct, not a fact of nature. Societies adopt licensure for reasons that are as much political as technical. To see this, let’s discuss some of the history of licensure in the United States.</p> | |||
| <p>As recently as 1906, not a single state in the US required licenses for any project. <a href="https://en.wikipedia.org/wiki/Regulation_and_licensure_in_engineering#United_States">Wyoming</a> was the first state to instate a licensure policy, entirely because irrigation projects kept blowing the budget. They theorized accredited engineers could give better estimates on project costs. It took 40 more years for all fifty states to get on board.</p> | |||
| <p>This all means that licensure isn’t part of the federal government: it’s a requirement by the states. Among other things, licenses aren’t necessarily transferable between states. If you get a license in Texas, you have to go through a process to be able to practice in California, in what’s called “comity”.</p> | |||
| <p>While licensure originated as part of cost overruns, it expanded so rapidly for an entirely different reason. Regulations are written in blood. Fields become regulated when the lack of regulation kills people. Until that happens on a wide scale with arbitrary programs, it’s unlikely that we’ll ever see the same licensure requirements for software. In the places where software <em>has</em> killed people, like Therac-25 and aircraft accidents, we see stricter regulations. Whether this leads to broader licensure requirements for software engineers remains to be seen.</p> | |||
| <p>You could argue that it’s immoral for us to not be licensed. This is an argument I’m sympathetic to. But it’s a normative argument, not a positive one. By saying “we should be licensed”, you are saying something about how the world <em>ought to</em> be. You are trying to answer the question “should we be held to higher standards?” But that’s not the question here. I don’t care right now where we should be going; I just want to know where we are right now. Whether or not we are engineers is irrelevant to whether or not we are <em>good</em> engineers.</p> | |||
| <p>In conclusion: licenses exist because we are part of society and have legal requirements, not because they are essential to what it means to do engineering. So while you might want to make software more licensed, as it stands the licensing question doesn’t change the essence of our work.</p> | |||
| <h2 id="the-truth">The Truth</h2> | |||
| <p>This leaves us back where we started: there aren’t any qualities we can point to and say “this is engineering, this is not”. It’s a standard Wittgenstein game problem: human constructs do not neatly fall into precise definitions. Rather something like “engineering” is a family of related concepts, where we judge whether something belongs based on how much it resembles other things in the family. In other words, “engineering” is “what engineers do”. In other other words, something becomes engineering if enough engineers say it’s engineering.</p> | |||
| <p>Consider chemical engineering. Chemical engineering is unlike mechanical, civil, or electrical engineering. Chemical engineers create processes to manufacture products at scale, often using experimentation and iteration. But nobody would disagree that it is engineering. Chemical engineering started in the late 1800s, before states licensed engineers. If chemical engineering started now, people would refuse to call it engineering. And they’d be wrong to refuse.</p> | |||
| <p>Once I realized this, my interviewing process changed a little. Instead of asking how they felt about certain engineering topics, I just asked them point blank. “Do you consider software engineering actually engineering?”</p> | |||
| <p>Of the 17 crossovers I talked to, 15 said yes.</p> | |||
| <p>That’s not the answer I expected going in. I assumed we weren’t engineers, that we’re actually very far from being engineers. But then again, I was never a “real” engineer. I don’t know what it’s like to be a “real” engineer, and so can’t compare software engineering to other forms. I don’t have the experience. These people did, and they considered software engineering real engineering.</p> | |||
| <blockquote> | |||
| <p>Even the product owners and project managers are in a sense engineers […] everyone is kind of an engineer, an engineer of sorts. <em>-Kate (chemical)</em></p> | |||
| </blockquote> | |||
| <p>It makes no sense to use “real” engineering in contrast to “software” engineering. Going forward I will instead use the term “trad” engineering.</p> | |||
| <h3 id="craft-vs-engineering">Craft vs Engineering</h3> | |||
| <blockquote> | |||
| <p>I’m gonna respond in a slightly different way. Not every software developer is a software engineer, just like not every single person who works in construction is an engineer. An engineer is a very specific set of skills. […] Software [engineering] is skill plus understanding, all the processes and life cycle and the consequences and all the things that you should be aware of [and] avoid. <em>-Dawn (Mechanical & Chemical)</em></p> | |||
| </blockquote> | |||
| <p>That said, many of the crossovers also added an additional qualification: software engineering is real engineering, but a lot of people who write software aren’t doing software engineering. This is not a problem with them, rather a problem with our field: we don’t have a rich enough vocabulary to talk about what these developers do. Not everybody who works with electricity is going to be an electrical engineer; many will be electricians. And this is okay. Electrical engineering is a very narrow skill set in the broader field of electric professions and plenty of people have other important skills in that space. But we use things like “programmer”, “software engineer”, and “software developer” interchangeably. What is the difference between a software engineer and a software developer?</p> | |||
| <p>Some people propose the word “software craftsman”. This term comes from the book <a href="https://learning.oreilly.com/library/view/software-craftsmanship-the/0201733862/">Software Craftsmanship: The New Imperative</a>, by Pete McBreen. In it he argues that software is not a kind of engineering, being much more free-form creative and flexible. We’re not line workers but artisans, artists, people who take pride in the craft we do and the flexibility of our states. As he puts it:</p> | |||
| <blockquote> | |||
| <p>Software development is all about the unknown. The production process for software is trivially easy—just copy a disk or CD. The software engineering metaphor fails because we understand production, a mechanical task, much better than we understand design, an intellectual task. <em>- <a href="https://learning.oreilly.com/library/view/software-craftsmanship-the/0201733862/ch06.html">Software Craftsmanship</a></em></p> | |||
| </blockquote> | |||
| <p>Many people have asked me why I care so much about this project. Why does it matter whether or not software is “really” engineering? Why can’t we just say that “software is software”? It’s because of these misconceptions. People have a stereotyped notion of what engineering looks like. Because software doesn’t look like the stereotype, they assume that we are wholly unlike engineering. The engineering disciplines have nothing to teach us. We are breaking pristine ground and have no broader history to guide us.</p> | |||
| <p>In contrast, if we are doing engineering, then we have a landmark. We can meaningfully compare and contrast the work we do as software engineers from the work that others do as traditional engineers. We can adapt their insights and watch for their pitfalls. We can draw upon the extant knowledge of our society to make better software.</p> | |||
| <p>I believe everything McBreen said about software is fairly reasonable, about how hard it is to predict things and how it’s intensely personally created. What he gets wrong is the assumption that engineering is <em>not</em> this way. Engineering is much richer, more creative, and more artistic than he thought. But of course he would have an imperfect view: he is, after all, not a traditional engineer.</p> | |||
| <hr> | |||
| <p>Here’s my final take. This is the belief I’ve settled on in synthesizing all the interviews I did and does not necessarily reflect how the crossovers think. I went into this thinking that software wasn’t really engineering. Maybe there were a few people who could count themselves as that but most of us were far below that threshold. I still believe that most of us are not engineers, because we’re working in domains that people don’t see as engineering. Most people don’t consider a website “engineered”. <em>However</em>, and this is a big however, there’s a much smaller gap between “software development” and “software engineering” than there is between “electrician” and “electrical engineer”, or between “trade” and “engineering” in all other fields. Most people can go between “software craft” and “software engineering” without significant retraining. We are separated from engineering by circumstance, not by essence, and we can choose to bridge that gap at will.</p> | |||
| <p>In the next essay, we’ll talk about the similarities and differences between traditional engineering and software engineering, and how they’re actually not all that different in the end.</p> | |||
| <p><em>Part two,</em> <a href="https://www.hillelwayne.com/post/crossover-project/we-are-not-special/">We Are Not Special</a>, <em>is available <a href="https://www.hillelwayne.com/post/crossover-project/we-are-not-special/">here</a>. If you enjoyed these essays, I have a <a href="https://buttondown.email/hillelwayne/">weekly newsletter</a> and am on <a href="https://twitter.com/hillelogram">Twitter</a>.</em></p> | |||
| <p><em>Thanks to | |||
| Glenn Vanderburg | |||
| , | |||
| <a href="https://chelseatroy.com/">Chelsea Troy</a> | |||
| , | |||
| <a href="https://twitter.com/craftworksxyz">Will Craft</a> | |||
| , and | |||
| <a href="https://www.danluu.com">Dan Luu</a> | |||
| for feedback, and to all of the engineers whom I interviewed.</em></p> | |||
| <h2 id="appendix-methodology">Appendix: Methodology</h2> | |||
| <p>I only searched out people who work professionally for at least one year in each field. In practice, work experience in trad engineering ranged from a year and a half on the low end to over 15 years on the high end. Interviews ranged from half an hour to two hours, leading to a total of 24 hours of recorded interview. Two of the interviewees were not recorded, but I took notes. The breakdown of the specialties is:</p> | |||
| <p><img src="/post/crossover-project/img/chart.png" alt="Pie chart of the engineer types"></p> | |||
| <ul> | |||
| <li>Civil engineers work on buildings. One person I talked to was a classic civil engineer, one specialized in designing mines, and two worked on oil rigs.</li> | |||
| <li>Mechanical engineers design physical machines.</li> | |||
| <li>Electrical engineers design circuits and electronics. Two worked on chipsets, while one was embedded in a submarine.</li> | |||
| <li>Chemical engineers create processes to make chemicals at scale. They are generally not creating entirely new chemicals or chemical products, but work on how to produce extant ones. “Chemicals” is a broad category here, covering everything from clean water to toothpaste.</li> | |||
| <li>Industrial Engineers create holistic systems and procedures in an industry. One designed data center layouts and the other worked on integrating disparate air traffic systems.</li> | |||
| </ul> | |||
| <p>These are very broad summaries of the specialties. Most engineers work in a subdomain of a specialty, such as automotive engineering or circuit design. There were also some fields of engineering I didn’t cover in my interviews. This includes aerospace and nuclear engineering. I suspect (<em>suspect</em>) that aerospace would be roughly similar to mechanical engineering and nuclear engineering would be roughly similar to civil engineering. But it remains a threat to validity nonetheless. </p> | |||
| <p>The other two threats to validity are geographic location and crossover type. The majority of the interviewees were either in the US or the UK, with the rest being from the EU or Canada. I did not get a chance to interview anybody from Latin America, South America, Africa, or Asia. Everybody interviewed crossed from traditional engineering to software engineering. I was not able to find anyone who crossed the other way.</p> | |||
| </article> | |||
| <hr> | |||
| <footer> | |||
| <p> | |||
| <a href="/david/" title="Aller à l’accueil"><svg class="icon icon-home"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-home"></use> | |||
| </svg> Accueil</a> • | |||
| <a href="/david/log/" title="Accès au flux RSS"><svg class="icon icon-rss2"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-rss2"></use> | |||
| </svg> RSS</a> • | |||
| <a href="http://larlet.com" title="Go to my English profile" data-instant><svg class="icon icon-user-tie"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-user-tie"></use> | |||
| </svg> Pro</a> • | |||
| <a href="mailto:david%40larlet.fr" title="Envoyer un courriel"><svg class="icon icon-mail"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-mail"></use> | |||
| </svg> Email</a> • | |||
| <abbr class="nowrap" title="Hébergeur : Alwaysdata, 62 rue Tiquetonne 75002 Paris, +33184162340"><svg class="icon icon-hammer2"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-hammer2"></use> | |||
| </svg> Légal</abbr> | |||
| </p> | |||
| <template id="theme-selector"> | |||
| <form> | |||
| <fieldset> | |||
| <legend><svg class="icon icon-brightness-contrast"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-brightness-contrast"></use> | |||
| </svg> Thème</legend> | |||
| <label> | |||
| <input type="radio" value="auto" name="chosen-color-scheme" checked> Auto | |||
| </label> | |||
| <label> | |||
| <input type="radio" value="dark" name="chosen-color-scheme"> Foncé | |||
| </label> | |||
| <label> | |||
| <input type="radio" value="light" name="chosen-color-scheme"> Clair | |||
| </label> | |||
| </fieldset> | |||
| </form> | |||
| </template> | |||
| </footer> | |||
| <script> | |||
| function loadThemeForm(templateName) { | |||
| const themeSelectorTemplate = document.querySelector(templateName) | |||
| const form = themeSelectorTemplate.content.firstElementChild | |||
| themeSelectorTemplate.replaceWith(form) | |||
| form.addEventListener('change', (e) => { | |||
| const chosenColorScheme = e.target.value | |||
| localStorage.setItem('theme', chosenColorScheme) | |||
| toggleTheme(chosenColorScheme) | |||
| }) | |||
| const selectedTheme = localStorage.getItem('theme') | |||
| if (selectedTheme && selectedTheme !== 'undefined') { | |||
| form.querySelector(`[value="${selectedTheme}"]`).checked = true | |||
| } | |||
| } | |||
| const prefersColorSchemeDark = '(prefers-color-scheme: dark)' | |||
| window.addEventListener('load', () => { | |||
| let hasDarkRules = false | |||
| for (const styleSheet of Array.from(document.styleSheets)) { | |||
| let mediaRules = [] | |||
| for (const cssRule of styleSheet.cssRules) { | |||
| if (cssRule.type !== CSSRule.MEDIA_RULE) { | |||
| continue | |||
| } | |||
| // WARNING: Safari does not have/supports `conditionText`. | |||
| if (cssRule.conditionText) { | |||
| if (cssRule.conditionText !== prefersColorSchemeDark) { | |||
| continue | |||
| } | |||
| } else { | |||
| if (cssRule.cssText.startsWith(prefersColorSchemeDark)) { | |||
| continue | |||
| } | |||
| } | |||
| mediaRules = mediaRules.concat(Array.from(cssRule.cssRules)) | |||
| } | |||
| // WARNING: do not try to insert a Rule to a styleSheet you are | |||
| // currently iterating on, otherwise the browser will be stuck | |||
| // in a infinite loop… | |||
| for (const mediaRule of mediaRules) { | |||
| styleSheet.insertRule(mediaRule.cssText) | |||
| hasDarkRules = true | |||
| } | |||
| } | |||
| if (hasDarkRules) { | |||
| loadThemeForm('#theme-selector') | |||
| } | |||
| }) | |||
| </script> | |||
| </body> | |||
| </html> | |||
+ 198
- 0
cache/2021/6514143dca5d96bf9e751236b800fba5/index.md
View File
| @@ -0,0 +1,198 @@ | |||
| title: Are We Really Engineers? | |||
| url: https://www.hillelwayne.com/post/crossover-project/are-we-really-engineers/ | |||
| hash_url: 6514143dca5d96bf9e751236b800fba5 | |||
| <p>I sat in front of Mat, idly chatting about tech and cuisine. Before now, I had known him mostly for his cooking pictures on Twitter, the kind that made me envious of suburbanites and their 75,000 BTU woks. But now he was the test subject for my new project, to see if it was going to be fruitful or a waste of time.</p> | |||
| <p>“What’s your job?”</p> | |||
| <p>“Right now I’m working on microservices for a social media management platform.”</p> | |||
| <p>“And before that?”</p> | |||
| <p>“Geological engineering. A lot of open pit mining, some amount of underground tunnel work. Hydropower work. Earth embankment dams because they come along with mines.”</p> | |||
| <p>He told me a story about his old job. His firm was hired to analyze a block cave in British Columbia. Block caves are a kind of mining project where you dig tunnels underneath the deposit to destabilize it. The deposit slowly collapses and leaks material into the tunnels, and then “you just print money”, as Mat called it. The big problem here? The block cave was a quarter mile under a rival company’s toxic waste dump. “In the event of an earthquake, could the waste flood the mine and kill everyone?” He had to prove it was safe. A different kind of work than what he was doing now.</p> | |||
| <p>As for another difference? “My personal blog has better security than some $100 million mining projects.”</p> | |||
| <p>This wasn’t going to be a waste of time, after all.</p> | |||
| <hr> | |||
| <p>Is software engineering “really” engineering? A lot of us call ourselves software engineers. Do we deserve that title? Are we mere pretenders to the idea of engineering? This is an important question, and like all important questions, it regularly sparks arguments online. On one hand you have the people who say we are not engineers because we do not live up to “engineering standards”. These people point to things like <a href="https://www.theatlantic.com/technology/archive/2017/09/saving-the-world-from-code/540393/">The Coming Software Apocalypse</a> as evidence that we don’t have it together. They say that we need things like certification, licensing, and rigorous design if we want to earn the title engineer.</p> | |||
| <p>On the other end of the horseshoe, we have people like Pete McBreen and Paul Graham who say that we are not engineers because engineering cannot apply to our domain. Engineers work on predictable projects with a lot of upfront planning and rigorous requirements. Software is dynamic, constantly changing, unpredictable. If we try to apply engineering practice to software then software would be 10 times as expensive and stuck in 1970.</p> | |||
| <p>For a long time I refused to call myself a software engineer and thought of other people with that title as poseurs. It was reading a short story against engineering that made me question my core assumptions. It was <a href="http://thecodelesscode.com/case/154">A Bridge to Nowhere</a>, one of the <a href="http://thecodelesscode.com/">Codeless Code stories</a>. The author argues that the techniques of bridge building don’t apply to software, since software clients can change their requirements and software gravity can sometimes reverse itself.</p> | |||
| <blockquote> | |||
| <p>The predictability of a true engineer’s world is an enviable thing. But ours is a world always in flux, where the laws of physics change weekly. If we did not quickly adapt to the unforeseen, the only foreseeable event would be our own destruction.</p> | |||
| </blockquote> | |||
| <p>That’s when I realized something about everybody involved in all of these arguments.</p> | |||
| <p>They’ve never built a bridge.</p> | |||
| <p>Nobody I read in these arguments, not one single person, ever worked as a “real” engineer. At best they had some classical training in the classroom, but we all know that looks nothing like reality. Nobody in this debate had anything more than stereotypes to work with. The difference between the engineering in our heads and in reality has been noticed by others before, most visibly by <a href="https://www.youtube.com/watch?v=RhdlBHHimeM">Glenn Vanderburg</a>. He read books on engineering to figure out the difference. But I wanted to go further.</p> | |||
| <p>If I wanted to know how software development compares and contrasts with “real” engineering, then I’d have to talk to “real” engineers. But thinking about that, I realized another problem: while “real” engineers could tell me what they did, they couldn’t tell me how their work <em>differed</em> from mine. Just as we make sweeping statements about real engineering without any evidence, they could make sweeping statements about software without knowing anything about it either. Even if they use a bit of software in the day-to-day work, that’s not the same as developing software as a job. Only a person who’s done <em>both</em> software development and “real” engineering can truthfully speak to the differences between them.</p> | |||
| <p>So that’s what I set out to find: people who used to be professional engineers and then became professional software developers. I call these people <strong>crossovers</strong>, hybrids between the two worlds. I interviewed 17 crossovers on common software misconceptions, how the two worlds relate to each other, whether we can truthfully call what we do engineering, and what the different fields can teach and learn from each other.</p> | |||
| <p>There’s a lot I want to talk about here, much more than can comfortably fit in one blog post. I divided the write up into three parts, dealing with my three core topics. Part one is about the term “engineering”. Is what we do engineering, and can we honestly call ourselves engineers?</p> | |||
| <h2 id="why-is-this-a-question">Why is this a Question?</h2> | |||
| <p>So first some ground rules: “software engineering” is a real field. Most people would agree that the software running on spacecraft counts as “real engineering”. The debate is about rest of the field. Are the people who build websites engineers? What about embedded hardware? I found in my research that people defending the title “engineer” rarely coherently define what an engineer <em>is</em>, usually boiling it down to “engineers solve problems”. The arguments against the title tend to be a little more developed, as are the arguments about transcending it.</p> | |||
| <p>The backlash against “software engineering” comes from two places. First, there are the gatekeepers. While at first we called ourselves programmers and “developers”, after 1985 more and more people started using the title “software engineer”. This “title bloat” is one of the things that led to the backlash: people using it as a title without trying to live up to some form of standards.</p> | |||
| <p>The other side of the backlash comes from the new software movements of the 90s and early 00s. Movements like Extreme Programming and Agile were rejections of applying standard project management techniques to software development. To them, engineering represented the old way of doing things, incompatible with the new way of software development. Software engineering was the past, a failed dead end, and artisanal software was the future.</p> | |||
| <p>Both of these backlashes come from the connotations of engineering. In the case of gatekeepers, it’s the positive connotations they believe we have not earned. In the case of artisanal craftship, it’s the negative baggage they believe we don’t deserve. Both of these are agenda-driven viewpoints, arguments based on how they want software to be. This is good for advocacy but doesn’t help us figure out where software is right now. And their agendas are based on a reference point they don’t understand themselves. None of the people arguing for or against software engineering as engineering have worked as engineers.</p> | |||
| <h2 id="common-misconceptions">Common Misconceptions</h2> | |||
| <p>Let’s start with common arguments that people make about whether to include software in engineering. Most people have an informal concept of what engineering is but not a strict definition. Like pornography, people know what engineering is when they see it. When I asked people to explicitly list the qualities of what makes something engineering, here are the most common responses:</p> | |||
| <ol> | |||
| <li>Making something complex as part of a team.</li> | |||
| <li>Making something physical.</li> | |||
| <li>Making something according to a rigorous spec with strict principles.</li> | |||
| <li>Using mathematical principles in their design.</li> | |||
| <li>Working on situations with high consequences, like loss of life.</li> | |||
| <li>Doing work with a certification and license.</li> | |||
| </ol> | |||
| <p>The first quality is too broad: almost all human professions involve making something complex in a team. It doesn’t capture “engineering”. The second quality is too restrictive: not all forms of engineering result in physical processes. In particular, industrial engineering rarely does. The third quality will be discussed in the next essay.</p> | |||
| <p>That leaves three claims to discuss here. All three are used to say software can’t be engineering. “We don’t use any math, unlike <em>real</em> engineers. Software doesn’t matter, unlike <em>real</em> engineers. We aren’t licensed, unlike <em>real</em> engineers.” As we’ll see, none of these quite hold up.</p> | |||
| <h3 id="engineering-is-mathematical">Engineering is mathematical</h3> | |||
| <p>This one I can address directly. The claim is that engineering involves a lot of hard math, while software involves very little math. The confusion here comes from our misunderstanding of mathematics. Much of the math that mechanical engineers use is <strong>continuous math</strong>. This is where we work over a continuous domain, like real numbers. Things like calculus, trigonometry, and differential equations are in this category. This is what most people in the US learn in high school, codifying it as what they think of as “math”.</p> | |||
| <p>In software, we don’t use these things, leading to the conception that we don’t use math. But we actually use <strong>discrete math</strong>, where we deal exclusively with non-continuous numbers. This includes things like graph theory, logic, and combinatorics. You might not realize that you are using these, but you do. They’re just so internalized in software that we don’t see them as math! In fact most of computer science is viewable as a branch of mathematics. Every time you simplify a conditional or work through the performance complexity of an algorithm, you are using math. Just because there are no integrals doesn’t mean we are mathless.</p> | |||
| <p>This falls in line with the rest of engineering. Different branches use different kinds of math in different ways. Industrial engineers are concerned with very different things than mechanical engineers are. Just because we use a different branch of math doesn’t mean we’re not doing engineering.</p> | |||
| <h3 id="engineering-is-high-consequence">Engineering is high-consequence</h3> | |||
| <p>I’m not sure what first disabused me of this notion. It might have been the conversation with Mike, an ex-mechanical engineer who now designs glaciology sensors. One of his last projects before the crossover was as part of a team working on a classified medical device. Most of it went smoothly, but then came the big blocker:</p> | |||
| <blockquote> | |||
| <p>The one the thing that gave us the most grief was the handle, which was a bent bit of wire with some plastic molded around it. Yeah, getting the wires to bend correctly and the plastic onto this shiny piece of anodized aluminium wire. […] That was the thing that took three months in the project.</p> | |||
| </blockquote> | |||
| <p>Three months of the project dedicated to getting the handle working right.</p> | |||
| <p>This is not to say the work wasn’t important. It does mean that the engineers spent a lot of time on something that’s low-consequence or non-safety critical. In retrospect, this shouldn’t have surprised me. The world is vast and our industry is huge. Someone needs to design the bridges, yes, but someone also needs to design all of the little things we use in our day-to-day life. Much of the engineering there is low-stakes, low-consequence, just like much software is.</p> | |||
| <blockquote> | |||
| <p>There’s a huge difference between if you’re making a cabinet for Bluetooth speakers, like ones that are on my desk, or you’re making an assembly for the landing gear of 737. You’re using some of the same tools, but your approach is wildly different. <em>- Nathan (mechanical)</em></p> | |||
| </blockquote> | |||
| <p>«But there are still <em>some</em> things that lead to loss of life!» Yes, and the same is true of software. An integer overflow <a href="https://www.theverge.com/2014/10/20/7014705/coding-error-911-fcc-washington">in the Intrado software</a> lead to a several hour 911 outage for millions of people. A biased algorithm <a href="https://www.technologyreview.com/2019/01/21/137783/algorithms-criminal-justice-ai/">unfairly sent people to jail</a>. Just like with “real” engineering, there’s also a wide swath of software that won’t kill people but will lead to millions of dollars in damages. This can be everything from Amazon going down to a game being unplayable.</p> | |||
| <h3 id="engineers-are-licensed">Engineers are licensed</h3> | |||
| <p>This is the claim I’ve heard most: licensure. There’s a difference between doing engineering and being an engineer, just as people practice medicine at home without being doctors. Maybe software engineering is possible, but without the licenses we are not software <em>engineers</em>. In Canada you can’t even call yourself a “Software Engineer” unless you’re accredited!</p> | |||
| <p>At least that’s what people in the United States say. Several Europeans I spoke to said the exact same thing about US system. Everybody seems to think the worst about their own system and the best about others. And about half of the engineers I spoke to weren’t licensed but were still considered professional engineers by their peers. They didn’t have the “certification”, but they had the <em>skills</em>.</p> | |||
| <p>In the US, you don’t need a license to practice any kind of engineering. You need a license to be a “principal engineer”, a.k.a. the person who formally signs off on plans as valid. But the engineers working under the principal engineer don’t need to be accredited and often are not. In fact many of them don’t even have formal training as engineers.</p> | |||
| <p>Here’s the problem with deciding engineering based on licenses: licenses are a political and social construct, not a fact of nature. Societies adopt licensure for reasons that are as much political as technical. To see this, let’s discuss some of the history of licensure in the United States.</p> | |||
| <p>As recently as 1906, not a single state in the US required licenses for any project. <a href="https://en.wikipedia.org/wiki/Regulation_and_licensure_in_engineering#United_States">Wyoming</a> was the first state to instate a licensure policy, entirely because irrigation projects kept blowing the budget. They theorized accredited engineers could give better estimates on project costs. It took 40 more years for all fifty states to get on board.</p> | |||
| <p>This all means that licensure isn’t part of the federal government: it’s a requirement by the states. Among other things, licenses aren’t necessarily transferable between states. If you get a license in Texas, you have to go through a process to be able to practice in California, in what’s called “comity”.</p> | |||
| <p>While licensure originated as part of cost overruns, it expanded so rapidly for an entirely different reason. Regulations are written in blood. Fields become regulated when the lack of regulation kills people. Until that happens on a wide scale with arbitrary programs, it’s unlikely that we’ll ever see the same licensure requirements for software. In the places where software <em>has</em> killed people, like Therac-25 and aircraft accidents, we see stricter regulations. Whether this leads to broader licensure requirements for software engineers remains to be seen.</p> | |||
| <p>You could argue that it’s immoral for us to not be licensed. This is an argument I’m sympathetic to. But it’s a normative argument, not a positive one. By saying “we should be licensed”, you are saying something about how the world <em>ought to</em> be. You are trying to answer the question “should we be held to higher standards?” But that’s not the question here. I don’t care right now where we should be going; I just want to know where we are right now. Whether or not we are engineers is irrelevant to whether or not we are <em>good</em> engineers.</p> | |||
| <p>In conclusion: licenses exist because we are part of society and have legal requirements, not because they are essential to what it means to do engineering. So while you might want to make software more licensed, as it stands the licensing question doesn’t change the essence of our work.</p> | |||
| <h2 id="the-truth">The Truth</h2> | |||
| <p>This leaves us back where we started: there aren’t any qualities we can point to and say “this is engineering, this is not”. It’s a standard Wittgenstein game problem: human constructs do not neatly fall into precise definitions. Rather something like “engineering” is a family of related concepts, where we judge whether something belongs based on how much it resembles other things in the family. In other words, “engineering” is “what engineers do”. In other other words, something becomes engineering if enough engineers say it’s engineering.</p> | |||
| <p>Consider chemical engineering. Chemical engineering is unlike mechanical, civil, or electrical engineering. Chemical engineers create processes to manufacture products at scale, often using experimentation and iteration. But nobody would disagree that it is engineering. Chemical engineering started in the late 1800s, before states licensed engineers. If chemical engineering started now, people would refuse to call it engineering. And they’d be wrong to refuse.</p> | |||
| <p>Once I realized this, my interviewing process changed a little. Instead of asking how they felt about certain engineering topics, I just asked them point blank. “Do you consider software engineering actually engineering?”</p> | |||
| <p>Of the 17 crossovers I talked to, 15 said yes.</p> | |||
| <p>That’s not the answer I expected going in. I assumed we weren’t engineers, that we’re actually very far from being engineers. But then again, I was never a “real” engineer. I don’t know what it’s like to be a “real” engineer, and so can’t compare software engineering to other forms. I don’t have the experience. These people did, and they considered software engineering real engineering.</p> | |||
| <blockquote> | |||
| <p>Even the product owners and project managers are in a sense engineers […] everyone is kind of an engineer, an engineer of sorts. <em>-Kate (chemical)</em></p> | |||
| </blockquote> | |||
| <p>It makes no sense to use “real” engineering in contrast to “software” engineering. Going forward I will instead use the term “trad” engineering.</p> | |||
| <h3 id="craft-vs-engineering">Craft vs Engineering</h3> | |||
| <blockquote> | |||
| <p>I’m gonna respond in a slightly different way. Not every software developer is a software engineer, just like not every single person who works in construction is an engineer. An engineer is a very specific set of skills. […] Software [engineering] is skill plus understanding, all the processes and life cycle and the consequences and all the things that you should be aware of [and] avoid. <em>-Dawn (Mechanical & Chemical)</em></p> | |||
| </blockquote> | |||
| <p>That said, many of the crossovers also added an additional qualification: software engineering is real engineering, but a lot of people who write software aren’t doing software engineering. This is not a problem with them, rather a problem with our field: we don’t have a rich enough vocabulary to talk about what these developers do. Not everybody who works with electricity is going to be an electrical engineer; many will be electricians. And this is okay. Electrical engineering is a very narrow skill set in the broader field of electric professions and plenty of people have other important skills in that space. But we use things like “programmer”, “software engineer”, and “software developer” interchangeably. What is the difference between a software engineer and a software developer?</p> | |||
| <p>Some people propose the word “software craftsman”. This term comes from the book <a href="https://learning.oreilly.com/library/view/software-craftsmanship-the/0201733862/">Software Craftsmanship: The New Imperative</a>, by Pete McBreen. In it he argues that software is not a kind of engineering, being much more free-form creative and flexible. We’re not line workers but artisans, artists, people who take pride in the craft we do and the flexibility of our states. As he puts it:</p> | |||
| <blockquote> | |||
| <p>Software development is all about the unknown. The production process for software is trivially easy—just copy a disk or CD. The software engineering metaphor fails because we understand production, a mechanical task, much better than we understand design, an intellectual task. <em>- <a href="https://learning.oreilly.com/library/view/software-craftsmanship-the/0201733862/ch06.html">Software Craftsmanship</a></em></p> | |||
| </blockquote> | |||
| <p>Many people have asked me why I care so much about this project. Why does it matter whether or not software is “really” engineering? Why can’t we just say that “software is software”? It’s because of these misconceptions. People have a stereotyped notion of what engineering looks like. Because software doesn’t look like the stereotype, they assume that we are wholly unlike engineering. The engineering disciplines have nothing to teach us. We are breaking pristine ground and have no broader history to guide us.</p> | |||
| <p>In contrast, if we are doing engineering, then we have a landmark. We can meaningfully compare and contrast the work we do as software engineers from the work that others do as traditional engineers. We can adapt their insights and watch for their pitfalls. We can draw upon the extant knowledge of our society to make better software.</p> | |||
| <p>I believe everything McBreen said about software is fairly reasonable, about how hard it is to predict things and how it’s intensely personally created. What he gets wrong is the assumption that engineering is <em>not</em> this way. Engineering is much richer, more creative, and more artistic than he thought. But of course he would have an imperfect view: he is, after all, not a traditional engineer.</p> | |||
| <hr> | |||
| <p>Here’s my final take. This is the belief I’ve settled on in synthesizing all the interviews I did and does not necessarily reflect how the crossovers think. I went into this thinking that software wasn’t really engineering. Maybe there were a few people who could count themselves as that but most of us were far below that threshold. I still believe that most of us are not engineers, because we’re working in domains that people don’t see as engineering. Most people don’t consider a website “engineered”. <em>However</em>, and this is a big however, there’s a much smaller gap between “software development” and “software engineering” than there is between “electrician” and “electrical engineer”, or between “trade” and “engineering” in all other fields. Most people can go between “software craft” and “software engineering” without significant retraining. We are separated from engineering by circumstance, not by essence, and we can choose to bridge that gap at will.</p> | |||
| <p>In the next essay, we’ll talk about the similarities and differences between traditional engineering and software engineering, and how they’re actually not all that different in the end.</p> | |||
| <p><em>Part two,</em> <a href="https://www.hillelwayne.com/post/crossover-project/we-are-not-special/">We Are Not Special</a>, <em>is available <a href="https://www.hillelwayne.com/post/crossover-project/we-are-not-special/">here</a>. If you enjoyed these essays, I have a <a href="https://buttondown.email/hillelwayne/">weekly newsletter</a> and am on <a href="https://twitter.com/hillelogram">Twitter</a>.</em></p> | |||
| <p><em>Thanks to | |||
| Glenn Vanderburg | |||
| , | |||
| <a href="https://chelseatroy.com/">Chelsea Troy</a> | |||
| , | |||
| <a href="https://twitter.com/craftworksxyz">Will Craft</a> | |||
| , and | |||
| <a href="https://www.danluu.com">Dan Luu</a> | |||
| for feedback, and to all of the engineers whom I interviewed.</em></p> | |||
| <h2 id="appendix-methodology">Appendix: Methodology</h2> | |||
| <p>I only searched out people who work professionally for at least one year in each field. In practice, work experience in trad engineering ranged from a year and a half on the low end to over 15 years on the high end. Interviews ranged from half an hour to two hours, leading to a total of 24 hours of recorded interview. Two of the interviewees were not recorded, but I took notes. The breakdown of the specialties is:</p> | |||
| <p><img src="/post/crossover-project/img/chart.png" alt="Pie chart of the engineer types"></p> | |||
| <ul> | |||
| <li>Civil engineers work on buildings. One person I talked to was a classic civil engineer, one specialized in designing mines, and two worked on oil rigs.</li> | |||
| <li>Mechanical engineers design physical machines.</li> | |||
| <li>Electrical engineers design circuits and electronics. Two worked on chipsets, while one was embedded in a submarine.</li> | |||
| <li>Chemical engineers create processes to make chemicals at scale. They are generally not creating entirely new chemicals or chemical products, but work on how to produce extant ones. “Chemicals” is a broad category here, covering everything from clean water to toothpaste.</li> | |||
| <li>Industrial Engineers create holistic systems and procedures in an industry. One designed data center layouts and the other worked on integrating disparate air traffic systems.</li> | |||
| </ul> | |||
| <p>These are very broad summaries of the specialties. Most engineers work in a subdomain of a specialty, such as automotive engineering or circuit design. There were also some fields of engineering I didn’t cover in my interviews. This includes aerospace and nuclear engineering. I suspect (<em>suspect</em>) that aerospace would be roughly similar to mechanical engineering and nuclear engineering would be roughly similar to civil engineering. But it remains a threat to validity nonetheless. </p> | |||
| <p>The other two threats to validity are geographic location and crossover type. The majority of the interviewees were either in the US or the UK, with the rest being from the EU or Canada. I did not get a chance to interview anybody from Latin America, South America, Africa, or Asia. Everybody interviewed crossed from traditional engineering to software engineering. I was not able to find anyone who crossed the other way.</p> | |||
+ 563
- 0
cache/2021/6be8b8649cd695b922a43d15f74d4773/index.html
View File
| @@ -0,0 +1,563 @@ | |||
| <!doctype html><!-- This is a valid HTML5 document. --> | |||
| <!-- Screen readers, SEO, extensions and so on. --> | |||
| <html lang="fr"> | |||
| <!-- Has to be within the first 1024 bytes, hence before the <title> | |||
| See: https://www.w3.org/TR/2012/CR-html5-20121217/document-metadata.html#charset --> | |||
| <meta charset="utf-8"> | |||
| <!-- Why no `X-UA-Compatible` meta: https://stackoverflow.com/a/6771584 --> | |||
| <!-- The viewport meta is quite crowded and we are responsible for that. | |||
| See: https://codepen.io/tigt/post/meta-viewport-for-2015 --> | |||
| <meta name="viewport" content="width=device-width,initial-scale=1"> | |||
| <!-- Required to make a valid HTML5 document. --> | |||
| <title>How to Build Good Software (archive) — David Larlet</title> | |||
| <meta name="description" content="Publication mise en cache pour en conserver une trace."> | |||
| <!-- That good ol' feed, subscribe :). --> | |||
| <link rel="alternate" type="application/atom+xml" title="Feed" href="/david/log/"> | |||
| <!-- Generated from https://realfavicongenerator.net/ such a mess. --> | |||
| <link rel="apple-touch-icon" sizes="180x180" href="/static/david/icons2/apple-touch-icon.png"> | |||
| <link rel="icon" type="image/png" sizes="32x32" href="/static/david/icons2/favicon-32x32.png"> | |||
| <link rel="icon" type="image/png" sizes="16x16" href="/static/david/icons2/favicon-16x16.png"> | |||
| <link rel="manifest" href="/static/david/icons2/site.webmanifest"> | |||
| <link rel="mask-icon" href="/static/david/icons2/safari-pinned-tab.svg" color="#07486c"> | |||
| <link rel="shortcut icon" href="/static/david/icons2/favicon.ico"> | |||
| <meta name="msapplication-TileColor" content="#f0f0ea"> | |||
| <meta name="msapplication-config" content="/static/david/icons2/browserconfig.xml"> | |||
| <meta name="theme-color" content="#f0f0ea"> | |||
| <!-- Documented, feel free to shoot an email. --> | |||
| <link rel="stylesheet" href="/static/david/css/style_2021-01-20.css"> | |||
| <!-- See https://www.zachleat.com/web/comprehensive-webfonts/ for the trade-off. --> | |||
| <link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_regular.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
| <link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_bold.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
| <link rel="preload" href="/static/david/css/fonts/triplicate_t4_poly_italic.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" crossorigin> | |||
| <link rel="preload" href="/static/david/css/fonts/triplicate_t3_regular.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
| <link rel="preload" href="/static/david/css/fonts/triplicate_t3_bold.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
| <link rel="preload" href="/static/david/css/fonts/triplicate_t3_italic.woff2" as="font" type="font/woff2" media="(prefers-color-scheme: dark)" crossorigin> | |||
| <script> | |||
| function toggleTheme(themeName) { | |||
| document.documentElement.classList.toggle( | |||
| 'forced-dark', | |||
| themeName === 'dark' | |||
| ) | |||
| document.documentElement.classList.toggle( | |||
| 'forced-light', | |||
| themeName === 'light' | |||
| ) | |||
| } | |||
| const selectedTheme = localStorage.getItem('theme') | |||
| if (selectedTheme !== 'undefined') { | |||
| toggleTheme(selectedTheme) | |||
| } | |||
| </script> | |||
| <meta name="robots" content="noindex, nofollow"> | |||
| <meta content="origin-when-cross-origin" name="referrer"> | |||
| <!-- Canonical URL for SEO purposes --> | |||
| <link rel="canonical" href="https://www.csc.gov.sg/articles/how-to-build-good-software"> | |||
| <body class="remarkdown h1-underline h2-underline h3-underline em-underscore hr-center ul-star pre-tick"> | |||
| <article> | |||
| <header> | |||
| <h1>How to Build Good Software</h1> | |||
| </header> | |||
| <nav> | |||
| <p class="center"> | |||
| <a href="/david/" title="Aller à l’accueil"><svg class="icon icon-home"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-home"></use> | |||
| </svg> Accueil</a> • | |||
| <a href="https://www.csc.gov.sg/articles/how-to-build-good-software" title="Lien vers le contenu original">Source originale</a> | |||
| </p> | |||
| </nav> | |||
| <hr> | |||
| <h2>Why Bad Software Happens to Good People</h2> | |||
| <p>Bad software is one of the few things in the world you cannot solve with money. Billion dollar airlines have flight search apps that are often inferior to those built by groups of students. Established taxi | |||
| companies the world over have terrible booking apps despite the threat they face from ride-sharing services. And painful corporate IT systems are usually projects with massive budgets, built over the course of many years. Whatever the cause of bad software is, it does not seem to be a lack of funding.</p> | |||
| <p>Surprisingly, the root cause of bad software has less to do with specific engineering choices, and more to do with how development projects are managed. The worst software projects often proceed in a very particular way:</p> | |||
| <p>The project owners start out wanting to build a specific solution and never | |||
| explicitly identify the problem they are trying to solve. They then gather a long list of requirements from a large group of stakeholders. This list is then handed off to a correspondingly large external development team, who get to work building this highly customised piece of software from scratch. Once all the requirements are met, everyone celebrates as the system is launched and the project is declared complete.</p> | |||
| <p><hr> | |||
| <h5><em> | |||
| The root cause of bad software has less to do with specific engineering choices, and more to do with how development projects are managed. | |||
| </em></h5> | |||
| <hr> | |||
| <p> However, though the system technically meets specifications, severe issues are | |||
| found when it is put in the hands of actual users. It is slow, confusing, and | |||
| filled with subtle bugs that make using it an exercise in frustration. Unfortunately, | |||
| by this time the external development team has been dismissed and there | |||
| are no resources left over to make the necessary fixes. By the time a new project | |||
| can be initiated years later, all knowledge of what caused these problems has left | |||
| the organisation and the cycle starts over again.</p> | |||
| <p></p></p> | |||
| <p class="small-text">A Conversation with Li Hongyi (Part 1)<br> | |||
| <em>Scroll down for more videos</em></p> | |||
| <p>The right coding language, system architecture, or interface design will | |||
| vary wildly from project to project. But there are characteristics particular | |||
| to software that consistently cause traditional management practices to fail, while allowing small startups to succeed with a shoestring budget:</p> | |||
| <p>• Reusing good software is easy; | |||
| it is what allows you to build good | |||
| things quickly;<br> | |||
| • Software is limited not by the amount | |||
| of resources put into building it, but | |||
| by how complex it can get before it | |||
| breaks down; and<br> | |||
| • The main value in software is not the | |||
| code produced, but the knowledge | |||
| accumulated by the people who | |||
| produced it.</p> | |||
| <p> | |||
| </p> | |||
| <p>Understanding these characteristics may not guarantee good outcomes, but it | |||
| does help clarify why so many projects produce bad outcomes. Furthermore, | |||
| these lead to some core operating principles that can dramatically improve | |||
| the chances of success:</p> | |||
| <p> | |||
| </p> | |||
| <p>1. Start as simple as possible;<br> | |||
| 2. Seek out problems and iterate; and<br> | |||
| 3. Hire the best engineers you can.</p> | |||
| <p>While there are many subtler factors to consider, these principles form a foundation that lets you get started building good software.</p> | |||
| <h2>Reusing Software Lets You Build Good Things Quickly</h2> | |||
| <p>Software is easy to copy. At a mechanical level, lines of code can literally be copied and pasted onto another computer. More generally, the internet is full of tutorials on how to build different kinds of systems using ready-made code modules that are available online. Modern software is almost never developed from scratch. Even the most innovative applications are built using existing software that has been combined and modified to achieve a new result.</p> | |||
| <p>The biggest source of reusable code modules is the open source community. Open source software is software in which code is freely published for anyone to see and use. Many of the largest contributors to the open source community are giant tech companies. If you want to use a state-of-the-art planet scalable database as Facebook does, just download the code for Cassandra that they open sourced in 2008. If you want to try out Google’s cutting-edge machine learning for yourself, download the TensorFlow system published in 2015. Using open source code does not just make your application development faster, it gives you access to technology that is far more sophisticated than anything | |||
| you could have developed yourself. For the most popular open source code, it is even more secure as there are many more people paying attention and fixing vulnerabilities. This is the reason digital technology has made such rapid progress: even the newest engineers can build upon the most advanced tools our profession has to offer.</p> | |||
| <p>The advent of cloud services has taken reusability even further, offering the full use of even proprietary systems for just a subscription fee. Need a simple website? Just configure one in a few clicks using a website building service like Squarespace or Wix. A database? Subscribe to a virtual one from Amazon Web Services or Microsoft Azure. Cloud services allow developers to benefit from specialisation; the service provider handles the setup, maintenance, and continued development of a reliable, high-quality piece of software that is used by all its subscribers. This allows software developers to stop wasting time on solved problems and instead focus on delivering actual value.</p> | |||
| <p>You cannot make technological progress if all your time is spent on rebuilding existing technology. Software engineering is about building automated systems, and one of the first things that gets automated away is routine software engineering work. The point is to understand what the right systems to reuse are, how to customise them to fit your unique requirements, and fixing novel problems | |||
| discovered along the way.</p> | |||
| <p><hr> | |||
| <h5><em> | |||
| Software engineering is about building automated systems, and one of the first things that gets automated away is routine software engineering work. | |||
| </em></h5> | |||
| <hr> | |||
| <h2>Software Is Limited by Complexity</h2> | |||
| <p>How useful a piece of software can be is usually limited by its complexity rather than the amount of resources invested in building it.</p> | |||
| <p>IT systems are often full of features but are still hated by users because of how confusing they become. In contrast, highly ranked mobile apps tend to be lauded for their simplicity and | |||
| intuitiveness. Learning to use software is hard. Beyond a point, new features actually make things worse for users because the accumulated complexity starts to become overwhelming. For example, after serving as the hub of Apple’s media ecosystem for almost 20 years, iTunes was split into three | |||
| different apps (for music, podcasts, and TV shows) this year, as its features had grown too complex for one app to handle. From a usability perspective, the limit is not how many features can be implemented, but rather what can fit into a simple intuitive interface.</p> | |||
| <p>Even ignoring usability, engineering progress slows to a halt once a project becomes too complex. Each new line of code added to an application has a chance of interacting with every other | |||
| line. The bigger an application’s codebase, the more bugs are introduced whenever a new feature is built. Eventually, the rate of work created from new bugs cancels out the rate of work done from feature development. This is known as “technical debt” and is the main challenge in professional software development. It is the reason why many large IT systems have issues that go unfixed for years. | |||
| Adding more engineers to the project just adds to the chaos: they start running | |||
| faster in place as the codebase keels over from its own weight.</p> | |||
| <hr> | |||
| <h5><em> | |||
| Building good software involves alternating cycles of expanding and reducing complexity. | |||
| </em></h5> | |||
| <hr> | |||
| <p>In such cases, the only way forward is to take a step back to rationalise and | |||
| simplify the codebase. The system architecture can be redesigned to limit | |||
| unexpected interactions. Non-critical features can be removed even if they | |||
| have already been built. Automated tools can be deployed to check for bugs and | |||
| badly written code. Bill Gates once said “Measuring programming progress by | |||
| lines of code is like measuring aircraft building progress by weight”. Human minds can only handle a finite amount of complexity, so how sophisticated a software system can get depends on | |||
| how efficiently this complexity budget is used.</p> | |||
| <p>Building good software involves alternating cycles of expanding and reducing complexity. As new features are developed, disorder naturally accumulates in the system. When this messiness | |||
| starts to cause problems, progress is suspended to spend time cleaning up. This two-step process is necessary because there is no such thing as platonically good engineering: it depends on your | |||
| needs and the practical problems you encounter. Even a simple user interface such as Google’s search bar contains a massive amount of complexity under the surface that cannot be perfected in a single iteration. The challenge is managing this cycle, letting it get messy enough to make meaningful progress, | |||
| but not letting it get so complicated that it becomes overwhelming.</p> | |||
| <hr> | |||
| <h5><em> | |||
| There is no such thing as platonically good engineering: it depends on your needs and the | |||
| practical problems you encounter. | |||
| </em></h5> | |||
| <hr> | |||
| <h2>Software Is about Developing Knowledge More than Writing Code</h2> | |||
| <p>In software development, most ideas are bad; this is not anyone’s fault. It is just | |||
| that the number of possible ideas is so large that any particular idea is probably | |||
| not going to work, even if it was chosen very carefully and intelligently. To make | |||
| progress, you need to start with a bunch of bad ideas, discard the worst, and | |||
| evolve the most promising ones. Apple, a paragon of visionary design, goes | |||
| through dozens of prototypes before landing on a final product. The final | |||
| product may be deceptively simple; it is the intricate knowledge of why this | |||
| particular solution was chosen over its alternatives that allows it to be good.</p> | |||
| <p>This knowledge continues to be important even after the product is built. If a new | |||
| team takes over the code for an unfamiliar piece of software, the software will soon | |||
| start to degrade. Operating systems will update, business requirements | |||
| will change, and security problems will be discovered that need to be fixed. Handling these subtle errors is often harder than building the software in the first place, since it requires intimate | |||
| knowledge of the system’s architecture and design principles.</p> | |||
| <p></p></p> | |||
| <p class="small-text">A Conversation with Li Hongyi (Part 2)<br> | |||
| <em>Scroll down for more videos</em></p> | |||
| <p>In the short term, an unfamiliar development team can address these problems with | |||
| stopgap fixes. Over time though, new bugs accumulate due to the makeshift nature | |||
| of the additional code. User interfaces become confusing due to mismatched | |||
| design paradigms, and system complexity increases as a whole. Software should be | |||
| treated not as a static product, but as a living manifestation of the development | |||
| team’s collective understanding.</p> | |||
| <p><hr> | |||
| <h5><em> | |||
| Software should be treated not as a static product, but as a | |||
| living manifestation of the development team’s collective understanding. | |||
| </em></h5> | |||
| <hr> | |||
| <p>This is why relying on external vendors for your core software development is | |||
| difficult. You may get a running system and its code, but the invaluable knowledge | |||
| of how it is built and what design choices were made leaves your organisation. | |||
| This is also why handing a system over to new vendors for “maintenance” often | |||
| causes problems. Even if the system is very well documented, some knowledge | |||
| is lost every time a new team takes over. Over the years, the system becomes a | |||
| patchwork of code from many different authors. It becomes harder and harder | |||
| to keep running; eventually, there is no one left who truly understands how | |||
| it works.</p> | |||
| <p>For your software to keep working well in the long term, it is important to | |||
| have your staff learning alongside the external help to retain critical engineering | |||
| knowledge in your organisation.</p> | |||
| <h2>3 Principles for Good Software Development</h2> | |||
| <h4>1. Start as Simple as Possible</h4></p> | |||
| <p>Projects that set out to be a “one-stop | |||
| shop” for a particular domain are often | |||
| doomed. The reasoning seems sensible | |||
| enough: What better way to ensure your | |||
| app solves people’s problems than by | |||
| having it address as many as possible? | |||
| After all, this works for physical stores | |||
| such as supermarkets. The difference is | |||
| that while it is relatively easy to add a | |||
| new item for sale once a physical store | |||
| is set up, an app with twice as many | |||
| features is more than twice as hard to | |||
| build and much harder to use.</p> | |||
| <p>Building good software requires focus: | |||
| starting with the simplest solution that | |||
| could solve the problem. A well-made | |||
| but simplistic app never has problems | |||
| adding necessary features. But a big IT | |||
| system that does a lot of things poorly | |||
| is usually impossible to simplify and | |||
| fix. Even successful “do it all” apps like | |||
| WeChat, Grab, and Facebook started | |||
| out with very specific functionality and | |||
| only expanded after they had secured | |||
| their place. Software projects rarely | |||
| fail because they are too small; they fail | |||
| because they get too big.</p> | |||
| <p><hr> | |||
| <h5><em> | |||
| Software projects rarely fail because they | |||
| are too small; they fail because they get too big. | |||
| </em></h5> | |||
| <hr> | |||
| <p>Unfortunately, keeping a project focused | |||
| is very hard in practice: just gathering | |||
| the requirements from all stakeholders | |||
| already creates a huge list of features.</p> | |||
| <p></p></p> | |||
| <p class="small-text">A Conversation with Li Hongyi (Part 3)<br> | |||
| <em>Scroll down for last video</em></p> | |||
| <p>One way to manage this bloat is by | |||
| using a priority list. Requirements are | |||
| all still gathered, but each are tagged | |||
| according to whether they are absolutely | |||
| critical features, high-value additions, | |||
| or nice-to-haves. This creates a much | |||
| lower-tension planning process because | |||
| features no longer need to be explicitly | |||
| excluded. Stakeholders can then more | |||
| sanely discuss which features are the | |||
| most important, without worrying about | |||
| something being left out of the project. | |||
| This approach also makes explicit the | |||
| trade-offs of having more features. | |||
| Stakeholders who want to increase | |||
| the priority for a feature have to also | |||
| consider what features they are willing | |||
| to deprioritise. Teams can start on the | |||
| most critical objectives, working their way | |||
| down the list as time and resources allow.</p> | |||
| <p>We followed a similar process for all | |||
| our most successful apps. Form.gov.sg | |||
| started out as a manual Outlook Macro | |||
| that took us six hours to set up for | |||
| our first user but today has processed | |||
| about a million public submissions. | |||
| Data.gov.sg started out as a direct copy | |||
| of an open source project and has since | |||
| grown to over 300,000 monthly visits. | |||
| Parking.sg had a massive list of almost | |||
| 200 possible features that we never | |||
| got around to building but still has over | |||
| 1.1 million users today. These systems | |||
| are well received not in spite of their | |||
| simplicity but because of it.</p> | |||
| <h4>2. Seek Out Problems and Iterate</h4> | |||
| <p>In truth, modern software is so complicated and changes so rapidly that no amount of | |||
| planning will eliminate all shortcomings. Like writing a good paper, awkward | |||
| early drafts are necessary to get a feel of what the final paper should be. To | |||
| build good software, you need to first build bad software, then actively seek out problems to improve on your solution.</p> | |||
| <p>This starts with something as simple as talking to the actual people you are trying to help. The goal is to understand the root problem you want to solve and avoid jumping to a solution based just on preconceived biases. When we first started on Parking.sg, our hypothesis was that enforcement officers found it frustrating to have to keep doing the mental calculations regarding paper coupons. However, after spending just one afternoon with an experienced officer, we discovered that | |||
| doing these calculations was actually quite simple for someone doing it professionally. That single conversation saved us months of potentially wasted effort and let us refocus our project on helping drivers instead. </p> | |||
| <p>Beware of bureaucratic goals masquerading as problem statements. “Drivers feel frustrated when dealing with parking coupons” is a problem. “We need to build an app for drivers as part of our Ministry Family Digitisation Plans” is not. “Users are annoyed at how hard it is to find information on government websites” is a problem. “As part of the Digital Government Blueprint, we need to rebuild our websites to conform to the new design service standards” is not. If our end goal is to make citizens’ lives better, we need to explicitly acknowledge the things that are making their lives worse.</p> | |||
| <p>Having a clear problem statement lets you experimentally test the viability of different solutions that are too hard to determine theoretically. Talking to a chatbot may not be any easier than navigating a website, and users may not want to install yet another app on their phones no matter how secure it makes the country. With software, apparently obvious solutions often have fatal flaws that do not show up until they are put to use. The aim is not yet to build the final product, but to first identify these problems as quickly and as cheaply as possible. Non-functional mock-ups to test interface designs. Semi-functional mock-ups to try different features. Prototype code, written hastily, could help garner feedback more quickly. Anything created at this stage should be treated as disposable. The desired output | |||
| of this process is not the code written, but a clearer understanding of what the right thing to build is.</p> | |||
| <p><hr> | |||
| <h5><em> | |||
| Beware of bureaucratic goals masquerading as problem statements. If our end goal is to make citizens’ lives better, we need to explicitly acknowledge the things that are making their lives worse. | |||
| </em></h5> | |||
| <hr> | |||
| <p>With a good understanding of the right solution, you can start work on building the actual product. You stop exploring new ideas and narrow down to identifying problems with your particular implementation. Begin with a small number of testers who will quickly spot the obvious bugs that need to be fixed. As problems are addressed, you can increasingly open up to a larger pool who will find more esoteric issues.</p> | |||
| <p>Most people only give feedback once. If you start by launching to a large audience, everyone will give you the same obvious feedback and you’ll have nowhere to go from there. Even the best product ideas built by the best engineers will start out with significant issues. The aim is to repeatedly refine the output, | |||
| sanding down rough edges until a good product emerges.</p> | |||
| <p>Even after all this iteration, after launch is when problems with a product matter | |||
| the most. A problem that happens only 0.1% of the time may not get noticed | |||
| during testing. But once you have a million users, every day the problem | |||
| goes unresolved is a thousand more angry people you have to deal with. You need to fix problems caused by new mobile devices, network outages, or security attacks before they cause | |||
| substantial harm to your users. With Parking.sg we built a series of secondary | |||
| systems that continuously check the main system for any discrepancies in | |||
| payments, duplicate parking sessions, and application crashes. Building up | |||
| an “immune system” over time lets you avoid being overwhelmed as new issues | |||
| inevitably come up.</p> | |||
| <p>Overall, the approach is to use these different feedback loops to efficiently | |||
| identify problems. Small feedback loops allow for quick and easy correction | |||
| but miss out on broader issues. Large feedback loops catch broader issues but | |||
| are slow and expensive. You want to use both, resolving as much as possible with | |||
| tight loops while still having wide loops to catch unexpected errors. Building | |||
| software is not about avoiding failure; it is about strategically failing as fast | |||
| as possible to get the information you need to build something good.</p> | |||
| <h4>3. Hire the Best Engineers You Can</h4></p> | |||
| <p>The key to having good engineering is having good engineers. Google, | |||
| Facebook, Amazon, Netflix, and Microsoft all run a dizzying number of the largest | |||
| technology systems in the world, yet, they famously have some of the most | |||
| selective interview processes while still competing fiercely to recruit the strongest | |||
| candidates. There is a reason that the salaries for even fresh graduates have | |||
| gone up so much as these companies have grown, and it is not because they | |||
| enjoy giving away money.</p> | |||
| <p>Both Steve Jobs and Mark Zuckerberg have said that the best engineers are at | |||
| least 10 times more productive than an average engineer. This is not because | |||
| good engineers write code 10 times faster. It is because they make better | |||
| decisions that save 10 times the work.</p> | |||
| <p>A good engineer has a better grasp of | |||
| existing software they can reuse, thus | |||
| minimising the parts of the system they | |||
| have to build from scratch. They have | |||
| a better grasp of engineering tools, | |||
| automating away most of the routine | |||
| aspects of their own job. Automation | |||
| also means freeing up humans to work | |||
| on solving unexpected errors, which the | |||
| best engineers are disproportionately | |||
| better at. Good engineers themselves | |||
| design systems that are more robust | |||
| and easier to understand by others. | |||
| This has a multiplier effect, letting their | |||
| colleagues build upon their work much | |||
| more quickly and reliably. Overall, good | |||
| engineers are so much more effective | |||
| not because they produce a lot more | |||
| code, but because the decisions they | |||
| make save you from work you did not | |||
| know could be avoided.</p> | |||
| <p></p> | |||
| <p class="small-text">A Conversation with Li Hongyi (Part 4)</p> | |||
| <p>This also means that small teams of the | |||
| best engineers can often build things | |||
| faster than even very large teams of | |||
| average engineers. They make good | |||
| use of available open source code and | |||
| sophisticated cloud services, and offload | |||
| mundane tasks onto automated testing | |||
| and other tools, so they can focus on the creative problem-solving aspects of | |||
| the job. They rapidly test different ideas | |||
| with users by prioritising key features | |||
| and cutting out unimportant work. This | |||
| is the central thesis of the classic book | |||
| “The <em>Mythical Man-Month</em>”<a href="#notes"><sup>1</sup></a>: in general, | |||
| adding more software engineers does | |||
| not make a project go faster, it only | |||
| makes it grow bigger.</p> | |||
| <p><hr> | |||
| <h5><em> | |||
| Building software is not about avoiding failure; it is about strategically failing as fast as possible to get the information you need to build something good. | |||
| </em></h5> | |||
| <hr> | |||
| <p>Smaller teams of good engineers will | |||
| also create fewer bugs and security | |||
| problems than larger teams of average | |||
| engineers. Similar to writing an essay, the | |||
| more authors there are, the more coding | |||
| styles, assumptions, and quirks there | |||
| are to reconcile in the final composite | |||
| product, exposing a greater surface area | |||
| for potential issues to arise. In contrast, | |||
| a system built by a smaller team of | |||
| good engineers will be more concise, | |||
| coherent, and better understood by | |||
| its creators. You cannot have security | |||
| without simplicity, and simplicity is rarely | |||
| the result of large-scale collaborations.</p> | |||
| <p>The more collaborative an engineering | |||
| effort, the better the engineers need | |||
| to be. Problems in an engineer’s code | |||
| affect not just his work but that of his | |||
| colleagues as well. In large projects, | |||
| bad engineers end up creating more | |||
| work for one another, as errors and | |||
| poor design choices snowball to create | |||
| massive issues. Big projects need to | |||
| be built on solid reliable code modules | |||
| in an efficient design with very clear | |||
| assumptions laid out. The better your | |||
| engineers, the bigger your system can | |||
| get before it collapses under its own | |||
| weight. This is why the most successful | |||
| tech companies insist on the best talent | |||
| despite their massive size. The hard limit | |||
| to system complexity is not the quantity | |||
| of engineering effort, but its quality.</p> | |||
| <h2>Conclusion</h2> | |||
| <p>Good software development starts | |||
| with building a clear understanding of | |||
| the problem you want to solve. This | |||
| lets you test many possible solutions | |||
| and converge on a good approach. | |||
| Development is accelerated by reusing | |||
| the right open source code and cloud | |||
| services, granting immediate access | |||
| to established software systems and | |||
| sophisticated new technology. The | |||
| development cycle alternates between | |||
| exploration and consolidation, quickly | |||
| and messily progressing on new ideas, | |||
| then focusing and simplifying to keep | |||
| the complexity manageable. As the | |||
| project moves forward, it gets tested | |||
| with successively larger groups of people | |||
| to eliminate increasingly uncommon | |||
| problems. Launching is when the real work | |||
| ramps up for a good development team: | |||
| layers of automated systems should be | |||
| built to handle issues quickly and prevent | |||
| harm to actual users. Ultimately, while | |||
| there are infinite intricacies to software | |||
| development, understanding this process | |||
| provides a basis to tackle the complexities | |||
| of how to build good software.</p></p> | |||
| </article> | |||
| <hr> | |||
| <footer> | |||
| <p> | |||
| <a href="/david/" title="Aller à l’accueil"><svg class="icon icon-home"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-home"></use> | |||
| </svg> Accueil</a> • | |||
| <a href="/david/log/" title="Accès au flux RSS"><svg class="icon icon-rss2"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-rss2"></use> | |||
| </svg> RSS</a> • | |||
| <a href="http://larlet.com" title="Go to my English profile" data-instant><svg class="icon icon-user-tie"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-user-tie"></use> | |||
| </svg> Pro</a> • | |||
| <a href="mailto:david%40larlet.fr" title="Envoyer un courriel"><svg class="icon icon-mail"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-mail"></use> | |||
| </svg> Email</a> • | |||
| <abbr class="nowrap" title="Hébergeur : Alwaysdata, 62 rue Tiquetonne 75002 Paris, +33184162340"><svg class="icon icon-hammer2"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-hammer2"></use> | |||
| </svg> Légal</abbr> | |||
| </p> | |||
| <template id="theme-selector"> | |||
| <form> | |||
| <fieldset> | |||
| <legend><svg class="icon icon-brightness-contrast"> | |||
| <use xlink:href="/static/david/icons2/symbol-defs.svg#icon-brightness-contrast"></use> | |||
| </svg> Thème</legend> | |||
| <label> | |||
| <input type="radio" value="auto" name="chosen-color-scheme" checked> Auto | |||
| </label> | |||
| <label> | |||
| <input type="radio" value="dark" name="chosen-color-scheme"> Foncé | |||
| </label> | |||
| <label> | |||
| <input type="radio" value="light" name="chosen-color-scheme"> Clair | |||
| </label> | |||
| </fieldset> | |||
| </form> | |||
| </template> | |||
| </footer> | |||
| <script> | |||
| function loadThemeForm(templateName) { | |||
| const themeSelectorTemplate = document.querySelector(templateName) | |||
| const form = themeSelectorTemplate.content.firstElementChild | |||
| themeSelectorTemplate.replaceWith(form) | |||
| form.addEventListener('change', (e) => { | |||
| const chosenColorScheme = e.target.value | |||
| localStorage.setItem('theme', chosenColorScheme) | |||
| toggleTheme(chosenColorScheme) | |||
| }) | |||
| const selectedTheme = localStorage.getItem('theme') | |||
| if (selectedTheme && selectedTheme !== 'undefined') { | |||
| form.querySelector(`[value="${selectedTheme}"]`).checked = true | |||
| } | |||
| } | |||
| const prefersColorSchemeDark = '(prefers-color-scheme: dark)' | |||
| window.addEventListener('load', () => { | |||
| let hasDarkRules = false | |||
| for (const styleSheet of Array.from(document.styleSheets)) { | |||
| let mediaRules = [] | |||
| for (const cssRule of styleSheet.cssRules) { | |||
| if (cssRule.type !== CSSRule.MEDIA_RULE) { | |||
| continue | |||
| } | |||
| // WARNING: Safari does not have/supports `conditionText`. | |||
| if (cssRule.conditionText) { | |||
| if (cssRule.conditionText !== prefersColorSchemeDark) { | |||
| continue | |||
| } | |||
| } else { | |||
| if (cssRule.cssText.startsWith(prefersColorSchemeDark)) { | |||
| continue | |||
| } | |||
| } | |||
| mediaRules = mediaRules.concat(Array.from(cssRule.cssRules)) | |||
| } | |||
| // WARNING: do not try to insert a Rule to a styleSheet you are | |||
| // currently iterating on, otherwise the browser will be stuck | |||
| // in a infinite loop… | |||
| for (const mediaRule of mediaRules) { | |||
| styleSheet.insertRule(mediaRule.cssText) | |||
| hasDarkRules = true | |||
| } | |||
| } | |||
| if (hasDarkRules) { | |||
| loadThemeForm('#theme-selector') | |||
| } | |||
| }) | |||
| </script> | |||
| </body> | |||
| </html> | |||
+ 371
- 0
cache/2021/6be8b8649cd695b922a43d15f74d4773/index.md
View File
| @@ -0,0 +1,371 @@ | |||
| title: How to Build Good Software | |||
| url: https://www.csc.gov.sg/articles/how-to-build-good-software | |||
| hash_url: 6be8b8649cd695b922a43d15f74d4773 | |||
| <h2>Why Bad Software Happens to Good People</h2> | |||
| <p>Bad software is one of the few things in the world you cannot solve with money. Billion dollar airlines have flight search apps that are often inferior to those built by groups of students. Established taxi | |||
| companies the world over have terrible booking apps despite the threat they face from ride-sharing services. And painful corporate IT systems are usually projects with massive budgets, built over the course of many years. Whatever the cause of bad software is, it does not seem to be a lack of funding.</p> | |||
| <p>Surprisingly, the root cause of bad software has less to do with specific engineering choices, and more to do with how development projects are managed. The worst software projects often proceed in a very particular way:</p> | |||
| <p>The project owners start out wanting to build a specific solution and never | |||
| explicitly identify the problem they are trying to solve. They then gather a long list of requirements from a large group of stakeholders. This list is then handed off to a correspondingly large external development team, who get to work building this highly customised piece of software from scratch. Once all the requirements are met, everyone celebrates as the system is launched and the project is declared complete.</p> | |||
| <hr> | |||
| <h5><em> | |||
| The root cause of bad software has less to do with specific engineering choices, and more to do with how development projects are managed. | |||
| </em></h5> | |||
| <hr> | |||
| <p> However, though the system technically meets specifications, severe issues are | |||
| found when it is put in the hands of actual users. It is slow, confusing, and | |||
| filled with subtle bugs that make using it an exercise in frustration. Unfortunately, | |||
| by this time the external development team has been dismissed and there | |||
| are no resources left over to make the necessary fixes. By the time a new project | |||
| can be initiated years later, all knowledge of what caused these problems has left | |||
| the organisation and the cycle starts over again.</p> | |||
| <p></p> | |||
| <p class="small-text">A Conversation with Li Hongyi (Part 1)<br> | |||
| <em>Scroll down for more videos</em></p> | |||
| <p>The right coding language, system architecture, or interface design will | |||
| vary wildly from project to project. But there are characteristics particular | |||
| to software that consistently cause traditional management practices to fail, while allowing small startups to succeed with a shoestring budget:</p> | |||
| <p>• Reusing good software is easy; | |||
| it is what allows you to build good | |||
| things quickly;<br> | |||
| • Software is limited not by the amount | |||
| of resources put into building it, but | |||
| by how complex it can get before it | |||
| breaks down; and<br> | |||
| • The main value in software is not the | |||
| code produced, but the knowledge | |||
| accumulated by the people who | |||
| produced it.</p> | |||
| <p> | |||
| </p> | |||
| <p>Understanding these characteristics may not guarantee good outcomes, but it | |||
| does help clarify why so many projects produce bad outcomes. Furthermore, | |||
| these lead to some core operating principles that can dramatically improve | |||
| the chances of success:</p> | |||
| <p> | |||
| </p> | |||
| <p>1. Start as simple as possible;<br> | |||
| 2. Seek out problems and iterate; and<br> | |||
| 3. Hire the best engineers you can.</p> | |||
| <p>While there are many subtler factors to consider, these principles form a foundation that lets you get started building good software.</p> | |||
| <h2>Reusing Software Lets You Build Good Things Quickly</h2> | |||
| <p>Software is easy to copy. At a mechanical level, lines of code can literally be copied and pasted onto another computer. More generally, the internet is full of tutorials on how to build different kinds of systems using ready-made code modules that are available online. Modern software is almost never developed from scratch. Even the most innovative applications are built using existing software that has been combined and modified to achieve a new result.</p> | |||
| <p>The biggest source of reusable code modules is the open source community. Open source software is software in which code is freely published for anyone to see and use. Many of the largest contributors to the open source community are giant tech companies. If you want to use a state-of-the-art planet scalable database as Facebook does, just download the code for Cassandra that they open sourced in 2008. If you want to try out Google’s cutting-edge machine learning for yourself, download the TensorFlow system published in 2015. Using open source code does not just make your application development faster, it gives you access to technology that is far more sophisticated than anything | |||
| you could have developed yourself. For the most popular open source code, it is even more secure as there are many more people paying attention and fixing vulnerabilities. This is the reason digital technology has made such rapid progress: even the newest engineers can build upon the most advanced tools our profession has to offer.</p> | |||
| <p>The advent of cloud services has taken reusability even further, offering the full use of even proprietary systems for just a subscription fee. Need a simple website? Just configure one in a few clicks using a website building service like Squarespace or Wix. A database? Subscribe to a virtual one from Amazon Web Services or Microsoft Azure. Cloud services allow developers to benefit from specialisation; the service provider handles the setup, maintenance, and continued development of a reliable, high-quality piece of software that is used by all its subscribers. This allows software developers to stop wasting time on solved problems and instead focus on delivering actual value.</p> | |||
| <p>You cannot make technological progress if all your time is spent on rebuilding existing technology. Software engineering is about building automated systems, and one of the first things that gets automated away is routine software engineering work. The point is to understand what the right systems to reuse are, how to customise them to fit your unique requirements, and fixing novel problems | |||
| discovered along the way.</p> | |||
| <hr> | |||
| <h5><em> | |||
| Software engineering is about building automated systems, and one of the first things that gets automated away is routine software engineering work. | |||
| </em></h5> | |||
| <hr> | |||
| <h2>Software Is Limited by Complexity</h2> | |||
| <p>How useful a piece of software can be is usually limited by its complexity rather than the amount of resources invested in building it.</p> | |||
| <p>IT systems are often full of features but are still hated by users because of how confusing they become. In contrast, highly ranked mobile apps tend to be lauded for their simplicity and | |||
| intuitiveness. Learning to use software is hard. Beyond a point, new features actually make things worse for users because the accumulated complexity starts to become overwhelming. For example, after serving as the hub of Apple’s media ecosystem for almost 20 years, iTunes was split into three | |||
| different apps (for music, podcasts, and TV shows) this year, as its features had grown too complex for one app to handle. From a usability perspective, the limit is not how many features can be implemented, but rather what can fit into a simple intuitive interface.</p> | |||
| <p>Even ignoring usability, engineering progress slows to a halt once a project becomes too complex. Each new line of code added to an application has a chance of interacting with every other | |||
| line. The bigger an application’s codebase, the more bugs are introduced whenever a new feature is built. Eventually, the rate of work created from new bugs cancels out the rate of work done from feature development. This is known as “technical debt” and is the main challenge in professional software development. It is the reason why many large IT systems have issues that go unfixed for years. | |||
| Adding more engineers to the project just adds to the chaos: they start running | |||
| faster in place as the codebase keels over from its own weight.</p> | |||
| <hr> | |||
| <h5><em> | |||
| Building good software involves alternating cycles of expanding and reducing complexity. | |||
| </em></h5> | |||
| <hr> | |||
| <p>In such cases, the only way forward is to take a step back to rationalise and | |||
| simplify the codebase. The system architecture can be redesigned to limit | |||
| unexpected interactions. Non-critical features can be removed even if they | |||
| have already been built. Automated tools can be deployed to check for bugs and | |||
| badly written code. Bill Gates once said “Measuring programming progress by | |||
| lines of code is like measuring aircraft building progress by weight”. Human minds can only handle a finite amount of complexity, so how sophisticated a software system can get depends on | |||
| how efficiently this complexity budget is used.</p> | |||
| <p>Building good software involves alternating cycles of expanding and reducing complexity. As new features are developed, disorder naturally accumulates in the system. When this messiness | |||
| starts to cause problems, progress is suspended to spend time cleaning up. This two-step process is necessary because there is no such thing as platonically good engineering: it depends on your | |||
| needs and the practical problems you encounter. Even a simple user interface such as Google’s search bar contains a massive amount of complexity under the surface that cannot be perfected in a single iteration. The challenge is managing this cycle, letting it get messy enough to make meaningful progress, | |||
| but not letting it get so complicated that it becomes overwhelming.</p> | |||
| <hr> | |||
| <h5><em> | |||
| There is no such thing as platonically good engineering: it depends on your needs and the | |||
| practical problems you encounter. | |||
| </em></h5> | |||
| <hr> | |||
| <h2>Software Is about Developing Knowledge More than Writing Code</h2> | |||
| <p>In software development, most ideas are bad; this is not anyone’s fault. It is just | |||
| that the number of possible ideas is so large that any particular idea is probably | |||
| not going to work, even if it was chosen very carefully and intelligently. To make | |||
| progress, you need to start with a bunch of bad ideas, discard the worst, and | |||
| evolve the most promising ones. Apple, a paragon of visionary design, goes | |||
| through dozens of prototypes before landing on a final product. The final | |||
| product may be deceptively simple; it is the intricate knowledge of why this | |||
| particular solution was chosen over its alternatives that allows it to be good.</p> | |||
| <p>This knowledge continues to be important even after the product is built. If a new | |||
| team takes over the code for an unfamiliar piece of software, the software will soon | |||
| start to degrade. Operating systems will update, business requirements | |||
| will change, and security problems will be discovered that need to be fixed. Handling these subtle errors is often harder than building the software in the first place, since it requires intimate | |||
| knowledge of the system’s architecture and design principles.</p> | |||
| <p></p> | |||
| <p class="small-text">A Conversation with Li Hongyi (Part 2)<br> | |||
| <em>Scroll down for more videos</em></p> | |||
| <p>In the short term, an unfamiliar development team can address these problems with | |||
| stopgap fixes. Over time though, new bugs accumulate due to the makeshift nature | |||
| of the additional code. User interfaces become confusing due to mismatched | |||
| design paradigms, and system complexity increases as a whole. Software should be | |||
| treated not as a static product, but as a living manifestation of the development | |||
| team’s collective understanding.</p> | |||
| <hr> | |||
| <h5><em> | |||
| Software should be treated not as a static product, but as a | |||
| living manifestation of the development team’s collective understanding. | |||
| </em></h5> | |||
| <hr> | |||
| <p>This is why relying on external vendors for your core software development is | |||
| difficult. You may get a running system and its code, but the invaluable knowledge | |||
| of how it is built and what design choices were made leaves your organisation. | |||
| This is also why handing a system over to new vendors for “maintenance” often | |||
| causes problems. Even if the system is very well documented, some knowledge | |||
| is lost every time a new team takes over. Over the years, the system becomes a | |||
| patchwork of code from many different authors. It becomes harder and harder | |||
| to keep running; eventually, there is no one left who truly understands how | |||
| it works.</p> | |||
| <p>For your software to keep working well in the long term, it is important to | |||
| have your staff learning alongside the external help to retain critical engineering | |||
| knowledge in your organisation.</p> | |||
| <h2>3 Principles for Good Software Development</h2> | |||
| <h4>1. Start as Simple as Possible</h4> | |||
| <p>Projects that set out to be a “one-stop | |||
| shop” for a particular domain are often | |||
| doomed. The reasoning seems sensible | |||
| enough: What better way to ensure your | |||
| app solves people’s problems than by | |||
| having it address as many as possible? | |||
| After all, this works for physical stores | |||
| such as supermarkets. The difference is | |||
| that while it is relatively easy to add a | |||
| new item for sale once a physical store | |||
| is set up, an app with twice as many | |||
| features is more than twice as hard to | |||
| build and much harder to use.</p> | |||
| <p>Building good software requires focus: | |||
| starting with the simplest solution that | |||
| could solve the problem. A well-made | |||
| but simplistic app never has problems | |||
| adding necessary features. But a big IT | |||
| system that does a lot of things poorly | |||
| is usually impossible to simplify and | |||
| fix. Even successful “do it all” apps like | |||
| WeChat, Grab, and Facebook started | |||
| out with very specific functionality and | |||
| only expanded after they had secured | |||
| their place. Software projects rarely | |||
| fail because they are too small; they fail | |||
| because they get too big.</p> | |||
| <hr> | |||
| <h5><em> | |||
| Software projects rarely fail because they | |||
| are too small; they fail because they get too big. | |||
| </em></h5> | |||
| <hr> | |||
| <p>Unfortunately, keeping a project focused | |||
| is very hard in practice: just gathering | |||
| the requirements from all stakeholders | |||
| already creates a huge list of features.</p> | |||
| <p></p> | |||
| <p class="small-text">A Conversation with Li Hongyi (Part 3)<br> | |||
| <em>Scroll down for last video</em></p> | |||
| <p>One way to manage this bloat is by | |||
| using a priority list. Requirements are | |||
| all still gathered, but each are tagged | |||
| according to whether they are absolutely | |||
| critical features, high-value additions, | |||
| or nice-to-haves. This creates a much | |||
| lower-tension planning process because | |||
| features no longer need to be explicitly | |||
| excluded. Stakeholders can then more | |||
| sanely discuss which features are the | |||
| most important, without worrying about | |||
| something being left out of the project. | |||
| This approach also makes explicit the | |||
| trade-offs of having more features. | |||
| Stakeholders who want to increase | |||
| the priority for a feature have to also | |||
| consider what features they are willing | |||
| to deprioritise. Teams can start on the | |||
| most critical objectives, working their way | |||
| down the list as time and resources allow.</p> | |||
| <p>We followed a similar process for all | |||
| our most successful apps. Form.gov.sg | |||
| started out as a manual Outlook Macro | |||
| that took us six hours to set up for | |||
| our first user but today has processed | |||
| about a million public submissions. | |||
| Data.gov.sg started out as a direct copy | |||
| of an open source project and has since | |||
| grown to over 300,000 monthly visits. | |||
| Parking.sg had a massive list of almost | |||
| 200 possible features that we never | |||
| got around to building but still has over | |||
| 1.1 million users today. These systems | |||
| are well received not in spite of their | |||
| simplicity but because of it.</p> | |||
| <h4>2. Seek Out Problems and Iterate</h4> | |||
| <p>In truth, modern software is so complicated and changes so rapidly that no amount of | |||
| planning will eliminate all shortcomings. Like writing a good paper, awkward | |||
| early drafts are necessary to get a feel of what the final paper should be. To | |||
| build good software, you need to first build bad software, then actively seek out problems to improve on your solution.</p> | |||
| <p>This starts with something as simple as talking to the actual people you are trying to help. The goal is to understand the root problem you want to solve and avoid jumping to a solution based just on preconceived biases. When we first started on Parking.sg, our hypothesis was that enforcement officers found it frustrating to have to keep doing the mental calculations regarding paper coupons. However, after spending just one afternoon with an experienced officer, we discovered that | |||
| doing these calculations was actually quite simple for someone doing it professionally. That single conversation saved us months of potentially wasted effort and let us refocus our project on helping drivers instead. </p> | |||
| <p>Beware of bureaucratic goals masquerading as problem statements. “Drivers feel frustrated when dealing with parking coupons” is a problem. “We need to build an app for drivers as part of our Ministry Family Digitisation Plans” is not. “Users are annoyed at how hard it is to find information on government websites” is a problem. “As part of the Digital Government Blueprint, we need to rebuild our websites to conform to the new design service standards” is not. If our end goal is to make citizens’ lives better, we need to explicitly acknowledge the things that are making their lives worse.</p> | |||
| <p>Having a clear problem statement lets you experimentally test the viability of different solutions that are too hard to determine theoretically. Talking to a chatbot may not be any easier than navigating a website, and users may not want to install yet another app on their phones no matter how secure it makes the country. With software, apparently obvious solutions often have fatal flaws that do not show up until they are put to use. The aim is not yet to build the final product, but to first identify these problems as quickly and as cheaply as possible. Non-functional mock-ups to test interface designs. Semi-functional mock-ups to try different features. Prototype code, written hastily, could help garner feedback more quickly. Anything created at this stage should be treated as disposable. The desired output | |||
| of this process is not the code written, but a clearer understanding of what the right thing to build is.</p> | |||
| <hr> | |||
| <h5><em> | |||
| Beware of bureaucratic goals masquerading as problem statements. If our end goal is to make citizens’ lives better, we need to explicitly acknowledge the things that are making their lives worse. | |||
| </em></h5> | |||
| <hr> | |||
| <p>With a good understanding of the right solution, you can start work on building the actual product. You stop exploring new ideas and narrow down to identifying problems with your particular implementation. Begin with a small number of testers who will quickly spot the obvious bugs that need to be fixed. As problems are addressed, you can increasingly open up to a larger pool who will find more esoteric issues.</p> | |||
| <p>Most people only give feedback once. If you start by launching to a large audience, everyone will give you the same obvious feedback and you’ll have nowhere to go from there. Even the best product ideas built by the best engineers will start out with significant issues. The aim is to repeatedly refine the output, | |||
| sanding down rough edges until a good product emerges.</p> | |||
| <p>Even after all this iteration, after launch is when problems with a product matter | |||
| the most. A problem that happens only 0.1% of the time may not get noticed | |||
| during testing. But once you have a million users, every day the problem | |||
| goes unresolved is a thousand more angry people you have to deal with. You need to fix problems caused by new mobile devices, network outages, or security attacks before they cause | |||
| substantial harm to your users. With Parking.sg we built a series of secondary | |||
| systems that continuously check the main system for any discrepancies in | |||
| payments, duplicate parking sessions, and application crashes. Building up | |||
| an “immune system” over time lets you avoid being overwhelmed as new issues | |||
| inevitably come up.</p> | |||
| <p>Overall, the approach is to use these different feedback loops to efficiently | |||
| identify problems. Small feedback loops allow for quick and easy correction | |||
| but miss out on broader issues. Large feedback loops catch broader issues but | |||
| are slow and expensive. You want to use both, resolving as much as possible with | |||
| tight loops while still having wide loops to catch unexpected errors. Building | |||
| software is not about avoiding failure; it is about strategically failing as fast | |||
| as possible to get the information you need to build something good.</p> | |||
| <h4>3. Hire the Best Engineers You Can</h4> | |||
| <p>The key to having good engineering is having good engineers. Google, | |||
| Facebook, Amazon, Netflix, and Microsoft all run a dizzying number of the largest | |||
| technology systems in the world, yet, they famously have some of the most | |||
| selective interview processes while still competing fiercely to recruit the strongest | |||
| candidates. There is a reason that the salaries for even fresh graduates have | |||
| gone up so much as these companies have grown, and it is not because they | |||
| enjoy giving away money.</p> | |||
| <p>Both Steve Jobs and Mark Zuckerberg have said that the best engineers are at | |||
| least 10 times more productive than an average engineer. This is not because | |||
| good engineers write code 10 times faster. It is because they make better | |||
| decisions that save 10 times the work.</p> | |||
| <p>A good engineer has a better grasp of | |||
| existing software they can reuse, thus | |||
| minimising the parts of the system they | |||
| have to build from scratch. They have | |||
| a better grasp of engineering tools, | |||
| automating away most of the routine | |||
| aspects of their own job. Automation | |||
| also means freeing up humans to work | |||
| on solving unexpected errors, which the | |||
| best engineers are disproportionately | |||
| better at. Good engineers themselves | |||
| design systems that are more robust | |||
| and easier to understand by others. | |||
| This has a multiplier effect, letting their | |||
| colleagues build upon their work much | |||
| more quickly and reliably. Overall, good | |||
| engineers are so much more effective | |||
| not because they produce a lot more | |||
| code, but because the decisions they | |||
| make save you from work you did not | |||
| know could be avoided.</p> | |||
| <p></p> | |||
| <p class="small-text">A Conversation with Li Hongyi (Part 4)</p> | |||
| <p>This also means that small teams of the | |||
| best engineers can often build things | |||
| faster than even very large teams of | |||
| average engineers. They make good | |||
| use of available open source code and | |||
| sophisticated cloud services, and offload | |||
| mundane tasks onto automated testing | |||
| and other tools, so they can focus on the creative problem-solving aspects of | |||
| the job. They rapidly test different ideas | |||
| with users by prioritising key features | |||
| and cutting out unimportant work. This | |||
| is the central thesis of the classic book | |||
| “The <em>Mythical Man-Month</em>”<a href="#notes"><sup>1</sup></a>: in general, | |||
| adding more software engineers does | |||
| not make a project go faster, it only | |||
| makes it grow bigger.</p> | |||
| <hr> | |||
| <h5><em> | |||
| Building software is not about avoiding failure; it is about strategically failing as fast as possible to get the information you need to build something good. | |||
| </em></h5> | |||
| <hr> | |||
| <p>Smaller teams of good engineers will | |||
| also create fewer bugs and security | |||
| problems than larger teams of average | |||
| engineers. Similar to writing an essay, the | |||
| more authors there are, the more coding | |||
| styles, assumptions, and quirks there | |||
| are to reconcile in the final composite | |||
| product, exposing a greater surface area | |||
| for potential issues to arise. In contrast, | |||
| a system built by a smaller team of | |||
| good engineers will be more concise, | |||
| coherent, and better understood by | |||
| its creators. You cannot have security | |||
| without simplicity, and simplicity is rarely | |||
| the result of large-scale collaborations.</p> | |||
| <p>The more collaborative an engineering | |||
| effort, the better the engineers need | |||
| to be. Problems in an engineer’s code | |||
| affect not just his work but that of his | |||
| colleagues as well. In large projects, | |||
| bad engineers end up creating more | |||
| work for one another, as errors and | |||
| poor design choices snowball to create | |||
| massive issues. Big projects need to | |||
| be built on solid reliable code modules | |||
| in an efficient design with very clear | |||
| assumptions laid out. The better your | |||
| engineers, the bigger your system can | |||
| get before it collapses under its own | |||
| weight. This is why the most successful | |||
| tech companies insist on the best talent | |||
| despite their massive size. The hard limit | |||
| to system complexity is not the quantity | |||
| of engineering effort, but its quality.</p> | |||
| <h2>Conclusion</h2> | |||
| <p>Good software development starts | |||
| with building a clear understanding of | |||
| the problem you want to solve. This | |||
| lets you test many possible solutions | |||
| and converge on a good approach. | |||
| Development is accelerated by reusing | |||
| the right open source code and cloud | |||
| services, granting immediate access | |||
| to established software systems and | |||
| sophisticated new technology. The | |||
| development cycle alternates between | |||
| exploration and consolidation, quickly | |||
| and messily progressing on new ideas, | |||
| then focusing and simplifying to keep | |||
| the complexity manageable. As the | |||
| project moves forward, it gets tested | |||
| with successively larger groups of people | |||
| to eliminate increasingly uncommon | |||
| problems. Launching is when the real work | |||
| ramps up for a good development team: | |||
| layers of automated systems should be | |||
| built to handle issues quickly and prevent | |||
| harm to actual users. Ultimately, while | |||
| there are infinite intricacies to software | |||
| development, understanding this process | |||
| provides a basis to tackle the complexities | |||
| of how to build good software.</p> | |||
+ 4
- 0
cache/2021/index.html
View File
| @@ -71,6 +71,8 @@ | |||
| <li><a href="/david/cache/2021/ab9f293c34ce421ab41465fcda8893fe/" title="Accès à l’article dans le cache local : Whistleblower Says Facebook Ignored Global Political Manipulation">Whistleblower Says Facebook Ignored Global Political Manipulation</a> (<a href="https://www.buzzfeednews.com/article/craigsilverman/facebook-ignore-political-manipulation-whistleblower-memo" title="Accès à l’article original distant : Whistleblower Says Facebook Ignored Global Political Manipulation">original</a>)</li> | |||
| <li><a href="/david/cache/2021/6be8b8649cd695b922a43d15f74d4773/" title="Accès à l’article dans le cache local : How to Build Good Software">How to Build Good Software</a> (<a href="https://www.csc.gov.sg/articles/how-to-build-good-software" title="Accès à l’article original distant : How to Build Good Software">original</a>)</li> | |||
| <li><a href="/david/cache/2021/3d37e77a601b6dd7043bbb17a4a2d199/" title="Accès à l’article dans le cache local : Le compromis de la vulgarisation">Le compromis de la vulgarisation</a> (<a href="https://tanialouis.fr/2020/12/27/le-compromis-de-la-vulgarisation/" title="Accès à l’article original distant : Le compromis de la vulgarisation">original</a>)</li> | |||
| <li><a href="/david/cache/2021/9c9968045ffe697a08dc13df854247f1/" title="Accès à l’article dans le cache local : I should have loved biology">I should have loved biology</a> (<a href="https://jsomers.net/i-should-have-loved-biology/" title="Accès à l’article original distant : I should have loved biology">original</a>)</li> | |||
| @@ -135,6 +137,8 @@ | |||
| <li><a href="/david/cache/2021/cfcd10768187ce1c3e598136cd8838b2/" title="Accès à l’article dans le cache local : Bad News Wrapped in Protein: Inside the Coronavirus Genome">Bad News Wrapped in Protein: Inside the Coronavirus Genome</a> (<a href="https://www.nytimes.com/interactive/2020/04/03/science/coronavirus-genome-bad-news-wrapped-in-protein.html" title="Accès à l’article original distant : Bad News Wrapped in Protein: Inside the Coronavirus Genome">original</a>)</li> | |||
| <li><a href="/david/cache/2021/6514143dca5d96bf9e751236b800fba5/" title="Accès à l’article dans le cache local : Are We Really Engineers?">Are We Really Engineers?</a> (<a href="https://www.hillelwayne.com/post/crossover-project/are-we-really-engineers/" title="Accès à l’article original distant : Are We Really Engineers?">original</a>)</li> | |||
| <li><a href="/david/cache/2021/c177668b263f39d20788f002446d2a47/" title="Accès à l’article dans le cache local : 6-month consequences of COVID-19 in patients discharged from hospital: a cohort study">6-month consequences of COVID-19 in patients discharged from hospital: a cohort study</a> (<a href="https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)32656-8/fulltext" title="Accès à l’article original distant : 6-month consequences of COVID-19 in patients discharged from hospital: a cohort study">original</a>)</li> | |||
| <li><a href="/david/cache/2021/0c6966a8e9543b52c361ac6de68f08e4/" title="Accès à l’article dans le cache local : Understanding ProRAW">Understanding ProRAW</a> (<a href="https://blog.halide.cam/understanding-proraw-4eed556d4c54" title="Accès à l’article original distant : Understanding ProRAW">original</a>)</li> | |||
Loading…