A place to cache linked articles (think custom and personal wayback machine)
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273
  1. title: The sad state of sysadmin in the age of containers
  2. url: http://www.vitavonni.de/blog/201503/2015031201-the-sad-state-of-sysadmin-in-the-age-of-containers.html
  3. hash_url: 5c5d6283fd3b87a52a81fea2a7b4c940
  4. <p>System administration is in a sad state. It in a mess.</p>
  5. <p>I'm not complaining about old-school sysadmins. They know how to keep
  6. systems running, manage update and upgrade paths.</p>
  7. <p>This rant is about containers, prebuilt VMs, and the incredible mess they
  8. cause because their concept lacks notions of "trust" and "upgrades".</p>
  9. <p>Consider for example Hadoop. <b>Nobody seems to know how to build Hadoop
  10. from scratch.</b> It's an incredible mess of dependencies, version requirements
  11. and build tools.</p>
  12. <p>None of these "fancy" tools still builds by a traditional <tt>make</tt>
  13. command. Every tool has to come up with their own, incomptaible, and
  14. non-portable "method of the day" of building.</p>
  15. <p>And since nobody is still able to compile things from scratch,
  16. <b>everybody just downloads precompiled binaries from random websites</b>.
  17. Often <b>without any authentication or signature</b>.</p>
  18. <p>NSA and virus heaven. <b>You don't need to exploit any security hole
  19. anymore.</b> Just make an "app" or "VM" or "Docker" image, and have people
  20. load your malicious binary to their network.</p>
  21. <p>The <a href="https://wiki.debian.org/Hadoop">Hadoop Wiki Page</a> of
  22. Debian is a typical example. Essentially, people have given up in 2010 to
  23. be able build Hadoop from source for Debian and offer nice packages.</p>
  24. <p>To build Apache Bigtop, you apparently first have to install puppet3.
  25. Let it download magic data from the internet.
  26. Then it tries to run <tt>sudo puppet</tt> to enable the NSA backdoors
  27. (for example, it will download and install an outdated precompiled
  28. JDK, because it considers you too stupid to install Java.)
  29. And then hope the gradle build doesn't throw a 200 line useless backtrace.</p>
  30. <p>I am not joking. It will try to execute commands such as e.g.
  31. </p><blockquote>
  32. <tt>/bin/bash -c "wget http://www.scala-lang.org/files/archive/scala-2.10.3.deb ; dpkg -x ./scala-2.10.3.deb /"</tt>
  33. </blockquote><p>
  34. Note that it doesn't even <i>install</i> the package properly, but extracts
  35. it to your root directory. The download does not check any signature, not even
  36. SSL certificates. (Source:
  37. <a href="https://github.com/apache/bigtop/blob/master/bigtop_toolchain/manifests/scala.pp">Bigtop puppet manifests</a>)
  38. </p>
  39. <p>Even if your build would work, it will involve Maven downloading
  40. unsigned binary code from the internet, and use that for building.</p>
  41. <p>Instead of writing clean, modular architecture, everything these days
  42. morphs into a huge mess of interlocked dependencies. Last I checked, the
  43. Hadoop classpath was already over 100 jars. I bet it is now 150, without
  44. even using any of the HBaseGiraphFlumeCrunchPigHiveMahoutSolrSparkElasticsearch
  45. (or any other of the Apache chaos) mess yet.</p>
  46. <p><b>Stack</b> is the new term for "I have no idea what I'm actually
  47. using".</p>
  48. <p><b>Maven</b>, <b>ivy</b> and <b>sbt</b> are the go-to tools for having
  49. your system download unsigned binary data from the internet and run it on your
  50. computer.</p>
  51. <p>And with containers, this mess gets even worse.</p>
  52. <p>Ever tried to <b>security update</b> a container?</p>
  53. <p>Essentially, the Docker approach boils down to downloading an
  54. unsigned binary, running it, and hoping it doesn't contain any backdoor
  55. into your companies network.</p>
  56. <p>Feels like downloading Windows shareware in the 90s to me.</p>
  57. <p>When will the first docker image appear which contains the Ask
  58. toolbar? The first internet worm spreading via flawed docker images?</p>
  59. <hr/>
  60. <p>Back then, years ago, Linux distributions were trying to provide you
  61. with a safe operating system. With signed packages, built from a web of trust.
  62. Some even work on reproducible builds.</p>
  63. <p>But then, everything got Windows-ized. "Apps" were the rage, which you
  64. download and run, without being concerned about security, or the ability to
  65. upgrade the application to the next version. Because "you only live
  66. once".</p>
  67. <p><b>Update:</b> it was pointed out that this started way before Docker:
  68. « <em>Docker is the new '<tt>curl | sudo bash</tt>'</em> ». That's right,
  69. but it's now pretty much mainstream to download and run untrusted software
  70. in your "datacenter". That is bad, really bad. Before, admins would try hard
  71. to prevent security holes, now they call themselves "devops" and happily
  72. introduce them to the network themselves!</p>