larlet-fr-david-cache/cache/5c5d6283fd3b87a52a81fea2a7b4c940/index.md

title: The sad state of sysadmin in the age of containers
url: http://www.vitavonni.de/blog/201503/2015031201-the-sad-state-of-sysadmin-in-the-age-of-containers.html
hash_url: 5c5d6283fd3b87a52a81fea2a7b4c940

<p>System administration is in a sad state. It in a mess.</p>
<p>I'm not complaining about old-school sysadmins. They know how to keep
systems running, manage update and upgrade paths.</p>
<p>This rant is about containers, prebuilt VMs, and the incredible mess they
cause because their concept lacks notions of "trust" and "upgrades".</p>
<p>Consider for example Hadoop. <b>Nobody seems to know how to build Hadoop
from scratch.</b> It's an incredible mess of dependencies, version requirements
and build tools.</p>
<p>None of these "fancy" tools still builds by a traditional <tt>make</tt>
command. Every tool has to come up with their own, incomptaible, and
non-portable "method of the day" of building.</p>
<p>And since nobody is still able to compile things from scratch,
<b>everybody just downloads precompiled binaries from random websites</b>.
Often <b>without any authentication or signature</b>.</p>
<p>NSA and virus heaven. <b>You don't need to exploit any security hole
anymore.</b> Just make an "app" or "VM" or "Docker" image, and have people
load your malicious binary to their network.</p>
<p>The <a href="https://wiki.debian.org/Hadoop">Hadoop Wiki Page</a> of
Debian is a typical example. Essentially, people have given up in 2010 to
be able build Hadoop from source for Debian and offer nice packages.</p>
<p>To build Apache Bigtop, you apparently first have to install puppet3.
Let it download magic data from the internet.
Then it tries to run <tt>sudo puppet</tt> to enable the NSA backdoors
(for example, it will download and install an outdated precompiled
JDK, because it considers you too stupid to install Java.)
And then hope the gradle build doesn't throw a 200 line useless backtrace.</p>
<p>I am not joking. It will try to execute commands such as e.g.
</p><blockquote>
<tt>/bin/bash -c "wget http://www.scala-lang.org/files/archive/scala-2.10.3.deb ; dpkg -x ./scala-2.10.3.deb /"</tt>
</blockquote><p>
Note that it doesn't even <i>install</i> the package properly, but extracts
it to your root directory. The download does not check any signature, not even
SSL certificates. (Source:
<a href="https://github.com/apache/bigtop/blob/master/bigtop_toolchain/manifests/scala.pp">Bigtop puppet manifests</a>)
</p>
<p>Even if your build would work, it will involve Maven downloading
unsigned binary code from the internet, and use that for building.</p>
<p>Instead of writing clean, modular architecture, everything these days
morphs into a huge mess of interlocked dependencies. Last I checked, the
Hadoop classpath was already over 100 jars. I bet it is now 150, without
even using any of the HBaseGiraphFlumeCrunchPigHiveMahoutSolrSparkElasticsearch
(or any other of the Apache chaos) mess yet.</p>
<p><b>Stack</b> is the new term for "I have no idea what I'm actually
using".</p>
<p><b>Maven</b>, <b>ivy</b> and <b>sbt</b> are the go-to tools for having
your system download unsigned binary data from the internet and run it on your
computer.</p>
<p>And with containers, this mess gets even worse.</p>
<p>Ever tried to <b>security update</b> a container?</p>
<p>Essentially, the Docker approach boils down to downloading an
unsigned binary, running it, and hoping it doesn't contain any backdoor
into your companies network.</p>
<p>Feels like downloading Windows shareware in the 90s to me.</p>
<p>When will the first docker image appear which contains the Ask
toolbar? The first internet worm spreading via flawed docker images?</p>
<hr/>
<p>Back then, years ago, Linux distributions were trying to provide you
with a safe operating system. With signed packages, built from a web of trust.
Some even work on reproducible builds.</p>
<p>But then, everything got Windows-ized. "Apps" were the rage, which you
download and run, without being concerned about security, or the ability to
upgrade the application to the next version. Because "you only live
once".</p>
<p><b>Update:</b> it was pointed out that this started way before Docker:
« <em>Docker is the new '<tt>curl | sudo bash</tt>'</em> ». That's right,
but it's now pretty much mainstream to download and run untrusted software
in your "datacenter". That is bad, really bad. Before, admins would try hard
to prevent security holes, now they call themselves "devops" and happily
introduce them to the network themselves!</p>