nix-super/doc/manual/introduction.xml

<chapter>
  <title>Introduction</title>

  <sect1>
    <title>The problem space</title>

    <para>
      Nix is a system for controlling the automatic creation and distribution
      of data, such as computer programs and other software artifacts.  This is
      a very general problem, and there are many applications that fall under
      this description.
    </para>

    <sect2>
      <title>Build management</title>

      <para>
	Build management tools are used to perform <emphasis>software
	  builds</emphasis>, that is, the construction of derived products
	(<emphasis>derivates)</emphasis>) such as executable programs from
	source code.  A commonly used build tool is Make, which is a standard
	tool on Unix systems. These tools have to deal with several issues:
	<itemizedlist>

	  <listitem>
	    <para>
	      <emphasis>Efficiency</emphasis>.  Since building large systems
	      can take a substantial amount of time, it is desirable that build
	      steps that have been performed in the past are not repeated
	      unnecessarily, i.e., if a new build differs from a previous build
	      only with respect to certain sources, then only the build steps
	      that (directly or indirectly) <emphasis>depend</emphasis> on
	      those sources should be redone.
	    </para>
	  </listitem>

	  <listitem>
	    <para>
	      <emphasis>Correctness</emphasis> is this context means that the
	      derivates produced by a build are always consistent with the
	      sources, that is, they are equal to what we would get if we were
	      to build the derivates from those sources.  This requirement is
	      trivially met when we do a full, unconditional build, but is far
	      from trivial under the requirement of efficiency, since it is not
	      easy to determine which derivates are affected by a change to a
	      source.
	    </para>
	  </listitem>

	  <listitem>
	    <para>
	      <emphasis>Variability</emphasis> is the property that a software
	      system can be built in a (potentially large) number of variants.
	      Variation exists both in <emphasis>time</emphasis>---the
	      evolution of different versions of an artifact---and in
	      <emphasis>space</emphasis>---the artifact might have
	      configuration options that lead to variants that differ in the
	      features they support (for example, a system might be built with
	      or without debugging information).
	    </para>

	    <para>
	      Build managers historically have had good support for variation
	      in time (rebuilding the system in an intelligent way when sources
	      change is one of the primary reasons to use a build manager), but
	      not always for variation in space.  For example,
	      <command>make</command> will not automatically ensure that
	      variant builds are properly isolated from each other (they will
	      in fact overwrite each other unless special precautions are
	      taken).
	    </para>
	  </listitem>

	  <listitem>
	    <para>
	      <emphasis>High-level system modelling language</emphasis>.  The
	      language in which one describes what and how derivates are to be
	      produced should have sufficient abstraction facilities to make it
	      easy to specify the derivation of even very large systems.  Also,
	      the language should be <emphasis>modular</emphasis> to enable
	      components from possible different sources to be easily combined.
	    </para>
	  </listitem>

	</itemizedlist>
      </para>

    </sect2>

    <sect2>
      <title>Package management</title>

      <para>
	After software has been built, is must also be
	<emphasis>deployed</emphasis> in the intended target environment, e.g.,
	the user's workstation.  Examples include the Red Hat package manager
	(RPM), Microsoft's MSI, and so on.  Here also we have several issues to
	contend with:
	<itemizedlist>
	  <listitem>
	    <para>
	      The <emphasis>creation</emphasis> of packages from some formal
	      description of what artifacts should be distributed in the
	      package.
	    </para>
	  </listitem>
	  <listitem>
	    <para>
	      The <emphasis>deployment</emphasis> of packages, that is, the
	      mechanism by which we get them onto the intended target
	      environment.  This can be as simple as copying a file, but
	      complexity comes from the wide range of possible installation
	      media (such as a network install), and the scalability of the
	      process (if a program must be installed on a thousand systems, we
	      do not want to visit each system and perform some manual steps to
	      install the program on that system; that is, the complexity for
	      the system administrator should be constant, not linear).
	    </para>
	  </listitem>
	</itemizedlist>
      </para>
    </sect2>

  </sect1>


  <!--######################################################################-->

  <sect1>
    <title>What Nix provides</title>

    <para>
      Here is a summary of Nix's main features:
    </para>

    <itemizedlist>

      <listitem>
	<para>
	  <emphasis>Reliable dependencies.</emphasis>  Builds of file system
	  objects depend on other file system object, such as source files,
	  tools, and so on.  We would like to ensure that a build does not
	  refer to any objects that have not been declared as inputs for that
	  build.  This is important for several reasons.  First, if any of the
	  inputs change, we need to rebuild the things that depend on them to
	  maintain consistency between sources and derivates.  Second, when we
	  <emphasis>deploy</emphasis> file system objects (that is, copy them
	  to a different system), we want to be certain that we copy everything
	  that we need.
	</para>

	<para>
	  Nix ensures this by building and storing file system objects in paths
	  that are infeasible to predict in advance.  For example, the
	  artifacts of a package <literal>X</literal> might be stored in
	  <filename>/nix/store/d58a0606ed616820de291d594602665d-X</filename>,
	  rather than in, say, <filename>/usr/lib</filename>.  The path
	  component <filename>d58a...</filename> is actually a cryptographic
	  hash of all the inputs (i.e., sources, requisites, and build flags)
	  used in building <literal>X</literal>, and as such is very fragile:
	  any change to the inputs will change the hash.  Therefore it is not
	  sensible to <emphasis>hard-code</emphasis> such a path into the build
	  scripts of a package <literal>Y</literal> that uses
	  <literal>X</literal> (as does happen with <quote>fixed</quote> paths
	  such as <filename>/usr/lib</filename>).  Rather, the build script of
	  package <literal>Y</literal> is parameterised with the actual
	  location of <literal>X</literal>, which is supplied by the Nix
	  system.
	</para>
      </listitem>

      <listitem>
	<para>
	  <emphasis>Support for variability.</emphasis>  
	</para>
	
	<para>
	  As stated above, the path name of a file system object contain a
	  cryptographic hash of all inputs involved in building it.  A change to
	  any of the inputs will cause the hash to change--and by extension,
	  the path name.  These inputs include both sources (variation in time)
	  and configuration options (variation in space).  Therefore variants
	  of the same package don't clash---they can co-exist peacefully within
	  the same file system.  So thanks to Nix's mechanism for reliably
	  dealing with dependencies, we obtain management of variants for free
	  (or, to quote Simon Peyton-Jone, it's not free, but it has already
	  been paid for).
	</para>

      </listitem>

      <listitem>
	<para>
	  <emphasis>Transparent source/binary deployment.</emphasis>
	</para>
      </listitem>

      <listitem>
	<para>
	  <emphasis>Easy configuration duplication.</emphasis>
	</para>
      </listitem>

      <listitem>
	<para>
	  <emphasis>Automatic storage management.</emphasis>
	</para>
      </listitem>

      <listitem>
	<para>
	  <emphasis>Atomic upgrades and rollbacks.</emphasis>
	</para>
      </listitem>

      <listitem>
	<para>
	  <emphasis>Support for many simultaneous configurations.</emphasis>
	</para>
      </listitem>

      <listitem>
	<para>
	  <emphasis>Portability.</emphasis>  Nix is quite portable.  Contrary
	  to build systems like those in, e.g., Vesta and ClearCase [sic?], it
	  does not rely on operating system extensions.
	</para>
      </listitem>

    </itemizedlist>

    <para>
      Here is what Nix doesn't yet provide, but will:
    </para>

    <itemizedlist>

      <listitem>
	<para>
	  <emphasis>Build management.</emphasis>  In principle it is already
	  possible to do build management using Fix (by writing builders that
	  perform appropriate build steps), but the Fix language is not yet
	  powerful enough to make this pleasant.  The <ulink
	    url='http://www.cs.uu.nl/~eelco/maak/'>Maak build manager</ulink>
	  should be retargeted to produce Nix expressions, or alternatively,
	  extend Fix with Maak's semantics and concrete syntax (since Fix needs
	  a concrete syntax anyway).  Another interesting idea is to write a
	  <command>make</command> implementation that uses Nix as a back-end to
	  support <ulink
	    url='http://www.research.att.com/~bs/bs_faq.html#legacy'>legacy</ulink> 
	  build files.
	</para>
      </listitem>

    </itemizedlist>

  </sect1>


  <!--######################################################################-->

  <sect1>
    <title>The Nix system</title>

    <para>
      ...
    </para>

    <para>
      Existing tools in this field generally both a underlying model (such as
      the derivation graph of build tools, or the versioning scheme that
      determines when two packages are <quote>compatible</quote> in a package
      management system) and a formalism that allows ...
    </para>

    <para>
      Following the principle of separation of mechanism and policy, the Nix
      system separates the <emphasis>low-level aspect</emphasis> of file system
      object management form the <emphasis>high-level aspect</emphasis> of the
      ...
    </para>

  </sect1>

</chapter>

<!--
local variables:
sgml-parent-document: ("book.xml" "chapter")
end:
-->