nix-super/doc/manual/release-notes.xml

<article xmlns="http://docbook.org/ns/docbook"
         xmlns:xlink="http://www.w3.org/1999/xlink"
         xml:id="sec-relnotes">

<title>Nix Release Notes</title>



<!--==================================================================-->

<section xml:id="ssec-relnotes-0.12"><title>Release 0.12 (TBA)</title>

<itemizedlist>

  <listitem>
    <para>Nix no longer uses Berkeley DB to store Nix store metadata.
    The principal advantages of the new storage scheme are: it works
    properly over decent implementations of NFS (allowing Nix stores
    to be shared between multiple machines); no recovery is needed
    when a Nix process crashes; no write access is needed for
    read-only operations; no more running out of Berkeley DB locks on
    certain operations.</para>

    <para>You still need to compile Nix with Berkeley DB support if
    you want Nix to automatically convert your old Nix store to the
    new schema.  If you don’t need this, you can build Nix with the
    <filename>configure</filename> option
    <option>--disable-old-db-compat</option>.</para>

    <para>After the automatic conversion to the new schema, you can
    delete the old Berkeley DB files:

    <screen>
$ cd /nix/var/nix/db
$ rm __db* log.* derivers references referrers reserved validpaths DB_CONFIG</screen>

    The new metadata is stored in the directories
    <filename>/nix/var/nix/db/info</filename> and
    <filename>/nix/var/nix/db/referrer</filename>.  Though the
    metadata is stored in human-readable plain-text files, they are
    not intended to be human-editable, as Nix is rather strict about
    the format.</para>

    <para>The new storage schema may or may not require less disk
    space than the Berkeley DB environment, mostly depending on the
    cluster size of your file system.  With 1 KiB clusters (which
    seems to be the <literal>ext3</literal> default nowadays) it
    usually takes up much less space.</para>
  </listitem>

  <listitem><para>There is a new substituter that copies paths
  directly from other (remote) Nix stores mounted somewhere in the
  filesystem.  For instance, you can speed up an installation by
  mounting some remote Nix store that already has the packages in
  question via NFS or <literal>sshfs</literal>.  The environment
  variable <envar>NIX_OTHER_STORES</envar> specifies the locations of
  the remote Nix directories,
  e.g. <literal>/mnt/remote-fs/nix</literal>.</para></listitem>

  <listitem><para>New <command>nix-store</command> operations
  <option>--dump-db</option> and <option>--load-db</option> to dump
  and reload the Nix database.</para></listitem>

  <listitem><para>The garbage collector has a number of new options to
  allow only some of the garbage to be deleted.  The option
  <option>--max-freed <replaceable>N</replaceable></option> tells the
  collector to stop after at least <replaceable>N</replaceable> bytes
  have been deleted.  The option <option>--max-links
  <replaceable>N</replaceable></option> tells it to stop after the
  link count on <filename>/nix/store</filename> has dropped below
  <replaceable>N</replaceable>.  This is useful on very large Nix
  stores on filesystems with a 32000 subdirectories limit (like
  <literal>ext3</literal>).  The option <option>--use-atime</option>
  causes store paths to be deleted in order of ascending last access
  time.  This allows non-recently used stuff to be deleted.  The
  option <option>--max-atime <replaceable>time</replaceable></option>
  specifies an upper limit to the last accessed time of paths that may
  be deleted.  For instance,

    <screen>
    $ nix-store --gc -v --max-atime $(date +%s -d "2 months ago")</screen>

  deletes everything that hasn’t been accessed in two months.</para></listitem>

  <listitem><para><command>nix-env</command> now uses optimistic
  profile locking when performing an operation like installing or
  upgrading, instead of setting an exclusive lock on the profile.
  This allows multiple <command>nix-env -i / -u / -e</command>
  operations on the same profile in parallel.  If a
  <command>nix-env</command> operation sees at the end that the profile
  was changed in the meantime by another process, it will just
  restart.  This is generally cheap because the build results are
  still in the Nix store.</para></listitem>

  <listitem><para>The option <option>--dry-run</option> is now
  supported by <command>nix-store -r</command> and
  <command>nix-build</command>.</para></listitem>

  <listitem><para>The information previously shown by
  <option>--dry-run</option> (i.e., which derivations will be built
  and which paths will be substituted) is now always shown by
  <command>nix-env</command>, <command>nix-store -r</command> and
  <command>nix-build</command>.  The total download size of
  substitutable paths is now also shown.  For instance, a build will
  show something like

    <screen>
the following derivations will be built:
  /nix/store/129sbxnk5n466zg6r1qmq1xjv9zymyy7-activate-configuration.sh.drv
  /nix/store/7mzy971rdm8l566ch8hgxaf89x7lr7ik-upstart-jobs.drv
  ...
the following paths will be downloaded/copied (30.02 MiB):
  /nix/store/4m8pvgy2dcjgppf5b4cj5l6wyshjhalj-samba-3.2.4
  /nix/store/7h1kwcj29ip8vk26rhmx6bfjraxp0g4l-libunwind-0.98.6
  ...</screen>      
  
  </para></listitem>

  <listitem><para>Language features:

    <itemizedlist>

      <listitem><para>@-patterns as in Haskell.  For instance, in a
      function definition

      <programlisting>f = args @ {x, y, z}: <replaceable>...</replaceable>;</programlisting>

      <varname>args</varname> refers to the argument as a whole, which
      is further pattern-matched against the attribute set pattern
      <literal>{x, y, z}</literal>.</para></listitem>

      <listitem><para>“<literal>...</literal>” (ellipsis) patterns.
      An attribute set pattern can now say <literal>...</literal>  at
      the end of the attribute name list to specify that the function
      takes <emphasis>at least</emphasis> the listed attributes, while
      ignoring additional attributes.  For instance,

      <programlisting>{stdenv, fetchurl, fuse, ...}: <replaceable>...</replaceable></programlisting>

      defines a function that accepts any attribute set that includes
      at least the three listed attributes.</para></listitem>

      <listitem><para>New primops:
      <varname>builtins.parseDrvName</varname> (split a package name
      string like <literal>"nix-0.12pre12876"</literal> into its name
      and version components, e.g. <literal>"nix"</literal> and
      <literal>"0.12pre12876"</literal>),
      <varname>builtins.compareVersions</varname> (compare two version
      strings using the same algorithm that <command>nix-env</command>
      uses), <varname>builtins.length</varname> (efficiently compute
      the length of a list), <varname>builtins.mul</varname> (integer
      multiplication), <varname>builtins.div</varname> (integer
      division).
      <!-- <varname>builtins.genericClosure</varname> -->
      </para></listitem>
      
    </itemizedlist>

  </para></listitem>

  <listitem><para><command>nix-prefetch-url</command> now supports
  <literal>mirror://</literal> URLs, provided that the environment
  variable <envar>NIXPKGS_ALL</envar> points at a Nixpkgs
  tree.</para></listitem>

  <listitem><para>Removed the commands
  <command>nix-pack-closure</command> and
  <command>nix-unpack-closure</command>.   You can do almost the same
  thing but much more efficiently by doing <literal>nix-store --export
  $(nix-store -qR <replaceable>paths</replaceable>) > closure</literal> and
  <literal>nix-store --import &lt;
  closure</literal>.</para></listitem>

  <listitem><para>Lots of bug fixes, including a big performance bug in
  the handling of <literal>with</literal>-expressions.</para></listitem>

</itemizedlist>

</section>


<!--==================================================================-->

<section xml:id="ssec-relnotes-0.11"><title>Release 0.11 (December 31,
2007)</title>

<para>Nix 0.11 has many improvements over the previous stable release.
The most important improvement is secure multi-user support.  It also
features many usability enhancements and language extensions, many of
them prompted by NixOS, the purely functional Linux distribution based
on Nix.  Here is an (incomplete) list:</para>


<itemizedlist>


  <listitem><para>Secure multi-user support.  A single Nix store can
  now be shared between multiple (possible untrusted) users.  This is
  an important feature for NixOS, where it allows non-root users to
  install software.  The old setuid method for sharing a store between
  multiple users has been removed.  Details for setting up a
  multi-user store can be found in the manual.</para></listitem>


  <listitem><para>The new command <command>nix-copy-closure</command>
  gives you an easy and efficient way to exchange software between
  machines.  It copies the missing parts of the closure of a set of
  store path to or from a remote machine via
  <command>ssh</command>.</para></listitem>


  <listitem><para>A new kind of string literal: strings between double
  single-quotes (<literal>''</literal>) have indentation
  “intelligently” removed.  This allows large strings (such as shell
  scripts or configuration file fragments in NixOS) to cleanly follow
  the indentation of the surrounding expression.  It also requires
  much less escaping, since <literal>''</literal> is less common in
  most languages than <literal>"</literal>.</para></listitem>


  <listitem><para><command>nix-env</command> <option>--set</option>
  modifies the current generation of a profile so that it contains
  exactly the specified derivation, and nothing else.  For example,
  <literal>nix-env -p /nix/var/nix/profiles/browser --set
  firefox</literal> lets the profile named
  <filename>browser</filename> contain just Firefox.</para></listitem>
  

  <listitem><para><command>nix-env</command> now maintains
  meta-information about installed packages in profiles.  The
  meta-information is the contents of the <varname>meta</varname>
  attribute of derivations, such as <varname>description</varname> or
  <varname>homepage</varname>.  The command <literal>nix-env -q --xml
  --meta</literal> shows all meta-information.</para></listitem>

  
  <listitem><para><command>nix-env</command> now uses the
  <varname>meta.priority</varname> attribute of derivations to resolve
  filename collisions between packages.  Lower priority values denote
  a higher priority.  For instance, the GCC wrapper package and the
  Binutils package in Nixpkgs both have a file
  <filename>bin/ld</filename>, so previously if you tried to install
  both you would get a collision.  Now, on the other hand, the GCC
  wrapper declares a higher priority than Binutils, so the former’s
  <filename>bin/ld</filename> is symlinked in the user
  environment.</para></listitem>


  <listitem><para><command>nix-env -i / -u</command>: instead of
  breaking package ties by version, break them by priority and version
  number.  That is, if there are multiple packages with the same name,
  then pick the package with the highest priority, and only use the
  version if there are multiple packages with the same
  priority.</para>

  <para>This makes it possible to mark specific versions/variant in
  Nixpkgs more or less desirable than others.  A typical example would
  be a beta version of some package (e.g.,
  <literal>gcc-4.2.0rc1</literal>) which should not be installed even
  though it is the highest version, except when it is explicitly
  selected (e.g., <literal>nix-env -i
  gcc-4.2.0rc1</literal>).</para></listitem>


  <listitem><para><command>nix-env --set-flag</command> allows meta
  attributes of installed packages to be modified.  There are several
  attributes that can be usefully modified, because they affect the
  behaviour of <command>nix-env</command> or the user environment
  build script:

    <itemizedlist>

      <listitem><para><varname>meta.priority</varname> can be changed
      to resolve filename clashes (see above).</para></listitem>

      <listitem><para><varname>meta.keep</varname> can be set to
      <literal>true</literal> to prevent the package from being
      upgraded or replaced.  Useful if you want to hang on to an older
      version of a package.</para></listitem>

      <listitem><para><varname>meta.active</varname> can be set to
      <literal>false</literal> to “disable” the package.  That is, no
      symlinks will be generated to the files of the package, but it
      remains part of the profile (so it won’t be garbage-collected).
      Set it back to <literal>true</literal> to re-enable the
      package.</para></listitem>

    </itemizedlist>

  </para></listitem>

  
  <listitem><para><command>nix-env -q</command> now has a flag
  <option>--prebuilt-only</option> (<option>-b</option>) that causes
  <command>nix-env</command> to show only those derivations whose
  output is already in the Nix store or that can be substituted (i.e.,
  downloaded from somewhere).  In other words, it shows the packages
  that can be installed “quickly”, i.e., don’t need to be built from
  source.  The <option>-b</option> flag is also available in
  <command>nix-env -i</command> and <command>nix-env -u</command> to
  filter out derivations for which no pre-built binary is
  available.</para></listitem>
  

  <listitem><para>The new option <option>--argstr</option> (in
  <command>nix-env</command>, <command>nix-instantiate</command> and
  <command>nix-build</command>) is like <option>--arg</option>, except
  that the value is a string.  For example, <literal>--argstr system
  i686-linux</literal> is equivalent to <literal>--arg system
  \"i686-linux\"</literal> (note that <option>--argstr</option>
  prevents annoying quoting around shell arguments).</para></listitem>


  <listitem><para><command>nix-store</command> has a new operation
  <option>--read-log</option> (<option>-l</option>)
  <parameter>paths</parameter> that shows the build log of the given
  paths.</para></listitem>

  
  <!--
  <listitem><para>TODO: semantic cleanups of string concatenation
  etc. (mostly in r6740).</para></listitem>
  -->


  <listitem><para>Nix now uses Berkeley DB 4.5.  The database is
  upgraded automatically, but you should be careful not to use old
  versions of Nix that still use Berkeley DB 4.4.</para></listitem>


  <!-- foo
  <listitem><para>TODO: option <option>- -reregister</option> in
  <command>nix-store - -register-validity</command>.</para></listitem>
  -->


  <listitem><para>The option <option>--max-silent-time</option>
  (corresponding to the configuration setting
  <literal>build-max-silent-time</literal>) allows you to set a
  timeout on builds — if a build produces no output on
  <literal>stdout</literal> or <literal>stderr</literal> for the given
  number of seconds, it is terminated.  This is useful for recovering
  automatically from builds that are stuck in an infinite
  loop.</para></listitem>


  <listitem><para><command>nix-channel</command>: each subscribed
  channel is its own attribute in the top-level expression generated
  for the channel.  This allows disambiguation (e.g. <literal>nix-env
  -i -A nixpkgs_unstable.firefox</literal>).</para></listitem>


  <listitem><para>The substitutes table has been removed from the
  database.  This makes operations such as <command>nix-pull</command>
  and <command>nix-channel --update</command> much, much
  faster.</para></listitem>


  <listitem><para><command>nix-pull</command> now supports
  bzip2-compressed manifests.  This speeds up
  channels.</para></listitem>


  <listitem><para><command>nix-prefetch-url</command> now has a
  limited form of caching.  This is used by
  <command>nix-channel</command> to prevent unnecessary downloads when
  the channel hasn’t changed.</para></listitem>


  <listitem><para><command>nix-prefetch-url</command> now by default
  computes the SHA-256 hash of the file instead of the MD5 hash.  In
  calls to <function>fetchurl</function> you should pass the
  <literal>sha256</literal> attribute instead of
  <literal>md5</literal>.  You can pass either a hexadecimal or a
  base-32 encoding of the hash.</para></listitem>

  
  <listitem><para>Nix can now perform builds in an automatically
  generated “chroot”.  This prevents a builder from accessing stuff
  outside of the Nix store, and thus helps ensure purity.  This is an
  experimental feature.</para></listitem>
  

  <listitem><para>The new command <command>nix-store
  --optimise</command> reduces Nix store disk space usage by finding
  identical files in the store and hard-linking them to each other.
  It typically reduces the size of the store by something like
  25-35%.</para></listitem>

  
  <listitem><para><filename>~/.nix-defexpr</filename> can now be a
  directory, in which case the Nix expressions in that directory are
  combined into an attribute set, with the file names used as the
  names of the attributes.  The command <command>nix-env
  --import</command> (which set the
  <filename>~/.nix-defexpr</filename> symlink) is
  removed.</para></listitem>


  <listitem><para>Derivations can specify the new special attribute
  <varname>allowedReferences</varname> to enforce that the references
  in the output of a derivation are a subset of a declared set of
  paths.  For example, if <varname>allowedReferences</varname> is an
  empty list, then the output must not have any references.  This is
  used in NixOS to check that generated files such as initial ramdisks
  for booting Linux don’t have any dependencies.</para></listitem>


  <listitem><para>The new attribute
  <varname>exportReferencesGraph</varname> allows builders access to
  the references graph of their inputs.  This is used in NixOS for
  tasks such as generating ISO-9660 images that contain a Nix store
  populated with the closure of certain paths.</para></listitem>


  <listitem><para>Fixed-output derivations (like
  <function>fetchurl</function>) can define the attribute
  <varname>impureEnvVars</varname> to allow external environment
  variables to be passed to builders.  This is used in Nixpkgs to
  support proxy configuration, among other things.</para></listitem>


  <listitem><para>Several new built-in functions:
  <function>builtins.attrNames</function>,
  <function>builtins.filterSource</function>,
  <function>builtins.isAttrs</function>,
  <function>builtins.isFunction</function>,
  <function>builtins.listToAttrs</function>,
  <function>builtins.stringLength</function>,
  <function>builtins.sub</function>,
  <function>builtins.substring</function>,
  <function>throw</function>,
  <function>builtins.trace</function>,
  <function>builtins.readFile</function>.</para></listitem>


</itemizedlist>

</section>



<!--==================================================================-->

<section><title>Release 0.10.1 (October 11, 2006)</title>

<para>This release fixes two somewhat obscure bugs that occur when
evaluating Nix expressions that are stored inside the Nix store
(<literal>NIX-67</literal>).  These do not affect most users.</para>

</section>



<!--==================================================================-->

<section><title>Release 0.10 (October 6, 2006)</title>

<note><para>This version of Nix uses Berkeley DB 4.4 instead of 4.3.
The database is upgraded automatically, but you should be careful not
to use old versions of Nix that still use Berkeley DB 4.3.  In
particular, if you use a Nix installed through Nix, you should run

<screen>
$ nix-store --clear-substitutes</screen>

first.</para></note>

<warning><para>Also, the database schema has changed slighted to fix a
performance issue (see below).  When you run any Nix 0.10 command for
the first time, the database will be upgraded automatically.  This is
irreversible.</para></warning>

<itemizedlist>

  
  <!-- Usability / features -->

  
  <listitem><para><command>nix-env</command> usability improvements:

    <itemizedlist>

      <listitem><para>An option <option>--compare-versions</option>
      (or <option>-c</option>) has been added to <command>nix-env
      --query</command> to allow you to compare installed versions of
      packages to available versions, or vice versa.  An easy way to
      see if you are up to date with what’s in your subscribed
      channels is <literal>nix-env -qc \*</literal>.</para></listitem>

      <listitem><para><literal>nix-env --query</literal> now takes as
      arguments a list of package names about which to show
      information, just like <option>--install</option>, etc.: for
      example, <literal>nix-env -q gcc</literal>.  Note that to show
      all derivations, you need to specify
      <literal>\*</literal>.</para></listitem>

      <listitem><para><literal>nix-env -i
      <replaceable>pkgname</replaceable></literal> will now install
      the highest available version of
      <replaceable>pkgname</replaceable>, rather than installing all
      available versions (which would probably give collisions)
      (<literal>NIX-31</literal>).</para></listitem>

      <listitem><para><literal>nix-env (-i|-u) --dry-run</literal> now
      shows exactly which missing paths will be built or
      substituted.</para></listitem>

      <listitem><para><literal>nix-env -qa --description</literal>
      shows human-readable descriptions of packages, provided that
      they have a <literal>meta.description</literal> attribute (which
      most packages in Nixpkgs don’t have yet).</para></listitem>

    </itemizedlist>
  
  </para></listitem>
  

  <listitem><para>New language features:

    <itemizedlist>

      <listitem><para>Reference scanning (which happens after each
      build) is much faster and takes a constant amount of
      memory.</para></listitem>

      <listitem><para>String interpolation.  Expressions like

<programlisting>
"--with-freetype2-library=" + freetype + "/lib"</programlisting>

      can now be written as

<programlisting>
"--with-freetype2-library=${freetype}/lib"</programlisting>

      You can write arbitrary expressions within
      <literal>${<replaceable>...</replaceable>}</literal>, not just
      identifiers.</para></listitem>
      
      <listitem><para>Multi-line string literals.</para></listitem>

      <listitem><para>String concatenations can now involve
      derivations, as in the example <code>"--with-freetype2-library="
      + freetype + "/lib"</code>.  This was not previously possible
      because we need to register that a derivation that uses such a
      string is dependent on <literal>freetype</literal>.  The
      evaluator now properly propagates this information.
      Consequently, the subpath operator (<literal>~</literal>) has
      been deprecated.</para></listitem>

      <listitem><para>Default values of function arguments can now
      refer to other function arguments; that is, all arguments are in
      scope in the default values
      (<literal>NIX-45</literal>).</para></listitem>

      <!--
      <listitem><para>TODO: domain checks (r5895).</para></listitem>
      -->

      <listitem><para>Lots of new built-in primitives, such as
      functions for list manipulation and integer arithmetic.  See the
      manual for a complete list.  All primops are now available in
      the set <varname>builtins</varname>, allowing one to test for
      the availability of primop in a backwards-compatible
      way.</para></listitem>

      <listitem><para>Real let-expressions: <literal>let x = ...;
      ... z = ...; in ...</literal>.</para></listitem>

    </itemizedlist>

  </para></listitem>


  <listitem><para>New commands <command>nix-pack-closure</command> and
  <command>nix-unpack-closure</command> than can be used to easily
  transfer a store path with all its dependencies to another machine.
  Very convenient whenever you have some package on your machine and
  you want to copy it somewhere else.</para></listitem>

  
  <listitem><para>XML support:

    <itemizedlist>

      <listitem><para><literal>nix-env -q --xml</literal> prints the
      installed or available packages in an XML representation for
      easy processing by other tools.</para></listitem>

      <listitem><para><literal>nix-instantiate --eval-only
      --xml</literal> prints an XML representation of the resulting
      term.  (The new flag <option>--strict</option> forces ‘deep’
      evaluation of the result, i.e., list elements and attributes are
      evaluated recursively.)</para></listitem>

      <listitem><para>In Nix expressions, the primop
      <function>builtins.toXML</function> converts a term to an XML
      representation.  This is primarily useful for passing structured
      information to builders.</para></listitem>

    </itemizedlist>

  </para></listitem>
  

  <listitem><para>You can now unambigously specify which derivation to
  build or install in <command>nix-env</command>,
  <command>nix-instantiate</command> and <command>nix-build</command>
  using the <option>--attr</option> / <option>-A</option> flags, which
  takes an attribute name as argument.  (Unlike symbolic package names
  such as <literal>subversion-1.4.0</literal>, attribute names in an
  attribute set are unique.)  For instance, a quick way to perform a
  test build of a package in Nixpkgs is <literal>nix-build
  pkgs/top-level/all-packages.nix -A
  <replaceable>foo</replaceable></literal>.  <literal>nix-env -q
  --attr</literal> shows the attribute names corresponding to each
  derivation.</para></listitem>
  

  <listitem><para>If the top-level Nix expression used by
  <command>nix-env</command>, <command>nix-instantiate</command> or
  <command>nix-build</command> evaluates to a function whose arguments
  all have default values, the function will be called automatically.
  Also, the new command-line switch <option>--arg
  <replaceable>name</replaceable>
  <replaceable>value</replaceable></option> can be used to specify
  function arguments on the command line.</para></listitem>

  
  <listitem><para><literal>nix-install-package --url
  <replaceable>URL</replaceable></literal> allows a package to be
  installed directly from the given URL.</para></listitem>


  <listitem><para>Nix now works behind an HTTP proxy server; just set
  the standard environment variables <envar>http_proxy</envar>,
  <envar>https_proxy</envar>, <envar>ftp_proxy</envar> or
  <envar>all_proxy</envar> appropriately.  Functions such as
  <function>fetchurl</function> in Nixpkgs also respect these
  variables.</para></listitem>


  <listitem><para><literal>nix-build -o
  <replaceable>symlink</replaceable></literal> allows the symlink to
  the build result to be named something other than
  <literal>result</literal>.</para></listitem>


  <!-- Stability / performance / etc. -->


  <listitem><para>Platform support:

    <itemizedlist>

      <listitem><para>Support for 64-bit platforms, provided a <link
      xlink:href="http://bugzilla.sen.cwi.nl:8080/show_bug.cgi?id=606">suitably
      patched ATerm library</link> is used.  Also, files larger than 2
      GiB are now supported.</para></listitem>

      <listitem><para>Added support for Cygwin (Windows,
      <literal>i686-cygwin</literal>), Mac OS X on Intel
      (<literal>i686-darwin</literal>) and Linux on PowerPC
      (<literal>powerpc-linux</literal>).</para></listitem>

      <listitem><para>Users of SMP and multicore machines will
      appreciate that the number of builds to be performed in parallel
      can now be specified in the configuration file in the
      <literal>build-max-jobs</literal> setting.</para></listitem>

    </itemizedlist>

  </para></listitem>

  
  <listitem><para>Garbage collector improvements:

    <itemizedlist>

      <listitem><para>Open files (such as running programs) are now
      used as roots of the garbage collector.  This prevents programs
      that have been uninstalled from being garbage collected while
      they are still running.  The script that detects these
      additional runtime roots
      (<filename>find-runtime-roots.pl</filename>) is inherently
      system-specific, but it should work on Linux and on all
      platforms that have the <command>lsof</command>
      utility.</para></listitem>

      <listitem><para><literal>nix-store --gc</literal>
      (a.k.a. <command>nix-collect-garbage</command>) prints out the
      number of bytes freed on standard output.  <literal>nix-store
      --gc --print-dead</literal> shows how many bytes would be freed
      by an actual garbage collection.</para></listitem>

      <listitem><para><literal>nix-collect-garbage -d</literal>
      removes all old generations of <emphasis>all</emphasis> profiles
      before calling the actual garbage collector (<literal>nix-store
      --gc</literal>).  This is an easy way to get rid of all old
      packages in the Nix store.</para></listitem>

      <listitem><para><command>nix-store</command> now has an
      operation <option>--delete</option> to delete specific paths
      from the Nix store.  It won’t delete reachable (non-garbage)
      paths unless <option>--ignore-liveness</option> is
      specified.</para></listitem>
      
    </itemizedlist>

  </para></listitem>

  
  <listitem><para>Berkeley DB 4.4’s process registry feature is used
  to recover from crashed Nix processes.</para></listitem>

  <!--  <listitem><para>TODO: shared stores.</para></listitem> -->

  <listitem><para>A performance issue has been fixed with the
  <literal>referer</literal> table, which stores the inverse of the
  <literal>references</literal> table (i.e., it tells you what store
  paths refer to a given path).  Maintaining this table could take a
  quadratic amount of time, as well as a quadratic amount of Berkeley
  DB log file space (in particular when running the garbage collector)
  (<literal>NIX-23</literal>).</para></listitem>

  <listitem><para>Nix now catches the <literal>TERM</literal> and
  <literal>HUP</literal> signals in addition to the
  <literal>INT</literal> signal.  So you can now do a <literal>killall
  nix-store</literal> without triggering a database
  recovery.</para></listitem>

  <listitem><para><command>bsdiff</command> updated to version
  4.3.</para></listitem>

  <listitem><para>Substantial performance improvements in expression
  evaluation and <literal>nix-env -qa</literal>, all thanks to <link
  xlink:href="http://valgrind.org/">Valgrind</link>.  Memory use has
  been reduced by a factor 8 or so.  Big speedup by memoisation of
  path hashing.</para></listitem>

  <listitem><para>Lots of bug fixes, notably:

    <itemizedlist>

      <listitem><para>Make sure that the garbage collector can run
      succesfully when the disk is full
      (<literal>NIX-18</literal>).</para></listitem>

      <listitem><para><command>nix-env</command> now locks the profile
      to prevent races between concurrent <command>nix-env</command>
      operations on the same profile
      (<literal>NIX-7</literal>).</para></listitem>

      <listitem><para>Removed misleading messages from
      <literal>nix-env -i</literal> (e.g., <literal>installing
      `foo'</literal> followed by <literal>uninstalling
      `foo'</literal>) (<literal>NIX-17</literal>).</para></listitem>

    </itemizedlist>

  </para></listitem>

  <listitem><para>Nix source distributions are a lot smaller now since
  we no longer include a full copy of the Berkeley DB source
  distribution (but only the bits we need).</para></listitem>

  <listitem><para>Header files are now installed so that external
  programs can use the Nix libraries.</para></listitem>

</itemizedlist>

</section>



<!--==================================================================-->

<section><title>Release 0.9.2 (September 21, 2005)</title>

<para>This bug fix release fixes two problems on Mac OS X:

<itemizedlist>

  <listitem><para>If Nix was linked against statically linked versions
  of the ATerm or Berkeley DB library, there would be dynamic link
  errors at runtime.</para></listitem>

  <listitem><para><command>nix-pull</command> and
  <command>nix-push</command> intermittently failed due to race
  conditions involving pipes and child processes with error messages
  such as <literal>open2: open(GLOB(0x180b2e4), >&amp;=9) failed: Bad
  file descriptor at /nix/bin/nix-pull line 77</literal> (issue
  <literal>NIX-14</literal>).</para></listitem>

</itemizedlist>

</para>

</section>



<!--==================================================================-->

<section><title>Release 0.9.1 (September 20, 2005)</title>

<para>This bug fix release addresses a problem with the ATerm library
when the <option>--with-aterm</option> flag in
<command>configure</command> was <emphasis>not</emphasis> used.</para>

</section>



<!--==================================================================-->

<section><title>Release 0.9 (September 16, 2005)</title>

<para>NOTE: this version of Nix uses Berkeley DB 4.3 instead of 4.2.
The database is upgraded automatically, but you should be careful not
to use old versions of Nix that still use Berkeley DB 4.2.  In
particular, if you use a Nix installed through Nix, you should run

<screen>
$ nix-store --clear-substitutes</screen>

first.</para>


<itemizedlist>

  <listitem><para>Unpacking of patch sequences is much faster now
  since we no longer do redundant unpacking and repacking of
  intermediate paths.</para></listitem>

  <listitem><para>Nix now uses Berkeley DB 4.3.</para></listitem>

  <listitem><para>The <function>derivation</function> primitive is
  lazier.  Attributes of dependent derivations can mutually refer to
  each other (as long as there are no data dependencies on the
  <varname>outPath</varname> and <varname>drvPath</varname> attributes
  computed by <function>derivation</function>).</para>

  <para>For example, the expression <literal>derivation
  attrs</literal> now evaluates to (essentially)

  <programlisting>
attrs // {
  type = "derivation";
  outPath = derivation! attrs;
  drvPath = derivation! attrs;
}</programlisting>

  where <function>derivation!</function> is a primop that does the
  actual derivation instantiation (i.e., it does what
  <function>derivation</function> used to do).  The advantage is that
  it allows commands such as <command>nix-env -qa</command> and
  <command>nix-env -i</command> to be much faster since they no longer
  need to instantiate all derivations, just the
  <varname>name</varname> attribute.</para>

  <para>Also, it allows derivations to cyclically reference each
  other, for example,

  <programlisting>
webServer = derivation {
  ...
  hostName = "svn.cs.uu.nl";
  services = [svnService];
};
&#x20;
svnService = derivation {
  ...
  hostName = webServer.hostName;
};</programlisting>

  Previously, this would yield a black hole (infinite recursion).</para>
  
  </listitem>

  <listitem><para><command>nix-build</command> now defaults to using
  <filename>./default.nix</filename> if no Nix expression is
  specified.</para></listitem>

  <listitem><para><command>nix-instantiate</command>, when applied to
  a Nix expression that evaluates to a function, will call the
  function automatically if all its arguments have
  defaults.</para></listitem>

  <listitem><para>Nix now uses libtool to build dynamic libraries.
  This reduces the size of executables.</para></listitem>

  <listitem><para>A new list concatenation operator
  <literal>++</literal>.  For example, <literal>[1 2 3] ++ [4 5
  6]</literal> evaluates to <literal>[1 2 3 4 5
  6]</literal>.</para></listitem>

  <listitem><para>Some currently undocumented primops to support
  low-level build management using Nix (i.e., using Nix as a Make
  replacement).  See the commit messages for <literal>r3578</literal>
  and <literal>r3580</literal>.</para></listitem>

  <listitem><para>Various bug fixes and performance
  improvements.</para></listitem>

</itemizedlist>

</section>



<!--==================================================================-->

<section><title>Release 0.8.1 (April 13, 2005)</title>

<para>This is a bug fix release.</para>

<itemizedlist>

  <listitem><para>Patch downloading was broken.</para></listitem>

  <listitem><para>The garbage collector would not delete paths that
  had references from invalid (but substitutable)
  paths.</para></listitem>

</itemizedlist>

</section>



<!--==================================================================-->

<section><title>Release 0.8 (April 11, 2005)</title>

<para>NOTE: the hashing scheme in Nix 0.8 changed (as detailed below).
As a result, <command>nix-pull</command> manifests and channels built
for Nix 0.7 and below will now work anymore.  However, the Nix
expression language has not changed, so you can still build from
source.  Also, existing user environments continue to work.  Nix 0.8
will automatically upgrade the database schema of previous
installations when it is first run.</para>

<para>If you get the error message

<screen>
you have an old-style manifest `/nix/var/nix/manifests/[...]'; please
delete it</screen>

you should delete previously downloaded manifests:

<screen>
$ rm /nix/var/nix/manifests/*</screen>

If <command>nix-channel</command> gives the error message

<screen>
manifest `http://catamaran.labs.cs.uu.nl/dist/nix/channels/[channel]/MANIFEST'
is too old (i.e., for Nix &lt;= 0.7)</screen>

then you should unsubscribe from the offending channel
(<command>nix-channel --remove
<replaceable>URL</replaceable></command>; leave out
<literal>/MANIFEST</literal>), and subscribe to the same URL, with
<literal>channels</literal> replaced by <literal>channels-v3</literal>
(e.g., <link
xlink:href='http://catamaran.labs.cs.uu.nl/dist/nix/channels-v3/nixpkgs-unstable'
/>).</para>

<para>Nix 0.8 has the following improvements:

<itemizedlist>

  <listitem><para>The cryptographic hashes used in store paths are now
  160 bits long, but encoded in base-32 so that they are still only 32
  characters long (e.g.,
  <filename>/nix/store/csw87wag8bqlqk7ipllbwypb14xainap-atk-1.9.0</filename>).
  (This is actually a 160 bit truncation of a SHA-256
  hash.)</para></listitem>

  <listitem><para>Big cleanups and simplifications of the basic store
  semantics.  The notion of “closure store expressions” is gone (and
  so is the notion of “successors”); the file system references of a
  store path are now just stored in the database.</para>

  <para>For instance, given any store path, you can query its closure:

  <screen>
$ nix-store -qR $(which firefox)
... lots of paths ...</screen>

  Also, Nix now remembers for each store path the derivation that
  built it (the “deriver”):

  <screen>
$ nix-store -qR $(which firefox)
/nix/store/4b0jx7vq80l9aqcnkszxhymsf1ffa5jd-firefox-1.0.1.drv</screen>

  So to see the build-time dependencies, you can do

  <screen>
$ nix-store -qR $(nix-store -qd $(which firefox))</screen>

  or, in a nicer format:

  <screen>
$ nix-store -q --tree $(nix-store -qd $(which firefox))</screen>

  </para>

  <para>File system references are also stored in reverse.  For
  instance, you can query all paths that directly or indirectly use a
  certain Glibc:

  <screen>
$ nix-store -q --referrers-closure \
    /nix/store/8lz9yc6zgmc0vlqmn2ipcpkjlmbi51vv-glibc-2.3.4</screen>

  </para>
  
  </listitem>

  <listitem><para>The concept of fixed-output derivations has been
  formalised.  Previously, functions such as
  <function>fetchurl</function> in Nixpkgs used a hack (namely,
  explicitly specifying a store path hash) to prevent changes to, say,
  the URL of the file from propagating upwards through the dependency
  graph, causing rebuilds of everything.  This can now be done cleanly
  by specifying the <varname>outputHash</varname> and
  <varname>outputHashAlgo</varname> attributes.  Nix itself checks
  that the content of the output has the specified hash.  (This is
  important for maintaining certain invariants necessary for future
  work on secure shared stores.)</para></listitem>

  <listitem><para>One-click installation :-) It is now possible to
  install any top-level component in Nixpkgs directly, through the web
  — see, e.g., <link
  xlink:href='http://catamaran.labs.cs.uu.nl/dist/nixpkgs-0.8/' />.
  All you have to do is associate
  <filename>/nix/bin/nix-install-package</filename> with the MIME type
  <literal>application/nix-package</literal> (or the extension
  <filename>.nixpkg</filename>), and clicking on a package link will
  cause it to be installed, with all appropriate dependencies.  If you
  just want to install some specific application, this is easier than
  subscribing to a channel.</para></listitem>

  <listitem><para><command>nix-store -r
  <replaceable>PATHS</replaceable></command> now builds all the
  derivations PATHS in parallel.  Previously it did them sequentially
  (though exploiting possible parallelism between subderivations).
  This is nice for build farms.</para></listitem>

  <listitem><para><command>nix-channel</command> has new operations
  <option>--list</option> and
  <option>--remove</option>.</para></listitem>

  <listitem><para>New ways of installing components into user
  environments:

  <itemizedlist>

    <listitem><para>Copy from another user environment:

    <screen>
$ nix-env -i --from-profile .../other-profile firefox</screen>

    </para></listitem>

    <listitem><para>Install a store derivation directly (bypassing the
    Nix expression language entirely):

    <screen>
$ nix-env -i /nix/store/z58v41v21xd3...-aterm-2.3.1.drv</screen>

    (This is used to implement <command>nix-install-package</command>,
    which is therefore immune to evolution in the Nix expression
    language.)</para></listitem>

    <listitem><para>Install an already built store path directly:

    <screen>
$ nix-env -i /nix/store/hsyj5pbn0d9i...-aterm-2.3.1</screen>

    </para></listitem>

    <listitem><para>Install the result of a Nix expression specified
    as a command-line argument:

    <screen>
$ nix-env -f .../i686-linux.nix -i -E 'x: x.firefoxWrapper'</screen>

    The difference with the normal installation mode is that
    <option>-E</option> does not use the <varname>name</varname>
    attributes of derivations.  Therefore, this can be used to
    disambiguate multiple derivations with the same
    name.</para></listitem>

  </itemizedlist></para></listitem>

  <listitem><para>A hash of the contents of a store path is now stored
  in the database after a succesful build.  This allows you to check
  whether store paths have been tampered with: <command>nix-store
  --verify --check-contents</command>.</para></listitem>

  <listitem>

    <para>Implemented a concurrent garbage collector.  It is now
    always safe to run the garbage collector, even if other Nix
    operations are happening simultaneously.</para>

    <para>However, there can still be GC races if you use
    <command>nix-instantiate</command> and <command>nix-store
    --realise</command> directly to build things.  To prevent races,
    use the <option>--add-root</option> flag of those commands.</para>

  </listitem>

  <listitem><para>The garbage collector now finally deletes paths in
  the right order (i.e., topologically sorted under the “references”
  relation), thus making it safe to interrupt the collector without
  risking a store that violates the closure
  invariant.</para></listitem>

  <listitem><para>Likewise, the substitute mechanism now downloads
  files in the right order, thus preserving the closure invariant at
  all times.</para></listitem>

  <listitem><para>The result of <command>nix-build</command> is now
  registered as a root of the garbage collector.  If the
  <filename>./result</filename> link is deleted, the GC root
  disappears automatically.</para></listitem>

  <listitem>

    <para>The behaviour of the garbage collector can be changed
    globally by setting options in
    <filename>/nix/etc/nix/nix.conf</filename>.

    <itemizedlist>

      <listitem><para><literal>gc-keep-derivations</literal> specifies
      whether deriver links should be followed when searching for live
      paths.</para></listitem>

      <listitem><para><literal>gc-keep-outputs</literal> specifies
      whether outputs of derivations should be followed when searching
      for live paths.</para></listitem>

      <listitem><para><literal>env-keep-derivations</literal>
      specifies whether user environments should store the paths of
      derivations when they are added (thus keeping the derivations
      alive).</para></listitem>

    </itemizedlist>

  </para></listitem>

  <listitem><para>New <command>nix-env</command> query flags
  <option>--drv-path</option> and
  <option>--out-path</option>.</para></listitem>

  <listitem><para><command>fetchurl</command> allows SHA-1 and SHA-256
  in addition to MD5.  Just specify the attribute
  <varname>sha1</varname> or <varname>sha256</varname> instead of
  <varname>md5</varname>.</para></listitem>

  <listitem><para>Manual updates.</para></listitem>

</itemizedlist>

</para>

</section>



<!--==================================================================-->

<section><title>Release 0.7 (January 12, 2005)</title>

<itemizedlist>

  <listitem><para>Binary patching.  When upgrading components using
  pre-built binaries (through nix-pull / nix-channel), Nix can
  automatically download and apply binary patches to already installed
  components instead of full downloads.  Patching is “smart”: if there
  is a <emphasis>sequence</emphasis> of patches to an installed
  component, Nix will use it.  Patches are currently generated
  automatically between Nixpkgs (pre-)releases.</para></listitem>

  <listitem><para>Simplifications to the substitute
  mechanism.</para></listitem>

  <listitem><para>Nix-pull now stores downloaded manifests in
  <filename>/nix/var/nix/manifests</filename>.</para></listitem>

  <listitem><para>Metadata on files in the Nix store is canonicalised
  after builds: the last-modified timestamp is set to 0 (00:00:00
  1/1/1970), the mode is set to 0444 or 0555 (readable and possibly
  executable by all; setuid/setgid bits are dropped), and the group is
  set to the default.  This ensures that the result of a build and an
  installation through a substitute is the same; and that timestamp
  dependencies are revealed.</para></listitem>

</itemizedlist>

</section>



<!--==================================================================-->

<section><title>Release 0.6 (November 14, 2004)</title>

<itemizedlist>

  <listitem>
    <para>Rewrite of the normalisation engine.

    <itemizedlist>

      <listitem><para>Multiple builds can now be performed in parallel
      (option <option>-j</option>).</para></listitem>

      <listitem><para>Distributed builds.  Nix can now call a shell
      script to forward builds to Nix installations on remote
      machines, which may or may not be of the same platform
      type.</para></listitem>

      <listitem><para>Option <option>--fallback</option> allows
      recovery from broken substitutes.</para></listitem>

      <listitem><para>Option <option>--keep-going</option> causes
      building of other (unaffected) derivations to continue if one
      failed.</para></listitem>

    </itemizedlist>

    </para>

  </listitem>

  <listitem><para>Improvements to the garbage collector (i.e., it
  should actually work now).</para></listitem>

  <listitem><para>Setuid Nix installations allow a Nix store to be
  shared among multiple users.</para></listitem>

  <listitem><para>Substitute registration is much faster
  now.</para></listitem>

  <listitem><para>A utility <command>nix-build</command> to build a
  Nix expression and create a symlink to the result int the current
  directory; useful for testing Nix derivations.</para></listitem>

  <listitem><para>Manual updates.</para></listitem>

  <listitem>

    <para><command>nix-env</command> changes:

    <itemizedlist>

      <listitem><para>Derivations for other platforms are filtered out
      (which can be overriden using
      <option>--system-filter</option>).</para></listitem>

      <listitem><para><option>--install</option> by default now
      uninstall previous derivations with the same
      name.</para></listitem>

      <listitem><para><option>--upgrade</option> allows upgrading to a
      specific version.</para></listitem>

      <listitem><para>New operation
      <option>--delete-generations</option> to remove profile
      generations (necessary for effective garbage
      collection).</para></listitem>

      <listitem><para>Nicer output (sorted,
      columnised).</para></listitem>

    </itemizedlist>
    
    </para>

  </listitem>

  <listitem><para>More sensible verbosity levels all around (builder
  output is now shown always, unless <option>-Q</option> is
  given).</para></listitem>

  <listitem>

    <para>Nix expression language changes:

    <itemizedlist>
      
      <listitem><para>New language construct: <literal>with
      <replaceable>E1</replaceable>;
      <replaceable>E2</replaceable></literal> brings all attributes
      defined in the attribute set <replaceable>E1</replaceable> in
      scope in <replaceable>E2</replaceable>.</para></listitem>

      <listitem><para>Added a <function>map</function>
      function.</para></listitem>

      <listitem><para>Various new operators (e.g., string
      concatenation).</para></listitem>

    </itemizedlist>

    </para>

  </listitem>

  <listitem><para>Expression evaluation is much
  faster.</para></listitem>

  <listitem><para>An Emacs mode for editing Nix expressions (with
  syntax highlighting and indentation) has been
  added.</para></listitem>

  <listitem><para>Many bug fixes.</para></listitem>

</itemizedlist>

</section>



<!--==================================================================-->

<section><title>Release 0.5 and earlier</title>

<para>Please refer to the Subversion commit log messages.</para>

</section>



</article>