Commit graph

864 commits

Author SHA1 Message Date
Eelco Dolstra
ef5f254a55 * `nix-store --build' now builds its arguments in parallel instead of
sequentially (within the limits set by `--jobs').  This should
  greatly improve the utilisation of the build farm when doing Nixpkgs
  builds.
2005-01-19 15:02:02 +00:00
Eelco Dolstra
06c77bf7a8 * Change extension .store' to .drv'.
* Re-enable `nix-store --query --requisites'.
2005-01-19 14:36:00 +00:00
Eelco Dolstra
863dcff6c5 * Started removing closure store expressions, i.e., the explicit
representation of closures as ATerms in the Nix store.  Instead, the
  file system pointer graph is now stored in the Nix database.  This
  has many advantages:

  - It greatly simplifies the implementation (we can drop the notion
    of `successors', and so on).

  - It makes registering roots for the garbage collector much easier.
    Instead of specifying the closure expression as a root, you can
    simply specify the store path that must be retained as a root.
    This could not be done previously, since there was no way to find
    the closure store expression containing a given store path.
    
  - Better traceability: it is now possible to query what paths are
    referenced by a path, and what paths refer to a path.
2005-01-19 11:16:11 +00:00
Eelco Dolstra
e9762e2d10 * Support arities > 6. 2005-01-19 11:04:24 +00:00
Eelco Dolstra
6d493751c3 * Get --readonly-mode to work again. 2005-01-18 11:15:50 +00:00
Eelco Dolstra
32aac8748a * Actually check that the result of fixed-output derivations matches
the specified hash.
2005-01-17 19:01:48 +00:00
Eelco Dolstra
f3dc231250 * Removed the `id' attribute hack.
* Formalise the notion of fixed-output derivations, i.e., derivations
  for which a cryptographic hash of the output is known in advance.
  Changes to such derivations should not propagate upwards through the
  dependency graph.  Previously this was done by specifying the hash
  component of the output path through the `id' attribute, but this is
  insecure since you can lie about it (i.e., you can specify any hash
  and then produce a completely different output).  Now the
  responsibility for checking the output is moved from the builder to
  Nix itself.

  A fixed-output derivation can be created by specifying the
  `outputHash' and `outputHashAlgo' attributes, the latter taking
  values `md5', `sha1', and `sha256', and the former specifying the
  actual hash in hexadecimal or in base-32 (auto-detected by looking
  at the length of the attribute value).  MD5 is included for
  compatibility but should be considered deprecated.

* Removed the `drvPath' pseudo-attribute in derivation results.  It's
  no longer necessary.

* Cleaned up the support for multiple output paths in derivation store
  expressions.  Each output now has a unique identifier (e.g., `out',
  `devel', `docs').  Previously there was no way to tell output paths
  apart at the store expression level.

* `nix-hash' now has a flag `--base32' to specify that the hash should
  be printed in base-32 notation.

* `fetchurl' accepts parameters `sha256' and `sha1' in addition to
  `md5'.

* `nix-prefetch-url' now prints out a SHA-1 hash in base-32.  (TODO: a
  flag to specify the hash.)
2005-01-17 16:55:19 +00:00
Eelco Dolstra
d58a11e019 * Shorten SHA-256 hashes used in store path name generation to 160
bits, then encode them in a radix-32 representation (using digits
  and letters except e, o, u, and t).  This produces store paths like
  /nix/store/4i0zb0z7f88mwghjirkz702a71dcfivn-aterm-2.3.1.  The nice
  thing about this is that the hash part of the file name is still 32
  characters, as before with MD5.

  (Of course, shortening SHA-256 to 160 bits makes it no better than
  SHA-160 in theory, but hopefully it's a bit more resistant to
  attacks; it's certainly a lot slower.)
2005-01-14 16:04:03 +00:00
Eelco Dolstra
9530cc3170 * Start move towards SHA-256 hashes instead of MD5.
* Start cleaning up unique store path generation (they weren't always
  unique; in particular the suffix ("-aterm-2.2", "-builder.sh") was
  not part of the hash, therefore changes to the suffix would cause
  multiple store objects with the same hash).
2005-01-14 13:51:38 +00:00
Eelco Dolstra
a7b94e87d7 * Missing file. 2005-01-14 13:50:09 +00:00
Eelco Dolstra
9ee88bb2f2 * Use absolute paths. 2005-01-14 13:50:00 +00:00
Eelco Dolstra
63791eb05b * Add SHA-256.
* Tests for the various hashes.
2005-01-14 12:03:04 +00:00
Eelco Dolstra
37b51a9aa6 * Removed some dead code. 2005-01-14 10:16:33 +00:00
Eelco Dolstra
7e8961f720 * Added SHA-1 support. nix-hash' now has an option --type sha1' to
select SHA-1 hashing.
2005-01-13 17:39:26 +00:00
Eelco Dolstra
73992371a3 * Refactoring to support SHA-1. 2005-01-13 15:44:44 +00:00
Eelco Dolstra
d46b4262dc * Bump version number to 0.8. 2005-01-12 13:23:12 +00:00
Eelco Dolstra
b17e7cf979 * Script to remove patches from manifests. 2005-01-12 10:40:59 +00:00
Eelco Dolstra
0bc41f632b * Print out less garbage. 2005-01-12 10:37:18 +00:00
Eelco Dolstra
7d75616f2c * NEWS and manual update for release 0.7. 2005-01-12 10:27:46 +00:00
Eelco Dolstra
6af4a5a71f * Prototype store optimiser. It searched the Nix store for identical
files and hard-links them to each other to save disk space.

  Currently it doesn't actually do the hard-linking, it just reports
  the amount of space saved if it did.
2005-01-05 09:58:12 +00:00
Eelco Dolstra
a03397be4c * Cygwin compatibility. 2005-01-04 17:38:26 +00:00
Eelco Dolstra
f28ea27d83 * Remove old stuff. 2004-12-31 11:07:32 +00:00
Eelco Dolstra
c53898cb65 * If a patch already exists, it must still be included in the manifest. 2004-12-31 11:07:12 +00:00
Eelco Dolstra
35b76a81c4 * More instrumentation (statistics go to /nix/var/log/nix/downloads). 2004-12-30 17:19:47 +00:00
Eelco Dolstra
3745cecc6a * Fix handling of chained patches: don't skip patches if intermediate
paths are missing, etc.
2004-12-30 17:09:57 +00:00
Eelco Dolstra
581bcb986f * Some logging for evaluation. 2004-12-30 16:34:54 +00:00
Eelco Dolstra
6270aa727d * Propagate patches from the source distribution to the destination
distribution insofar they are applicable.
2004-12-29 22:17:26 +00:00
Eelco Dolstra
4f07ebc67e * Integrated bsdiff/bspatch 4.2 (from
http://www.daemonology.net/bsdiff/bsdiff-4.2.tar.gz) into the source
  tree.  The license is a bit peculiar, but it does allow verbatim
  copying, which is what we do here (i.e., so don't make any changes
  to the sources).
2004-12-29 22:08:48 +00:00
Eelco Dolstra
54d8f08588 * Reject patches larger than the full archives they produce. 2004-12-29 19:32:55 +00:00
Eelco Dolstra
2fdb27e7f2 * Atomic file replacement is good. 2004-12-29 19:04:21 +00:00
Eelco Dolstra
e1e9c036f9 * A utility to generate patches between releases based on their
manifests.
2004-12-29 18:58:15 +00:00
Eelco Dolstra
77fc1c6c5c * Use aterm 2.3.1. 2004-12-29 17:29:24 +00:00
Eelco Dolstra
9022cf9adf * A small utility to add the Size and NarHash fields to old manifests. 2004-12-28 21:12:00 +00:00
Eelco Dolstra
4bf58d5379 * Added a function to write manifests. 2004-12-28 21:11:28 +00:00
Eelco Dolstra
3d1b2101cc * Place manifests in /nix/var/nix/manifests.
* Use the new patch downloader.
2004-12-20 16:38:50 +00:00
Eelco Dolstra
7eed57e784 * Sync with changed substitute mechanism.
* Accept the NarHash line.
* Clear substitutes in `nix-channel --update'.
2004-12-20 14:57:03 +00:00
Eelco Dolstra
96c3d8a615 * I love test sets. 2004-12-20 14:38:04 +00:00
Eelco Dolstra
8b9697e575 * An operation `nix-store --clear-substitutes' to remove all
registered substitute mappings.
2004-12-20 14:16:55 +00:00
Eelco Dolstra
fa9259f5f5 * Simplify the substitute mechanism:
- Drop the store expression.  So now a substitute is just a
    command-line invocation (a program name + arguments).  If you
    register a substitute you are responsible for registering the
    expression that built it (if any) as a root of the garbage
    collector.
  - Drop the substitutes-rev DB table.
2004-12-20 13:43:32 +00:00
Eelco Dolstra
015beb7cd0 * Typo: genericBuilder -> genericBuild. 2004-12-17 13:46:07 +00:00
Eelco Dolstra
4d25b0b0bb * Fix nix-pull. 2004-12-16 15:31:50 +00:00
Eelco Dolstra
f4041cc175 * Commit old changed to bdiff.sh - but bdiff.sh is obsolete. 2004-12-16 14:59:05 +00:00
Eelco Dolstra
77970f8daf * Remove `prebuilts.conf' file, it's not like anybody was using it.
* Add /nix/var/nix/manifests directory.
2004-12-16 14:31:49 +00:00
Eelco Dolstra
e3b051aeeb * Include the size of the bzipped archive (necessary for computing the
cheapest download path), as well as the hash of the contents of the
  path (necessary for checking patch applicability).
2004-12-13 16:56:18 +00:00
Eelco Dolstra
862f4c154e * Patch deployment. `download.pl' (intended to be used in the
substitute mechanism) creates a store path by downloading full NAR
  archives and/or patches specified in the available manifests.

  Any combination of present paths, full downloads, and patches can be
  used to construct the target path.  In particular, patches can be
  chained in sequence; and full NAR archives of the target path can be
  omitted (i.e., patch-only deployment is possible).  A shortest path
  algorithm is used to find the smallest set of files to be downloaded
  (the edge weights are currently file sizes, but one can imagine
  taking the network speed to the various source into account).

  Patches are binary deltas between two store paths.  To be precise,
  they are the output of the `bsdiff' program applied to the NAR
  archives obtained by dumping (`nix-store --dump') the two store
  paths.  The advantage of diff'ing NAR archives (and not, say, doing
  file-by-file diffs) is that file renames/moves are handled
  automatically.  The disadvantage is that we cannot optimise creation
  of unchanged files (by hard-linking).
2004-12-13 13:47:38 +00:00
Eelco Dolstra
dca48aed34 * Allow an optional hash to be provided. This prevents redundant
fetches.
2004-12-13 13:35:36 +00:00
Eelco Dolstra
71926ee188 * Print out statistics comparing our performance to bzip2. 2004-11-29 21:04:28 +00:00
Eelco Dolstra
13f77276d1 * utime() follows symlinks, so don't change the mtime if the file is a
symlink.
2004-11-29 19:22:16 +00:00
Eelco Dolstra
eee6fe478e * Proof-of-concept for binary patch deployment. 2004-11-29 19:12:55 +00:00
Eelco Dolstra
f17553a212 * Remove debug statement. 2004-11-29 15:30:44 +00:00