Commit graph

7817 commits

Author SHA1 Message Date
Eelco Dolstra
249988a787 * Allow the output/expression id to be forced to a certain
value; this potentially dangerous feature enables better 
  sharing for those paths for which the content is known in 
  advance (e.g., because a content hash is given).
* Fast builds: if we can expand all output paths of a derive 
  expression, we don't have to build.
2003-07-21 20:07:12 +00:00
Eelco Dolstra
49231fbe41 * Changes to the command line syntax of Nix.
* A function to find all Nix expressions whose output ids are
  completely contained in some set.  Useful for uploading relevant Nix
  expressions to a shared cache.
2003-07-21 14:46:01 +00:00
Eelco Dolstra
401452e57a * Memoize the evaluation of Fix expressions to speed up computation. 2003-07-21 08:55:49 +00:00
Eelco Dolstra
7984cfc7c1 * Argh, another short-write problem. Added wrappers around
read()/write() to fix this once and for all.
2003-07-20 21:11:43 +00:00
Eelco Dolstra
667a6afb9d * Remove accidentally added file. 2003-07-20 19:30:53 +00:00
Eelco Dolstra
6f1a0f948d * Refactorings. 2003-07-20 19:29:38 +00:00
Eelco Dolstra
b3fc38bf6a * For debugging: `nix --verify' to check the consistency of the
database and store.
2003-07-17 12:27:55 +00:00
Eelco Dolstra
71cc3ceae5 * Preserve the executable bit. 2003-07-17 11:25:14 +00:00
Eelco Dolstra
54664b6fb7 * The write() system call can write less than the requested
number of bytes, e.g., in case of a signal like SIGSTOP.  
  This caused `nix --dump' to fail sometimes.

  Note that this bug went unnoticed because the call to `nix 
  --dump' is in a pipeline, and the shell ignores non-zero 
  exit codes from all but the last element in the pipeline.  
  Is there any way to check the result of the initial elements
  in the pipeline?  (In other words, is it at all possible to 
  write reliable shell scripts?)
2003-07-16 21:24:02 +00:00
Eelco Dolstra
6822fd7bf4 * Bug fix: slices are transitive, so if we detect that an
input path is referenced in an output paths, we also have to 
  add all ids referenced by that input path.
* Better debug assertions to catch these sorts of errors.
2003-07-16 20:33:29 +00:00
Eelco Dolstra
9d56ca219f * Substitute fixes. 2003-07-16 20:00:51 +00:00
Eelco Dolstra
b9ecadee6e * Fix the -qr query. 2003-07-16 11:05:59 +00:00
Eelco Dolstra
c11bbcfd26 * Fix self-referential outputs.
* Fix -qp query.
2003-07-16 08:30:26 +00:00
Eelco Dolstra
d41d085b77 * Get Fix and Nix to work again. 2003-07-15 22:28:27 +00:00
Eelco Dolstra
7b3f44e05b * The new normaliser now passes the unit tests. 2003-07-15 21:24:05 +00:00
Eelco Dolstra
f5b6fa5256 * Basic work on allowing derive expressions to build multiple paths.
This is not entirely trivial since this introduces the possibility
  of mutual recursion.
* Made normal forms self-contained.
* Use unique ids, not content hashes, for content referencing.
2003-07-15 16:28:54 +00:00
Eelco Dolstra
3509299aca * After building, scan for actual file system references as
opposed to declared references.  This prunes the reference
  graph, thus allowing better garbage collection and more
  efficient derivate distribution.
2003-07-14 10:23:11 +00:00
Eelco Dolstra
135b7d54db * Don't check for staleness by default. 2003-07-13 21:43:57 +00:00
Eelco Dolstra
e6363b05ae * Pass $(prefix) and other variables through -D..., not
through config.h, to prevent silly Autoconf problems.
2003-07-13 19:26:00 +00:00
Eelco Dolstra
9c620e4afa * Generate the scripts so that we can substitute the prefix
etc. correctly.
* Fixed nix-switch.
2003-07-13 18:58:03 +00:00
Eelco Dolstra
73b163c1a1 * Fix a bug that caused Fix not to be deterministic (due to addToStore
returning different paths if the hash of the path to be added was
  already available in the store under a different name).
2003-07-11 08:41:03 +00:00
Eelco Dolstra
c834a5c597 * Fix handling of pipes (read(2) may not return the required
number of bytes in one call).
2003-07-11 08:16:15 +00:00
Eelco Dolstra
e5fbf58041 * A command to register successor fstate expressions.
Unifying substitutes and successors isn't very feasible for now,
  since substitutes are only used when no path with a certain is
  known.  Therefore, a normal form of some expression stored as a
  substitute would not be used unless the expression itself was
  missing.
2003-07-10 18:48:11 +00:00
Eelco Dolstra
1d1c3691d2 * The policy-free derivate sharing now *almost* works. :-) For any
hash for which no local expansion is available, Nix can execute a
  `substitute' which should produce a path with such a hash.

  This is policy-free since Nix does not in any way specify how the
  substitute should work, i.e., it's an arbitrary (unnormalised)
  fstate expression.  For example, `nix-pull' registers substitutes
  that fetch Nix archives from the network (through `wget') and unpack
  them, but any other method is possible as well.  This is an
  improvement over the old Nix sharing scheme, which had a policy
  (fetching through `wget') built in.

  The sharing scheme doesn't work completely yet because successors
  from fstate rewriting have to be registered on the receiving side.
  Probably the whole successor stuff can be folded up into the
  substitute mechanism; this would be a nice simplification.
2003-07-10 15:11:48 +00:00
Eelco Dolstra
d072485d28 * Get `nix-push' working again. It now uses Nix/Fix to create Nix
archives (using the package in corepkgs/nar).
* queryPathByHash -> expandHash, and it takes an argument specifying
  the target path (which may be empty).
* Install the core Fix packages in $prefix/share/fix.  TODO: bootstrap
  Nix and install Nix as a Fix package.
2003-07-10 13:41:28 +00:00
Eelco Dolstra
5d4b90b689 * Actually go through the search directories when looking for files. 2003-07-10 09:21:40 +00:00
Eelco Dolstra
6011bd0da2 * Outline of the new scheme for derivate distribution. 2003-07-09 16:12:40 +00:00
Eelco Dolstra
2b95a9dc05 * When computing the set of paths referenced by an expression, also
include the paths of the subterms.
2003-07-09 15:02:03 +00:00
Eelco Dolstra
9a99dc736d * Canonicalise paths so that Fix produces identical Nix
expressions for identical inputs.
2003-07-08 20:26:22 +00:00
Eelco Dolstra
cab3f4977a * A path canonicaliser that doesn't depend on the existence of paths
(i.e., it doesn't use realpath(3), which is broken in any case).
  Therefore it doesn't resolve symlinks.
2003-07-08 19:58:41 +00:00
Eelco Dolstra
333f4963de * The output of a Derive() node is not a referenced path. 2003-07-08 15:33:06 +00:00
Eelco Dolstra
40274c1f4f * A command to query the paths referenced by an fstate expression.
* Use a temporary directory for build actions.
2003-07-08 13:22:08 +00:00
Eelco Dolstra
a279137327 * Get --dump' and --delete' to work again. 2003-07-08 10:00:46 +00:00
Eelco Dolstra
85a913a3e7 * Renamed id' -> name' to remove the implication of uniqueness. 2003-07-08 09:59:00 +00:00
Eelco Dolstra
0b38b43bab * deletePath() now removes the path from the hash2paths mapping. 2003-07-08 09:54:47 +00:00
Eelco Dolstra
ab644ad10b * BaseName() primitive for the generation of more sensible names
(especially in fetchurl.fix).
2003-07-08 09:53:46 +00:00
Eelco Dolstra
a5a90f501e * Get rid of the `netsources' database.
* Rename the `refs' database to `hash2paths'.
2003-07-08 08:35:06 +00:00
Eelco Dolstra
be96c2189c * --realise' -> --install'. 2003-07-07 09:29:40 +00:00
Eelco Dolstra
5895c160c4 * Make dbRefs a mapping from Hash to [Path]. 2003-07-07 09:25:26 +00:00
Eelco Dolstra
609a224848 * Fixed `make check' as well. 2003-07-07 07:44:57 +00:00
Eelco Dolstra
224c585aba * Refactoring on the file names. 2003-07-07 07:43:58 +00:00
Eelco Dolstra
7952a8053c * A utility `nix-hash' to compute Nix path hashes. 2003-07-06 15:11:02 +00:00
Eelco Dolstra
bfa5d77211 * Bug fix: properly check result of open(). 2003-07-06 15:08:39 +00:00
Eelco Dolstra
82e3d8fafe * Got Fix working again. 2003-07-06 14:20:47 +00:00
Eelco Dolstra
f826e432aa * Refactoring: move initialisation and argument parsing into a shared
file.
2003-07-04 15:42:03 +00:00
Eelco Dolstra
01b34fe584 * Cleanup. 2003-07-04 15:29:58 +00:00
Eelco Dolstra
207ff2caf0 * Caching of expression successors. 2003-07-04 12:18:06 +00:00
Eelco Dolstra
40b5936691 * Realisation of Derive(...) expressions. 2003-06-27 14:56:12 +00:00
Eelco Dolstra
3da9687854 * Realisation of File(...) expressions. 2003-06-27 13:55:12 +00:00
Eelco Dolstra
d4c3edfaba * Normalisation. 2003-06-27 09:55:31 +00:00
Eelco Dolstra
3ec5252582 * Improved syntax and semantics for Nix expressions. 2003-06-25 15:50:37 +00:00
Eelco Dolstra
2b07b0e7eb * Minor cleanups. 2003-06-25 14:58:56 +00:00
Eelco Dolstra
692b562342 * `nix --delete' command. 2003-06-23 14:40:49 +00:00
Eelco Dolstra
c0cbaef4be * `nix --restore' command. 2003-06-23 14:08:34 +00:00
Eelco Dolstra
5f5cab0ac7 * A function to restore from a Nix archive.
* addValue() can now import any dumpable FS object.
2003-06-23 13:27:59 +00:00
Eelco Dolstra
85effedca3 * Flags to indicate how values are specified on the command line
(--hash, --file, --name).
2003-06-20 14:11:31 +00:00
Eelco Dolstra
5079ccb455 * Move most of Nix into a library (libnix.a).
* Run `test' on `make check'.
2003-06-20 10:53:04 +00:00
Eelco Dolstra
1849aa2a72 * Refactoring: move dump function into archive.cc. 2003-06-20 10:40:25 +00:00
Eelco Dolstra
38e12df631 * `nix --dump' command. 2003-06-18 14:34:43 +00:00
Eelco Dolstra
aeaffec785 * Dump symlinks. 2003-06-18 14:34:03 +00:00
Eelco Dolstra
94cf1f86bb * Lambdas, applications, substitutions. 2003-06-18 12:36:12 +00:00
Eelco Dolstra
fab7b128b9 * Automake sucks. 2003-06-18 08:07:28 +00:00
Eelco Dolstra
34fcf5fa0c * Started integrating the new evaluation model into Nix.
* Cleaned up command-line syntax.
2003-06-17 21:12:58 +00:00
Eelco Dolstra
7a96da3627 * Test for expression dereferencing. 2003-06-17 15:47:25 +00:00
Eelco Dolstra
6656993f83 * Derefencing of hashed expressions. 2003-06-17 15:45:43 +00:00
Eelco Dolstra
a7ab242fb4 * Simplify the evaluator. 2003-06-17 13:37:44 +00:00
Eelco Dolstra
c739e20585 * Argument processing. 2003-06-16 21:01:18 +00:00
Eelco Dolstra
727beb798a * Canonicalization: when hashing directories, sort the directory
entries by name.
2003-06-16 16:16:09 +00:00
Eelco Dolstra
2f04e7102e * Path hashing. 2003-06-16 15:59:23 +00:00
Eelco Dolstra
a09e66da5a * Description of path hashing algorithm. 2003-06-16 14:19:32 +00:00
Eelco Dolstra
822794001c * Started implementing the new evaluation model.
* Lots of refactorings.
* Unit tests.
2003-06-16 13:33:38 +00:00
Eelco Dolstra
b9f09b3268 * AST for Nix expressions. 2003-06-16 07:03:40 +00:00
Eelco Dolstra
21fe717ce2 * Refactoring: hash class. 2003-06-15 13:41:32 +00:00
Eelco Dolstra
f66055fa1e * Set umask to 0022 on startup. 2003-05-30 20:53:36 +00:00
Eelco Dolstra
5908663f42 * Send log output to stderr. 2003-05-30 17:01:21 +00:00
Eelco Dolstra
d1f5fd7216 * Log the output of builders to $prefix/var/log/nix. 2003-05-30 15:21:52 +00:00
Eelco Dolstra
64582f54be * Open the database read-only when enumerating tables. 2003-05-30 15:18:30 +00:00
Eelco Dolstra
4d21cda0cd * Fix for big-endian platforms: check for endianness in MD5 computations.
This is done at runtime, which is inefficient, but I can't be bothered
  to write an Autoconf test right now.
2003-05-27 09:55:47 +00:00
Eelco Dolstra
9efad76595 * FreeBSD / ISO C++ compatibility fixes. 2003-05-26 15:09:10 +00:00
Eelco Dolstra
d8bdf5b06e * Removed some debug code that prevented packages from building. 2003-05-26 14:19:25 +00:00
Eelco Dolstra
8b930a0c94 * Some refactoring. 2003-05-26 13:45:00 +00:00
Eelco Dolstra
f8d91f20e6 * Nix can now fetch prebuilts (and other files) from the network, iff
a mapping from the hash to a url has been registered through `nix
  regurl'.

* Bug fix in nix: don't pollute stdout when running tar, it made
  nix-switch barf.

* Bug fix in nix-push-prebuilts: don't create a subdirectory on the
  target when rsync'ing.
2003-05-26 09:44:18 +00:00
Eelco Dolstra
7dd91d3779 * Prebuilt package sharing. We allow transparent binary deployment by
sharing package directories (i.e., the result of building a Nix
  descriptor).

  `nix-pull-prebuilts' obtains a list of all known prebuilts by
  consulting the paths and URLs specified in
  $prefix/etc/nix/prebuilts.conf.  The mappings ($pkghash,
  $prebuilthash) and ($prebuilthash, $location) are registered with
  Nix so that it can use the prebuilt with hash $prebuilthash when
  installing a package with hash $pkghash by downloading and unpacking
  $location.

  `nix-push-prebuilts' creates prebuilts for all packages for which no
  prebuilt is known to exist.  It can then optionally upload these
  to the network through rsync.

  `nix-[pull|push]-prebuilts' just provide a policy.  Nix provides the
  mechanism through the `nix [export|regprebuilt|regurl]' commands.
2003-05-25 22:42:19 +00:00
Eelco Dolstra
0ef4b6d0f8 * Cleaned up the semantics of Fix expressions.
* Conditionals and variables in Fix expressions.  This allows, e.g.,

    Descr(
    [ Bind("pkgId", "subversion-0.21.0")

    , Bind("httpsClient", Bool(True))
    , Bind("httpServer", Bool(True))

    , Bind("ssl", If(Var("httpsClient"), Fix("./openssl-0.9.7b.fix"), ""))

    , Bind("httpd", If(Var("httpServer"), Fix("./httpd-2.0.45.fix"), ""))
    ...
    ])

  which introduces domain feature variables httpsClient and httpServer
  (i.e., whether Subversion is built with https client and webdav
  server support); the values of the variables influences package
  dependencies and the build scripts.

  The next step is to allow that packages can express constraints on
  each other.  E.g., StrategoXT is dependent on an ATerm library with
  the "gcc" variant enabled.  In fact, this may cause several
  Nix instantiations to be created from a single Fix descriptor.  If
  possible, Fix should try to find the least set of instantiations
  that obeys the constraints.
2003-04-25 15:33:50 +00:00
Eelco Dolstra
fcc5ae151b * Remove build directory from a package directory after building it. 2003-04-25 15:01:15 +00:00
Eelco Dolstra
24b3d0759e * File removed. 2003-04-24 11:41:50 +00:00
Eelco Dolstra
9713e8577f * getpkg, delpkg, and so on now accept multiple arguments. 2003-04-24 11:41:24 +00:00
Eelco Dolstra
e59c3246b9 * Redirect stdout to stderr when executing the build script. 2003-04-23 07:21:57 +00:00
Eelco Dolstra
b762f4df7f * In `fix --instantiate', only print out the hashes of the Nix
descriptors generated out of Fix descriptors specified on the
  command line.  This allows us to say:

    nix-switch $(fix -i ./test/fixdescriptors/system.fix)
2003-04-09 12:37:49 +00:00
Eelco Dolstra
aa8fda4b54 * We no longer use nix-populate standalone, rather we use it as a
build action for `system' packages (like system.fix) that have
  dependencies on all packages we want to activate.

  So the command sequence to switch to a new activation configuration
  of the system would be:

    $ fix -i .../fixdescriptors/system.fix
    ...
    system.fix -> 89cf4713b37cc66989304abeb9ea189f

    $ nix-switch 89cf4713b37cc66989304abeb9ea189f

* A nix-profile.sh script that can be included in .bashrc.
2003-04-09 12:26:48 +00:00
Eelco Dolstra
f56b7312b2 * Descriptor importing in Fix. 2003-04-08 15:36:54 +00:00
Eelco Dolstra
cc6eafb3d0 * FreeBSD / gcc 2.95 compatibility fixes.
* A script to generate prebuilt registration scripts.
2003-04-08 14:05:15 +00:00
Eelco Dolstra
0d2b24cdd1 * `Fix' is a high-level descriptor instantiator for Nix. It replaces
nix-instantiate.
2003-04-08 12:00:51 +00:00
Eelco Dolstra
814b256da4 * Better installation: make directories, create database.
* Fixed the register script.
2003-04-06 22:19:26 +00:00
Eelco Dolstra
136c00e881 * Autoconf / Automake configuration and building. 2003-04-04 16:14:56 +00:00
Eelco Dolstra
ab723e341a * Minor refactoring: use iterators to process arguments. 2003-04-04 12:02:40 +00:00
Eelco Dolstra
c68dca5dac * Script to register pre-built packages. 2003-04-02 15:34:44 +00:00
Eelco Dolstra
5bc26fb73f * Importing and exporting of pre-built packages. 2003-04-02 15:34:05 +00:00
Eelco Dolstra
383f9bb0f1 * Use ATerms for Nix descriptors. 2003-04-01 14:00:47 +00:00
Eelco Dolstra
31f177ef0a * Check for collissions. 2003-03-28 16:27:23 +00:00
Eelco Dolstra
278ea4097e * Don't fork in `nix run'. 2003-03-28 10:33:34 +00:00
Eelco Dolstra
f915f77349 * Allow arguments to be passed to programs in `nix run'. 2003-03-28 09:53:22 +00:00
Eelco Dolstra
0f40a560ca * Added a script nix-activate which builds a list of "activated"
packages (i.e., the packages that should appear in the user's $PATH,
  and so on).  Based on this list, the script nix-populate creates a
  hierarchy of symlinks to the relevant files in those packages (e.g.,
  for pkg/bin and pkg/lib).  

  A nice property of nix-populate is that on each run it creates a
  *new* tree, rather than updating the old one.  It then atomically
  switches over to the new tree.  This allows atomic upgrades or
  rollbacks on the set of activated packages.
2003-03-25 16:36:25 +00:00
Eelco Dolstra
3f1a1457e9 * Integrate hash into instantiated descriptor file names.
* Use MD5::Digest.
2003-03-25 11:39:51 +00:00
Eelco Dolstra
73c53935d0 * For efficiency: md5 integrated into nix.
* Command `nix ensure' which is like `nix getpkg' except that if the
  has refers to a run action it will just ensure that the imports are
  there.
* Command `nix closure' to print out the closure of the set of
  descriptors under the import relation, starting at a set of roots.
  This can be used for garbage collection (e.g., given a list of
  `activated' packages, we can delete all packages not reachable from
  those).
* Command `nix graph' to print out a Dot graph of the dependency
  graph.
* `nix-addroot' adds a root for the (unimplemented) garbage collector.
2003-03-24 17:49:56 +00:00
Eelco Dolstra
2dc84e5569 * Descriptors now have a "system" field specifying the platform that
the build or run action should be perfomed on.  This ensures that
  descriptors have different hashes on different platforms.
2003-03-24 12:49:40 +00:00
Eelco Dolstra
9d2f128252 * Refactoring. 2003-03-24 11:50:20 +00:00
Eelco Dolstra
20d165c344 * A command to run programs in Nix packages, that is, to execute a run
action.  Run actions are described by uniquely hashed descriptors,
  just like build actions.  Therefore run actions can have
  dependencies, but these need not be the same as the build time
  dependencies (e.g., at runtime we can link against a different
  version of a dynamic library).  Example:

    nix run 31d6bf4c171282367065e0deecd7c579

  will run the Pan 0.13.91 newsreader with gtkspell support.
2003-03-23 23:24:09 +00:00
Eelco Dolstra
800d8e950f * Added a command to list installed packages. 2003-03-21 15:58:40 +00:00
Eelco Dolstra
2e59698b78 * Added a command to verify the consistency of the database. 2003-03-21 15:53:35 +00:00
Eelco Dolstra
e582ee67cd * Fetch sources from the network. 2003-03-21 14:10:06 +00:00
Eelco Dolstra
cadc3852e4 * nix-instantiate now instantiantes the closure of the set of
descriptor templates under the import relation.  I.e., we can now
  say:

    nix-instantiate outdir foo.nix

  which will create descriptors for foo.nix and all imported packages
  in outdir/.
2003-03-20 22:23:48 +00:00
Eelco Dolstra
f7a98e081d * Various updates. 2003-03-20 16:53:00 +00:00
Eelco Dolstra
b3594e9eaf * A script to instantiate package descriptors from templates. 2003-03-20 16:52:30 +00:00
Eelco Dolstra
8999f923ea * Improved Nix. Resources (package descriptors and other source
files) are now referenced using their cryptographic hashes.  

  This ensures that if two package descriptors have the same contents,
  then they describe the same package.  This property is not as
  trivial as it sounds: generally import relations cause this property
  not to hold w.r.t. temporality.  But since imports also use hashes
  to reference other packages, equality follows by induction.
2003-03-14 16:43:14 +00:00
Eelco Dolstra
18ebf518de * Converted to C++. 2003-03-13 16:28:32 +00:00
Eelco Dolstra
75d788b0f2 * Initial version of nix. 2003-03-13 14:24:49 +00:00