Commit graph

1252 commits

Author SHA1 Message Date
Eelco Dolstra
45d822f29c * Primops (not yet finished). 2010-03-26 15:45:53 +00:00
Eelco Dolstra
cad8726b2c * Implemented the ==' and !=' operators. These now use a deep
equality test, so they also work for (finite) attribute sets and
  lists.
2010-03-26 13:27:26 +00:00
Eelco Dolstra
8da118e4d0 * Measure stack usage. 2010-03-25 16:35:24 +00:00
Eelco Dolstra
c2ba4313fb * Implemented lists. 2010-03-25 15:38:37 +00:00
Eelco Dolstra
25eedf085d * Quick and dirty implementation of with'. with e1; e2' is
basically desugared to `let <with> = e1; e2', and `lookupVar' looks
  in each <with> in the environment chain for an attribute with the
  specified name.
2010-03-25 14:51:04 +00:00
Eelco Dolstra
3c9f8fc9b6 * Don't convert variable names to strings. 2010-03-25 13:10:04 +00:00
Eelco Dolstra
f450384ded * Implement blackholing. 2010-03-25 12:51:14 +00:00
Eelco Dolstra
ef8bd919fc * Implement `...' and default function arguments. 2010-03-25 12:45:23 +00:00
Eelco Dolstra
8a10360c91 * Simplify @-patterns: only {attrs}@name' or name@{attrs}' are now
allowed.  So `name1@name2', `{attrs1}@{attrs2}' and so on are now no
  longer legal.  This is no big loss because they were not useful
  anyway.

  This also changes the output of builtins.toXML for @-patterns
  slightly.
2010-03-25 12:19:41 +00:00
Eelco Dolstra
7482349fe8 * Implemented attribute set pattern matches. 2010-03-24 23:40:00 +00:00
Eelco Dolstra
0fd3648d34 * Store values in environments. 2010-03-24 12:41:08 +00:00
Eelco Dolstra
b70bd8fe56 * Reduce the number of value allocations in eval() by moving
responsibility for allocation of the result to the caller.
2010-03-24 12:11:38 +00:00
Eelco Dolstra
d31c59eb17 * Plain lambdas. 2010-03-24 11:06:05 +00:00
Eelco Dolstra
e8f7978274 2010-03-23 19:19:52 +00:00
Eelco Dolstra
0910ae9568 * Start of an evaluator that uses call-by-need (with thunk updating)
instead of (memoised) call-by-name.
2010-03-23 17:30:50 +00:00
Eelco Dolstra
141294ff38 * Clean up error messages in killUser(). 2010-03-19 11:36:34 +00:00
Eelco Dolstra
f0c473c5f7 * Fix building on Cygwin (http://hydra.nixos.org/build/325071). 2010-03-16 13:01:52 +00:00
Nicolas Pierron
741b7577c1 Merge r20344 & r20346. 2010-03-14 11:58:07 +00:00
Lluís Batlle i Rossell
2fb0df83e9 Uh. somehow a Makefile tab got in as spaces. Fixing. 2010-03-11 21:22:52 +00:00
Lluís Batlle i Rossell
13cce8ec45 Making 'bin2c' to be built with the compiler for the local system.
I copied the configure.ac code about CC_FOR_BUILD from libX11.
2010-03-11 20:56:25 +00:00
Ludovic Courtès
05e15049a5 Show the build user's group in /etc/group in chroots.
* src/libstore/build.cc (nix::DerivationGoal::startBuilder): Create
  /etc/group showing the build user's group.
2010-03-11 14:47:04 +00:00
Ludovic Courtès
c752c9f41a Fix thinko in r20547.
* src/libstore/build.cc (nix::DerivationGoal::startBuilder): Fix the GID
  of the build user in /etc/passwd.
2010-03-11 10:33:04 +00:00
Ludovic Courtès
2e8eaca573 Clear supplementary groups of `nixbld' in /etc/passwd in chroots.
* src/libstore/build.cc (nix::DerivationGoal::startBuilder): Don't
  display any supplementary groups for `nixbld' in /etc/passwd.
2010-03-11 10:21:23 +00:00
Eelco Dolstra
84a4dd5ff0 * Don't use fdatasync since it doesn't work on Snow Leopard.
* Don't refer to config.h in util.hh, because config.h is not
  installed (http://hydra.nixos.org/build/303053).
2010-02-24 15:46:06 +00:00
Ludovic Courtès
20186a4079 Don't rely on `PATH_MAX' on GNU. 2010-02-10 15:55:50 +00:00
Eelco Dolstra
4e17be7981 * Revert r19797, and use a simpler solution: just don't monitor build
hooks for silence.  It's unnecessary because the remote nix-store
  command is already monitoring the real build.
2010-02-03 21:38:41 +00:00
Eelco Dolstra
f859a8d3c3 * While waiting for a lock, print a sign of life every 5 minutes.
This prevents remote builders from being killed by the
  `max-silent-time' inactivity monitor while they are waiting for a
  long garbage collection to finish.  This happens fairly often in the
  Hydra build farm.
2010-02-03 21:22:57 +00:00
Eelco Dolstra
4bbbe25802 * Remove most Cygwin-specific code. Cygwin 1.7 implements advisory
POSIX locks, and simulates Unix-style file deletion semantics
  sufficiently.  Note that this means that Nix won't work on Cygwin
  1.5 anymore.
2010-02-02 15:28:36 +00:00
Eelco Dolstra
2723d9b56e * If fdatasync() isn't available, use fsync(). 2010-02-02 11:57:49 +00:00
Eelco Dolstra
07ffdc2862 * Added an option "fsync-metadata" to fsync() changes to
/nix/var/nix/db.
* Removed the function writeStringToFile since it does (almost) the
  same thing as writeFile.
2010-01-29 12:22:58 +00:00
Eelco Dolstra
ad529fb89f * Don't consider a store path valid if its info file exists but is
zero bytes long.  That makes Nix more robust in case of crashes
  (especially on ext4).
2010-01-29 11:53:58 +00:00
Eelco Dolstra
fdcaf37361 * Made `nix-store -qR --include-outputs' much faster if there are
multiple paths specified on the command line (from O(n * m) to O(n +
  m), where n is the number of arguments and m is the size of the
  closure).
2010-01-25 17:18:44 +00:00
Eelco Dolstra
5388944e8d * Make the garbage collector do the right thing when `gc-keep-outputs'
is enabled by not depending on the deriver.
2010-01-25 16:04:32 +00:00
Eelco Dolstra
f0c0277970 * On startup, set the default SIGCHLD handler. This is so that Nix
works correctly in weird environments where the SIGCHLD handler is
  set to "ignore".
2010-01-12 12:22:38 +00:00
Eelco Dolstra
ef92a14bfe * Include config.h before the C library headers, because it defines
_FILE_OFFSET_BITS=64.  Without it, functions like stat() fail on
  large file sizes.  This happened with a Nix store on squashfs:

  $ nix-store --dump /tmp/mnt/46wzqnk4cbdwh1dclhrpqnnz1icak6n7-local-net-cmds > /dev/null
  error: getting attributes of path `/tmp/mnt/46wzqnk4cbdwh1dclhrpqnnz1icak6n7-local-net-cmds': Value too large for defined data type

  $ stat /tmp/mnt/46wzqnk4cbdwh1dclhrpqnnz1icak6n7-local-net-cmds
  File: `/tmp/mnt/46wzqnk4cbdwh1dclhrpqnnz1icak6n7-local-net-cmds'
  Size: 0               Blocks: 36028797018963968 IO Block: 1024   regular empty file

  (This is a bug in squashfs or mksquashfs, but it shouldn't cause Nix
  to fail.)
2009-12-17 14:12:44 +00:00
Eelco Dolstra
945d8218fb * Build correctly against newer ATerm releases. Fixes "error: 'union'
tag used in naming 'struct _ATerm'".
2009-12-16 15:29:50 +00:00
Eelco Dolstra
bcd6cdf0d8 * Give a better error message when trying to build something and
readOnlyMode is set.
2009-12-09 17:45:22 +00:00
Eelco Dolstra
13618b191e * Grrr. 2009-11-24 13:28:46 +00:00
Eelco Dolstra
aa5a768720 * GCC 4.4 is stricter about the EOF macro
(http://hydra.nixos.org/build/156340).
2009-11-24 12:56:26 +00:00
Eelco Dolstra
9b8fda796b * Templatise getIntArg / string2Int. 2009-11-24 12:26:25 +00:00
Eelco Dolstra
8022015552 * In the garbage collector, don't count files with a link count > 1 in
the "bytes/blocks freed" statistics.
2009-11-24 10:51:52 +00:00
Eelco Dolstra
f9e766db98 * Randomise the order in which we delete entries to make the collector
less biased towards deleting paths that come alphabetically first
  (e.g. /nix/store/000...).  This matters when using --max-freed etc.
2009-11-24 09:53:18 +00:00
Eelco Dolstra
ca50c83fbb 2009-11-23 21:21:29 +00:00
Eelco Dolstra
3d55f1eb57 * A command `nix-store --query --roots <paths>' to find the garbage
collector roots that point (directly or indirectly) to the given
  paths.
2009-11-23 18:16:25 +00:00
Eelco Dolstra
ae6bf87273 * `nix-store --gc --print-roots': also print the path of the actual
root symlink, not just its target.  E.g.:

  /nix/var/nix/profiles/system-99-link -> /nix/store/76kwf88657nq7wgk1hx3l1z5q91zb9wd-system
2009-11-23 17:23:12 +00:00
Eelco Dolstra
c364d5d1e3 * Made the garbage collector a lot faster. It no longer computes the
complete set of live and dead paths before starting the actual
  deletion, but determines liveness on demand.  I.e. for any path in
  the store, it first tries to delete all the referrers, and then the
  path itself.  This means that the collector can start deleting paths
  almost immediately.
2009-11-23 16:34:24 +00:00
Eelco Dolstra
8824d60fe5 * Remove the --use-atime / --max-atime garbage collector flags. Many
(Linux) machines no longer maintain the atime because it's too
  expensive, and on the machines where --use-atime is useful (like the
  buildfarm), reading the atimes on the entire Nix store takes way too
  much time to make it practical.
2009-11-20 17:12:38 +00:00
Eelco Dolstra
327a232c85 * Remove support for old (before Nix 0.12pre12020) databases. 2009-11-06 01:15:44 +00:00
Eelco Dolstra
8520542071 * When building in a chroot, make a copy of a file if hard-linking
fails.  This is likely to happen after a `nix-store --optimise',
  because some files may have 32000 links (NIX-111).
2009-10-22 08:28:33 +00:00
Eelco Dolstra
6b9f6b0222 * Remove a prototype for a function that no longer exists. 2009-10-22 08:12:38 +00:00
Eelco Dolstra
deb342fb08 * builtins.trace: in the common case that the value is a string, then
show the string, not the ATerm, so we get `trace: bla' instead of
  `trace: Str("bla",[])'.
2009-10-22 08:10:12 +00:00
Eelco Dolstra
437077c39d * Added a primop unsafeDiscardOutputDependency needed by Disnix to
pass derivation paths to a builder without actually building them.
2009-10-21 15:05:30 +00:00
Eelco Dolstra
6f7d7bc1de * Give a useful error message when an evaluation error occurs while
trying to upgrade a package.
2009-10-13 09:30:17 +00:00
Sander van der Burg
53a4981fa2 Added optional parameter which adds -lnsl -lsocket to make the Nix package manager work on OpenSolaris 2009-10-08 14:50:37 +00:00
Eelco Dolstra
96f1517831 * Support platforms that don't have O_ASYNC (e.g. OpenSolaris
apparently).
2009-09-30 11:32:04 +00:00
Eelco Dolstra
1a8f8fd86f * OpenSolaris compatibility. 2009-09-30 09:54:29 +00:00
Eelco Dolstra
c7057fc1f2 * And some more. 2009-09-24 07:39:55 +00:00
Eelco Dolstra
193f59e077 * Fix a build failure on Fedora 11. rename() needs <stdio.h>. 2009-09-24 07:21:29 +00:00
Eelco Dolstra
1332dd1ed3 * tryEval shouldn't catch all exceptions of type Error, since not all
of them leave the evaluator in a continuable state.  Also, it should
  be less chatty.
2009-09-23 19:19:26 +00:00
Eelco Dolstra
676e07902e * Darwin hack. 2009-09-23 18:04:55 +00:00
Eelco Dolstra
64e89980e8 * Create some state directories automatically as a convenience. 2009-09-23 17:05:51 +00:00
Rob Vermaas
48b58617e9 * include wait.h for WEXITSTATUS 2009-09-23 12:57:15 +00:00
Eelco Dolstra
0dbd4638e0 * Two primops: builtins.intersectAttrs and builtins.functionArgs.
intersectAttrs returns the (right-biased) intersection between two
  attribute sets, e.g. every attribute from the second set that also
  exists in the first.  functionArgs returns the set of attributes
  expected by a function.

  The main goal of these is to allow the elimination of most of
  all-packages.nix.  Most package instantiations in all-packages.nix
  have this form:

    foo = import ./foo.nix {
      inherit a b c;
    };

  With intersectAttrs and functionArgs, this can be written as:

    foo = callPackage (import ./foo.nix) { };

  where

   callPackage = f: args:
     f ((builtins.intersectAttrs (builtins.functionArgs f) pkgs) // args);

  I.e., foo.nix is called with all attributes from "pkgs" that it
  actually needs (e.g., pkgs.a, pkgs.b and pkgs.c).  (callPackage can
  do any other generic package-level stuff we might want, such as
  applying makeOverridable.)  Of course, the automatically supplied
  arguments can be overriden if needed, e.g.

    foo = callPackage (import ./foo.nix) {
      c = c_version_2;
    };

  but for the vast majority of packages, this won't be needed.

  The advantages are to reduce the amount of typing needed to add a
  dependency (from three sites to two), and to reduce the number of
  trivial commits to all-packages.nix.  For the former, there have
  been two previous attempts:

    - Use "args: with args;" in the package's function definition.
      This however obscures the actual expected arguments of a
      function, which is very bad.

    - Use "{ arg1, arg2, ... }:" in the package's function definition
      (i.e. use the ellipis "..." to allow arbitrary additional
      arguments), and then call the function with all of "pkgs" as an
      argument.  But this inhibits error detection if you call it with
      an misspelled (or obsolete) argument.
2009-09-15 13:01:46 +00:00
Michael Raskin
3bca8931e8 Adding tryEval builtin. It allows to catch presence of errors in an expression. 2009-08-25 16:06:46 +00:00
Eelco Dolstra
c6890d6b5c * Replace newlines in table output with spaces. 2009-07-02 08:52:12 +00:00
Eelco Dolstra
749dd97a54 * Support integers and lists of strings in meta fields. This is
useful for fields like meta.maintainers, meta.priority (which can be
  a proper integer now) and even meta.license (if there are multiple
  licenses).
2009-06-30 15:53:39 +00:00
Eelco Dolstra
f2c3fc5191 * Don't show trace information by default (`--show-trace' to enable).
NixOS evaluation errors in particular look intimidating and
  generally aren't very useful.  Ideally the builtins.throw messages
  should be self-contained.
2009-06-30 13:28:29 +00:00
Eelco Dolstra
14bc3ce3d6 * Canonicalise timestamps in the Nix store to 1 (1970-01-01 00:00:01
UTC) rather than 0 (00:00:00).  1 is a better choice because some
  programs use 0 as a special value.  For instance, the Template
  Toolkit uses a timestamp of 0 to denote the non-existence of a file,
  so it barfs on files in the Nix store (see
  template-toolkit-nix-store.patch in Nixpkgs).  Similarly, Maya 2008
  fails to load script directories with a timestamp of 0 and can't be
  patched because it's closed source.

  This will also shut up those "implausibly old time stamp" GNU tar
  warnings.
2009-06-13 16:30:58 +00:00
Eelco Dolstra
a64bbe049e * Change the scoping of "inherit (e) ..." in recs so that the
attributes of the rec are in scope of `e'.  This is useful in
  expressions such as

    rec {
      lib = import ./lib;
      inherit (lib) concatStrings;
    }

  It does change the semantics of expressions such as

    let x = {y = 1;}; in rec { x = {y = 2;}; inherit (x) y; }.y

  This now returns 2 instead of 1.  However, no code in Nixpkgs or
  NixOS seems to rely on the old behaviour.
2009-05-15 13:46:13 +00:00
Eelco Dolstra
d407d572fd * Some syntactic sugar for attribute sets: allow {x.y.z = ...;} as a
shorthand for {x = {y = {z = ...;};};}.  This is especially useful
  for NixOS configuration files, e.g.

    {
      services = {
        sshd = {
          enable = true;
          port = 2022;
        };
      };
    }

  can now be written as

    {
      services.sshd.enable = true;
      services.sshd.port = 2022;
    }

  However, it is currently not permitted to write
  
    {
      services.sshd = {enable = true;};
      services.sshd.port = 2022;
    }

  as this is considered a duplicate definition of `services.sshd'.
2009-05-15 12:35:23 +00:00
Eelco Dolstra
e42975490f * Check for duplicate attributes in fixAttrs, rather than doing a
separate traversal after parsing.  Likewise, check for duplicate
  pattern variables right away.
2009-05-14 14:29:45 +00:00
Eelco Dolstra
50d11b90ca * Allow unsafe (unspecified) comparisons between attrsets unless
NIX_NO_UNSAFE_EQ is set, for now.
2009-05-12 11:06:24 +00:00
Eelco Dolstra
c34e6d71bc * Disallow equality tests between attribute sets. This was always
broken, but now the evaluator checks for it to prevent Nix
  expressions from relying on undefined behaviour.  Equality tests are
  implemented using a shallow pointer equality test between ATerms.
  However, because attribute sets are lazy and contain position
  information, this can give false positives.  For instance,
  previously

    let y = {x = 1;}; in y == y

  evaluated to true, while the equivalent expression

    {x = 1;} == {x = 1;}

  evaluated to false.  So disallow these tests for now.  (Eventually
  we may want to implement deep equality tests for attribute sets,
  like lib.eqStrict.)
  
* Idem: disallow comparisons between functions.

* Implemented deep comparisons of lists.  This had the same problem as
  attribute sets - the elements in the list weren't evaluated.  For
  instance,

    ["xy"] == [("x" + "y")]

  evaluated to false.  Now it works properly.
2009-05-11 15:50:14 +00:00
Eelco Dolstra
7660e2a068 * Remove a right recursion that causes the parser to barf on very long
lists.  The comment about ATreverse requiring unbounded stack space
  was unfounded anyway.
2009-05-07 11:35:52 +00:00
Michael Raskin
098cb9d233 Add an ftruncate call paired with fallocate to play safe with some FSes (namely, BtrFS fallocate sets file size to allocated size, i.e. multiple of block size) 2009-05-04 08:10:24 +00:00
Eelco Dolstra
d5eab2fc82 * Use foreach in a lot of places. 2009-04-21 11:52:16 +00:00
Eelco Dolstra
8f1bf28505 * nix-store --verify: don't bail out if a referenced path is missing.
(It can't fix it though.)
2009-04-21 11:06:27 +00:00
Eelco Dolstra
4e646b0ddb * Fix a few "comparison is always false/true due to limited range of
data type" warnings on 64-bit platforms.  The one in parser.y is
  likely to be a real bug.
2009-04-16 12:03:17 +00:00
Eelco Dolstra
dfb863f333 * Don't cache transient build hook problems. 2009-04-15 06:25:02 +00:00
Eelco Dolstra
351bf658f9 * Do a substitution even if --max-jobs == 0. 2009-03-31 21:14:07 +00:00
Eelco Dolstra
cff2b2a13a * Doh! 2009-03-31 15:50:03 +00:00
Eelco Dolstra
160a60d663 2009-03-30 19:35:55 +00:00
Eelco Dolstra
805144b705 * Make the poll interval configurable. 2009-03-30 11:34:03 +00:00
Eelco Dolstra
7377195297 * With `--max-jobs 0', print a nicer error message than "Assertion
`!awake.empty()' failed."
2009-03-29 18:06:00 +00:00
Eelco Dolstra
737423a89c * Use polling to wait for a remote build slot when using a build hook
(that is, call the build hook with a certain interval until it
  accepts the build).
* build-remote.pl was totally broken: for all system types other than
  the local system type, it would send all builds to the *first*
  machine of the appropriate type.
2009-03-29 18:00:45 +00:00
Eelco Dolstra
6e946c8e72 * Scan for references and compute the SHA-256 hash of the output in
one pass.  This halves the amount of I/O.
2009-03-28 20:51:33 +00:00
Eelco Dolstra
c7152c8f97 * Don't use the non-standard __gnu_cxx::stdio_filebuf class. 2009-03-28 19:41:53 +00:00
Eelco Dolstra
3a2bbe7f8a * Simplify communication with the hook a bit (don't use file
descriptors 3/4, just use stdin/stderr).
2009-03-28 19:29:55 +00:00
Eelco Dolstra
f54e800366 * The `fixedOutput' variable didn't get initialised when using a build
hook, causing negative caching to fail randomly.
2009-03-28 16:12:27 +00:00
Eelco Dolstra
cb85bc396e * Create /nix/store if it doesn't exist. 2009-03-27 14:19:04 +00:00
Eelco Dolstra
5dd8fb2069 * Don't use ULLONG_MAX in maxFreed - use 0 to mean "no limit".
18446744073709551615ULL breaks on GCC 3.3.6 (`integer constant is
  too large for "long" type').
2009-03-26 11:02:07 +00:00
Eelco Dolstra
92f525ecf4 * Negative caching, i.e. caching of build failures. Disabled by
default.  This is mostly useful for Hydra.
2009-03-25 21:05:42 +00:00
Eelco Dolstra
a046858a22 * Doh. 2009-03-24 23:40:24 +00:00
Eelco Dolstra
12c8c64efa 2009-03-24 14:07:37 +00:00
Eelco Dolstra
3a87163b24 * Shut up a GCC warning. 2009-03-23 13:52:52 +00:00
Eelco Dolstra
cacff1be88 * No longer block while waiting for a lock on a store path. Instead
poll for it (i.e. if we can't acquire the lock, then let the main
  select() loop wait for at most a few seconds and then try again).
  This improves parallelism: if two nix-store processes are both
  trying to build a path at the same time, the second one shouldn't
  block; it should first see if it can build other goals.  Also, it
  prevents the deadlocks that have been occuring in Hydra lately,
  where a process waits for a lock held by another process that's
  waiting for a lock held by the first.

  The downside is that polling isn't really elegant, but POSIX doesn't
  provide a way to wait for locks in a select() loop.  The only
  solution would be to spawn a thread for each lock to do a blocking
  fcntl() and then signal the main thread, but that would require
  pthreads.
2009-03-23 01:05:54 +00:00
Eelco Dolstra
58969fa2bf * Refactoring. 2009-03-22 23:53:05 +00:00
Eelco Dolstra
7a57b2920b * Better error message when nix-store --import is applied to garbage
(previously it would likely say "implementation cannot deal with >
  32-bit integers").
2009-03-22 17:51:45 +00:00
Eelco Dolstra
77d272623f * NAR archives: handle files larger than 2^32 bytes. Previously it
would just silently store only (fileSize % 2^32) bytes.
* Use posix_fallocate if available when unpacking archives.
* Provide a better error message when trying to unpack something that
  isn't a NAR archive.
2009-03-22 17:36:43 +00:00
Eelco Dolstra
2897286487 * Unify exportReferencesGraph and exportBuildReferencesGraph, and make
sure that it works as expected when you pass it a derivation.  That
  is, we have to make sure that all build-time dependencies are built,
  and that they are all in the input closure (otherwise remote builds
  might fail, for example).  This is ensured at instantiation time by
  adding all derivations and their sources to inputDrvs and inputSrcs.
2009-03-18 17:36:42 +00:00