Commit graph

7613 commits

Author SHA1 Message Date
Eelco Dolstra
cf7e645a48 * Regression test for __overrides. 2010-10-22 15:15:12 +00:00
Eelco Dolstra
41c45a9b31 * Store Value nodes outside of attribute sets. I.e., Attr now stores
a pointer to a Value, rather than the Value directly.  This improves
  the effectiveness of garbage collection a lot: if the Value is
  stored inside the set directly, then any live pointer to the Value
  causes all other attributes in the set to be live as well.
2010-10-22 14:47:42 +00:00
Eelco Dolstra
64c3325b0b * Make building against the Boehm GC a configure option. 2010-10-22 13:39:15 +00:00
Eelco Dolstra
76feaf016a * Keep some more stats. 2010-10-20 15:48:00 +00:00
Eelco Dolstra
e879a0371b * Use the Boehm garbage collector to reclaim unused memory in the Nix
expression evaluator.
2010-10-20 11:38:30 +00:00
Eelco Dolstra
b0c11cda7e * Evaluator garbage collection branch. 2010-10-20 09:08:39 +00:00
Eelco Dolstra
64fd29855a * Wrap deleteFromStore() in a transaction. Otherwise there might be a
race with other processes that add new referrers to a path,
  resulting in the garbage collector crashing with "foreign key
  constraint failed".  (Nix/4)
* Make --gc --print-dead etc. interruptible.
2010-10-14 15:55:51 +00:00
Rob Vermaas
8dadcede65 nix manual: fix 'install' -> 'uninstall' in garbage collection section of introduction 2010-10-06 19:04:04 +00:00
Eelco Dolstra
7119d38287 * In the referrers test, lower the nesting depth from 2500 to 1000 to
prevent hitting a stack overflow bug in the garbage collector.
2010-10-04 22:26:38 +00:00
Eelco Dolstra
705868a8a9 * Make sure that config.h is included before the system headers,
because it defines _FILE_OFFSET_BITS.  Without this, on 
  OpenSolaris the system headers define it to be 32, and then 
  the 32-bit stat() ends up being called with a 64-bit "struct 
  stat", or vice versa.

  This also ensures that we get 64-bit file sizes everywhere.

* Remove the redundant call to stat() in parseExprFromFile().
  The file cannot be a symlink because that's the exit condition
  of the loop before.
2010-10-04 17:55:38 +00:00
Eelco Dolstra
95f4f2cf61 * If std::tr1::unordered_set is unavailable, use std::set. 2010-10-04 16:16:19 +00:00
Eelco Dolstra
36a23e86b6 * "type -P" isn't portable. 2010-10-04 15:50:08 +00:00
Eelco Dolstra
bfa6ee7d91 * Don't use SSH's `-tt' flag because it doesn't seem to work
on OpenSolaris when using connection sharing.  Instead have
  the remote side check for disconnection and kill the process 
  group when that happens.
2010-10-04 12:30:46 +00:00
Eelco Dolstra
71dfe4b90b * Sync with the trunk. 2010-10-04 11:44:47 +00:00
Eelco Dolstra
450837bcc8 * In printMsg(), ignore failing writes to stderr if we're in an
exception handler, otherwise throw an exception.  We need to ignore
  write errors in exception handlers to ensure that cleanup code runs
  to completion if the other side of stderr has been closed
  unexpectedly.
2010-10-04 11:23:07 +00:00
Eelco Dolstra
4aa9245083 * Hack needed for GCC 4.3.2 on OpenSolaris. 2010-10-04 10:51:16 +00:00
Eelco Dolstra
923736df38 * Doh. Remove debug message. 2010-09-14 12:47:19 +00:00
Eelco Dolstra
e4907411c2 * Only do "pragma journal_mode = ..." if the current journal mode
differs from the desired mode.  There is an open SQLite ticket
  `Executing "PRAGMA journal_mode" may delete journal file while it is
  in use.'
2010-09-01 11:36:22 +00:00
Eelco Dolstra
bf0dde9597 * Always print hook output on stderr, even if --no-build-output is
set.
* In the build hook, print a trace message to allow Hydra to pick up
  the name of the remote machine used for the build.
2010-08-31 12:36:24 +00:00
Eelco Dolstra
e2e168f7c2 `nix-store --verify' improvements:
* If a path has disappeared, check its referrers first, and don't try
  to invalidate paths that have valid referrers.  Otherwise we get a
  foreign key constraint violation.
* Read the whole Nix store directory instead of statting each valid
  path, which is slower.
* Acquire the global GC lock.
2010-08-31 11:47:31 +00:00
Eelco Dolstra
80e722278c * When using the build hook, distinguish between the stderr of the
hook script proper, and the stdout/stderr of the builder.  Only the
  latter should be saved in /nix/var/log/nix/drvs.
* Allow the verbosity to be set through an option.
* Added a flag --quiet to lower the verbosity level.
2010-08-30 14:53:03 +00:00
Eelco Dolstra
20acd43c25 * Disable the GC reachability check for now (when creating new roots),
as it's hopelessly inefficient.
2010-08-30 14:11:57 +00:00
Eelco Dolstra
766f708418 * Experimental feature: allow a derivation to tell the build hook that
it requires a certain feature on the build machine, e.g.

    requiredSystemFeatures = [ "kvm" ];

  We need this in Hydra to make sure that builds that require KVM
  support are forwarded to machines that have KVM support.  Probably
  this should also be enforced for local builds.
2010-08-27 13:18:13 +00:00
Eelco Dolstra
df50916e46 * Oops - "null" was displayed as "true". 2010-08-27 12:10:56 +00:00
Eelco Dolstra
e41ecbf730 2010-08-27 11:09:04 +00:00
Eelco Dolstra
e437b08250 * Made the build hook mechanism more efficient. Rather than starting
the hook every time we want to ask whether we can run a remote build
  (which can be very often), we now reuse a hook process for answering
  those queries until it accepts a build.  So if there are N
  derivations to be built, at most N hooks will be started.
2010-08-25 20:44:28 +00:00
Eelco Dolstra
1a396f3789 * Don't call "cat". 2010-08-25 12:19:30 +00:00
Eelco Dolstra
95deba581d * In the build hook, temporarily register the derivation and its
output as GC roots.  This prevents a race if the garbage collector
  is running during the build.
2010-08-25 11:54:11 +00:00
Eelco Dolstra
1e5f5ea2e9 * Correctly handle SSH failing to establish a connection. Because
this didn't work right, the build hook wouldn't fall back to using
  other machines of the desired type.
2010-08-24 14:27:07 +00:00
Eelco Dolstra
034f608e00 * Don't complain if the stored hash of a path is zero (unknown). 2010-08-24 14:25:33 +00:00
Eelco Dolstra
f58f51f380 * Handle the unlikely case where a derivation has no dependencies at
all.
2010-08-24 11:45:44 +00:00
Eelco Dolstra
955d11aae7 * Sync with the trunk. 2010-08-24 09:56:28 +00:00
Eelco Dolstra
c67eccc26d * nix-build: pass --cores. 2010-08-24 09:21:59 +00:00
Eelco Dolstra
2c8e070e5d * Bump the version number. 2010-08-17 15:39:35 +00:00
Eelco Dolstra
ed133e6e64 * rpmBuild already includes the disk image in the output name. 2010-08-17 14:08:44 +00:00
Eelco Dolstra
2de17f4edc * Update date. 2010-08-17 10:06:26 +00:00
Eelco Dolstra
86f65edf4e * Document --cores in the manual. 2010-08-17 07:22:05 +00:00
Eelco Dolstra
b75e1043a3 * Typo. 2010-08-16 13:23:45 +00:00
Eelco Dolstra
8ec6594d6d * Remove the "tarball" jobset argument. 2010-08-16 13:01:31 +00:00
Eelco Dolstra
12721a3a9a * Nix 0.16 release notes. 2010-08-16 12:38:32 +00:00
Eelco Dolstra
5fb824e896 * Urgh, this was supposed to go in the trunk... 2010-08-12 13:36:56 +00:00
Eelco Dolstra
5c5ab2bc12 * Don't link against pthreads. This was added way back in r211
because Berkeley DB needed it on some platforms, but we don't use
  BDB anymore.

  On FreeBSD, if you link against pthreads, then the main thread gets
  a 2 MB stack which cannot be overriden (it ignores "ulimit -s"):

    http://www.mail-archive.com/freebsd-hackers@freebsd.org/msg62445.html
  
  This is not enough for Nix.  For instance, the garbage collector can
  fail if there is a pathologically deep chain of references
  (http://hydra.nixos.org/build/556199).  2 MB is also not enough for
  many Nix expressions.
  
  Arguably the garbage collector shouldn't use recursion, because in
  NixOS unprivileged users can DOS the garbage collector by creating a
  sufficiently deeply nested chain of references.  But getting rid of
  recursion is a bit harder.
2010-08-12 13:34:34 +00:00
Eelco Dolstra
6846ed8b44 * Make --cores work when building through the Nix daemon. 2010-08-12 09:21:50 +00:00
Eelco Dolstra
5f9aad44ca * nix-build: recognise --cores. 2010-08-11 15:28:02 +00:00
Eelco Dolstra
d7875d1648 * Use SQLite 3.7.0.1. 2010-08-06 07:51:27 +00:00
Eelco Dolstra
587dc8aa00 * Sync with the trunk. 2010-08-04 17:48:29 +00:00
Eelco Dolstra
fd9c77dfc7 * Use SQLite 3.7.0's write-ahead logging (WAL mode). This is a lot
faster than the old mode when fsyncs are enabled, because it only
  performs an fsync() when doing a checkpoint, rather than at every
  commit.  Some timings for doing a "nix-instantiate /etc/nixos/nixos
  -A system" after modifying the stdenv setup script:

    42.5s - SQLite 3.6.23 with truncate mode and fsync
     3.4s - SQLite 3.6.23 with truncate mode and no fsync
    32.1s - SQLite 3.7.0 with truncate mode and fsync
    16.8s - SQLite 3.7.0 with WAL mode and fsync, auto-checkpoint
            every 1000 pages
     8.3s - SQLite 3.7.0 with WAL mode and fsync, auto-checkpoint
            every 8192 pages
     1.7s - SQLite 3.7.0 with WAL mode and no fsync

  The default is now to use WAL mode with fsyncs.  Because WAL doesn't
  work on remote filesystems such as NFS (as it uses shared memory),
  truncate mode can be re-enabled by setting the "use-sqlite-wal"
  option to false.
2010-08-04 17:35:59 +00:00
Eelco Dolstra
750be19ae8 * Remove "auto" and "guess" as synonyms for 0 in the handling of
build-cores and --cores.  They're superfluous and just complicate
  the parsing.
2010-08-04 12:23:59 +00:00
Eelco Dolstra
7f893b7a43 * Allow derivations to hint that they should not be built remotely
using the build hook mechanism, by setting the derivation attribute
  "preferLocalBuild" to true.  This has a few use cases:

  - The user environment builder.  Since it just creates a bunch of
    symlinks without much computation, there is no reason to do it
    remotely.  In fact, doing it remotely requires the entire closure
    of the user environment to be copied to the remote machine, which
    is extremely wasteful.

  - `fetchurl'.  Performing the download on a remote machine and then
    copying it to the local machine involves twice as much network
    traffic as performing the download locally, and doesn't save any
    CPU cycles on the local machine.
2010-08-04 12:13:58 +00:00
Eelco Dolstra
315d8fbd75 * Set the default system filter to "*". This ensures that (for
instance) "nix-env -i wine" works on x86_64-linux, even though Wine
  is built on i686-linux.  In the event that there are multiple
  matching derivations, prefer those built for the current system.
2010-08-04 09:32:42 +00:00