Commit graph

1213 commits

Author SHA1 Message Date
Eelco Dolstra
0c478d2f4d * Ignore carriage returns. 2008-11-25 02:30:35 +00:00
Eelco Dolstra
5024bde8f4 * Handle prematurely ended logfiles, i.e. make sure we emit enough
close tags.
2008-11-25 01:06:15 +00:00
Eelco Dolstra
a55113411f * Nix daemon: reload the configuration file after forking (NIX-100). 2008-11-20 12:25:11 +00:00
Eelco Dolstra
aab530e971 * Primop builtins.storePath for declaring a store path as a
dependency.  `storePath /nix/store/bla' gives exactly the same
  result as `toPath /nix/store/bla', except that the former includes
  /nix/store/bla in the dependency context of the string.

  Useful in some generated Nix expressions like nix-push, which now
  finally does the right thing wrt distributed builds.  (Previously
  the path to be packed wasn't an explicit dependency, so it wouldn't
  be copied to the remote machine.)
2008-11-19 23:26:19 +00:00
Eelco Dolstra
e13da525a7 * Files in the info directory starting with "." are temporary files
and don't indicate path validity.
2008-11-19 16:27:07 +00:00
Eelco Dolstra
5d250ad1ea * nix-store --dump-db: be more streamy. 2008-11-19 16:26:34 +00:00
Eelco Dolstra
3f4ed681c2 * Prevent zombies. Previous the SIGCHLD handler only reaped one
zombie at a time, so if multiple children died before the handler
  got to run, some of them would not be cleaned up.
2008-11-14 16:50:01 +00:00
Eelco Dolstra
6fedb7aa0f * Restore SIGPIPE to SIG_DFL when running the builder. This prevents
subtle and often hard-to-reproduce bugs where programs in pipes
  either barf with a "Broken pipe" message or not, depending on the
  exact timing conditions.  This particularly happened in GNU M4 (and
  Bison, which uses M4).
2008-11-14 15:46:45 +00:00
Eelco Dolstra
a519bb0635 * Some somewhat ad hoc mechanism to allow the build farm to monitor
build progress.
2008-11-12 11:08:27 +00:00
Eelco Dolstra
96598e7b06 * Pass the --no-build-output flag to the daemon. 2008-11-11 15:11:10 +00:00
Eelco Dolstra
4166b11a53 * Add /dev/pts to the default nix.conf. 2008-11-11 14:59:20 +00:00
Eelco Dolstra
2b7c839b4e * Typo. 2008-11-11 14:58:37 +00:00
Eelco Dolstra
709b55ee02 * Put the chroots under /nix/var/nix/chroots to reduce the risk of
disasters involving `rm -rf' on bind mounts.  Will try the
  definitive fix (per-process mounts, apparently possible via the
  CLONE_NEWNS flag in clone()) some other time.
2008-10-29 15:34:48 +00:00
Ludovic Courtès
c98ea254dc libstore: Always mount `/dev/pts' individually.
This fixes problems such as Tcl's PTY handling:

  ERROR: The system has no more ptys.  Ask your system administrator to
  create more.
2008-10-16 21:04:32 +00:00
Eelco Dolstra
9d6d50269b * Bug fix for building on some old installations (contributed by Pjotr). 2008-10-16 14:16:03 +00:00
Eelco Dolstra
fa61ee70ee * Fix `--from-profile'. 2008-09-18 09:08:54 +00:00
Eelco Dolstra
f32fef1b07 * GC option `--max-atime' that specifies an upper limit to the last
accessed time of paths that may be deleted.  Anything more recently
  used won't be deleted.  The time is specified in time_t,
  e.g. seconds since 1970-01-01 00:00:00 UTC; use `date +%s' to
  convert to time_t from the command line. 

  Example: to delete everything that hasn't been used in the last two
  months:

  $ nix-store --gc -v --max-atime $(date +%s -d "2 months ago")
2008-09-17 14:52:35 +00:00
Eelco Dolstra
4af2fdba6d * Typo. 2008-09-17 13:00:55 +00:00
Eelco Dolstra
c987061aa4 * Some refactoring. Better output with `-v' for --use-atime. 2008-09-17 12:54:07 +00:00
Eelco Dolstra
77afd97a99 * nix-store --gc / --delete: show how many store paths were deleted. 2008-09-17 12:53:33 +00:00
Eelco Dolstra
7ab68961e4 * Garbage collector: added an option `--use-atime' to delete paths in
order of ascending last access time.  This is useful in conjunction
  with --max-freed or --max-links to prefer deleting non-recently used
  garbage, which is good (especially in the build farm) since garbage
  may become live again.

  The code could easily be modified to accept other criteria for
  ordering garbage by changing the comparison operator used by the
  priority queue in collectGarbage().
2008-09-17 10:02:55 +00:00
Eelco Dolstra
7933cdc6dc * When writing the user environment manifest, filter out non-string
attributes from the meta attribute.  Not doing so caused nix-env to
  barf on the "psi" package, which has a meta.function attribute,
  the textual serialisation of which causes a gigantic string to be
  produced --- so big that it causes nix-env to run out of memory.

  Note however that "meta" really only should contain strings.  
  meta.function should be passthru.function.
2008-09-02 09:21:38 +00:00
Eelco Dolstra
7718b19389 * Explicitly set PWD to prevent problems with chroot builds. In
particular, dietlibc cannot figure out the cwd because the inode of
  the current directory doesn't appear in .. (because getdents returns
  the inode of the mount point).
2008-08-27 17:20:25 +00:00
Eelco Dolstra
9cc0da8453 * Create a /tmp with 1777 permission in the chroot. Some builders
need a writable /tmp (they don't respect $TMPDIR).
2008-08-27 16:03:03 +00:00
Eelco Dolstra
99dc3e613a * Require that __overrides is defined as a non-recursive attribute
(which means it can only be defined via "inherit"), otherwise we get
  scoping bugs, since __overrides can't be recursive (or at least, it
  would be hard).
2008-08-26 14:05:59 +00:00
Eelco Dolstra
d06be428f6 * Disable chroot builds for fixed-output derivations so that we don't
need /etc in the chroot (in particular, /etc/resolv.conf for
  fetchurl).  Not having /etc/resolv.conf in the chroot is a good
  thing, since we don't want normal derivations to download files.
2008-08-25 15:49:22 +00:00
Eelco Dolstra
abec1c0004 * Evaluate attributes in sorted order for better determinism. 2008-08-25 14:31:29 +00:00
Eelco Dolstra
c4f1c2114b * Minor simplification. 2008-08-25 14:15:56 +00:00
Eelco Dolstra
49829da8b4 * Doh. 2008-08-25 13:32:27 +00:00
Eelco Dolstra
b428adc267 * Strip off the `.nix' suffix from the attribute name for files in
~/.nix-defexpr,  otherwise the attribute cannot be selected with the
  `-A' option.  Useful if you want to stick a Nix expression directly
  in ~/.nix-defexpr.
2008-08-25 13:31:57 +00:00
Michael Raskin
b7ff182b6e Fixing an obvious typo in override code. I do not know whether it works correctly after the change, but at least it ca nbe compiled now. 2008-08-14 22:01:43 +00:00
Eelco Dolstra
ca07f3e370 * Another experimental feature: a way to truly override attributes in
a rec.  This will be very useful to allow end-user customisation of
  all-packages.nix, for instance globally overriding GCC or some other
  dependency.  The // operator doesn't cut it: you could replace the
  "gcc" attribute, but all other attributes would continue to
  reference the original value due to the substitution semantics of
  rec.

  The syntax is a bit hacky but this is to allow backwards
  compatibility.
2008-08-14 16:59:37 +00:00
Eelco Dolstra
9279174dde * Added an experimental feature suggested by Andres: ellipses ("...")
in attribute set pattern matches.  This allows defining a function
  that takes *at least* the listed attributes, while ignoring
  additional attributes.  For instance,

    {stdenv, fetchurl, fuse, ...}:
    
    stdenv.mkDerivation {
      ...
    };
    
  defines a function that requires an attribute set that contains the 
  specified attributes but ignores others.  The main advantage is that
  we can then write in all-packages.nix

    aefs = import ../bla/aefs pkgs;

  instead of

    aefs = import ../bla/aefs {
      inherit stdenv fetchurl fuse;
    };

  This saves a lot of typing (not to mention not having to update
  all-packages.nix with purely mechanical changes).  It saves as much
  typing as the "args: with args;" style, but has the advantage that
  the function arguments are properly declared (not implicit in what
  the body of the "with" uses).
2008-08-14 14:00:44 +00:00
Eelco Dolstra
db4f4a8425 * Backward compatibility check to prevent nixos-rebuild from barfing
when upgrading Nix.
2008-08-14 13:02:19 +00:00
Eelco Dolstra
1b962fc720 * @-patterns as in Haskell. For instance, in a function definition
f = args @ {x, y, z}: ...;

  `args' refers to the argument as a whole, which is further
  pattern-matched against the attribute set pattern {x, y, z}.
2008-08-14 12:53:29 +00:00
Eelco Dolstra
e818838412 * "pattern" non-terminal. 2008-08-14 10:14:34 +00:00
Eelco Dolstra
efe4b690ae * Refactoring: combine functions that take an attribute set and
functions that take a single argument (plain lambdas) into one AST
  node (Function) that contains a Pattern node describing the
  arguments.  Current patterns are single lazy arguments (VarPat) and
  matching against an attribute set (AttrsPat).

  This refactoring allows other kinds of patterns to be added easily,
  such as Haskell-style @-patterns, or list pattern matching.
2008-08-14 10:04:22 +00:00
Eelco Dolstra
c03b729319 * Increase the sleep periods a bit to make the test less likely to
fail on slow machines.  Of course it would be better if this test
  wasn't timing dependent...
2008-08-14 09:26:30 +00:00
Eelco Dolstra
5664b6d7ba * Removed the "valid values" feature. Nobody uses it anyway. 2008-08-11 13:36:40 +00:00
Eelco Dolstra
98b07466fb * Better error checking of the data from the substituters. 2008-08-05 10:57:53 +00:00
Eelco Dolstra
339c142009 * Use optimistic profile locking for nix-env operations like `-i' and
`-u'.  Instead of acquiring an exclusive lock on the profile for the
  entire duration of the operation, we just perform the operation
  optimistically (without an exclusive lock), and check at the end
  whether the profile changed while we were busy (i.e., the symlink
  target changed).  If so, the operation is restarted.  Restarting is
  generally cheap, since the build results are still in the Nix store.
  Most of the time, only the user environment has to be rebuilt.
2008-08-04 16:21:45 +00:00
Eelco Dolstra
001b3f06ec * `nix-env --set': support --dry-run. 2008-08-04 14:58:50 +00:00
Eelco Dolstra
a1d310b6b5 * `nix-store --realise': print what paths will be built/downloaded,
just like nix-env.
* `nix-store --realise': --dry-run option.
2008-08-04 13:44:46 +00:00
Eelco Dolstra
5adbb0aabe * build.cc: only use a substituter if it returns info for a path. 2008-08-04 13:15:35 +00:00
Eelco Dolstra
5b1052663a * Always show what paths we're going to build/download (as in
--dry-run).  Maybe there should be an option to turn this on/off?
2008-08-04 13:11:09 +00:00
Eelco Dolstra
c4f98941ed * nix-env --dry-run: show the total size of the substituter
downloads.
2008-08-04 12:29:04 +00:00
Eelco Dolstra
03427e76f1 * querySubstitutablePathInfo: work properly when run via the daemon.
* --dry-run: print the paths that we don't know how to build/substitute.
2008-08-04 11:44:50 +00:00
Eelco Dolstra
3c92ea399d * Make nix-env --dry-run print the paths to be substituted correctly
again.  (After the previous substituter mechanism refactoring I
  didn't update the code that obtains the references of substitutable
  paths.)  This required some refactoring: the substituter programs
  are now kept running and receive/respond to info requests via
  stdin/stdout.
2008-08-02 12:54:35 +00:00
Eelco Dolstra
fc691e1cbd * Print a better error message when a non-derivation attribute set is
coerced to a string.
2008-07-24 14:52:25 +00:00
Eelco Dolstra
989176c56e * Allow read-only access to the store (e.g., non-root users on NixOS
can do operations like "nix-store -qR <path>" even without the Nix
  daemon).
2008-07-18 15:34:46 +00:00
Eelco Dolstra
8bc591a6f0 * Use the copy-from-other-stores substituter by default. Of course,
it only does something if $NIX_OTHER_STORES (not really a good
  name...) is set.
* Do globbing on the elements of $NIX_OTHER_STORES.  E.g. you could
  set it to /mnts/*/nix or something.
* Install substituters in libexec/nix/substituters.
2008-07-18 13:05:10 +00:00
Eelco Dolstra
7cd88b1dec * Generalised the dependencyClosure primop to builtins.genericClosure,
which is hopefully more useful.
* New primops: length, mul, div.
2008-07-11 13:29:04 +00:00
Eelco Dolstra
d567baabbd * Export the nix-env derivation name parsing and version comparison
logic through the `parseDrvName' and `compareVersions' primops.
  This will allow expressions to easily check whether some dependency
  is a specific needed version or falls in some version range.  See
  tests/lang/eval-okay-versions.nix for examples.
2008-07-01 10:10:32 +00:00
Eelco Dolstra
6c8641a542 2008-06-18 19:17:05 +00:00
Eelco Dolstra
5af84139a8 * --max-freed: support values >= 4 GB. 2008-06-18 15:20:33 +00:00
Eelco Dolstra
d3aa183beb * Garbage collector: option `--max-freed' to stop after at least N
bytes have been freed, `--max-links' to stop when the Nix store
  directory has fewer than N hard links (the latter being important
  for very large Nix stores on filesystems with a 32000 subdirectories
  limit).
2008-06-18 14:20:16 +00:00
Eelco Dolstra
a8f3b02092 * `nix-store --optimise': handle files with >= 32000 hard links.
(There can easily be more than 32000 occurrences of the empty file.)
2008-06-18 14:13:00 +00:00
Eelco Dolstra
a72709afd8 * Some refactoring: put the GC options / results in separate structs.
* The garbage collector now also prints the number of blocks freed.
2008-06-18 09:34:17 +00:00
Eelco Dolstra
f351834f77 * nix-worker: clean up the temporary root for the worker processes
in /nix/var/nix/temproots.
2008-06-14 16:03:02 +00:00
Eelco Dolstra
94fd46fa1c * Note. 2008-06-14 16:02:31 +00:00
Eelco Dolstra
826b271d9a * Garbage collector: don't do a complete topological sort of the Nix
store under the reference relation, since that means that the
  garbage collector will need a long time to start deleting paths.
  Instead just delete the referrers of a path first.
2008-06-13 18:25:24 +00:00
Eelco Dolstra
30c9f909b2 * Print some progress info during the early GC stages. 2008-06-13 17:21:20 +00:00
Eelco Dolstra
b0e92f6d47 * Merged the no-bdb branch (-r10900:HEAD
https://svn.nixos.org/repos/nix/nix/branches/no-bdb).
2008-06-09 13:52:45 +00:00
Sander van der Burg
c41a3ec3a9 First attempt to update Nix SDF grammar to match the actual bison grammar 2008-06-04 14:36:46 +00:00
Eelco Dolstra
bd955e15e1 * GCC 4.3.0 (Fedora 9) compatibility fixes. Reported by Gour and
Armijn Hemel.
2008-05-21 11:17:31 +00:00
Eelco Dolstra
72034ab35d * sockaddr_un doesn't allow path names of more than 108 characters.
This isn't usually a problem, except that it causes tests to fail
  when performed in a directory with a very long path name.  So chdir
  to the socket directory and use a relative path name.
2008-04-09 05:57:01 +00:00
Eelco Dolstra
329025253d * Use /tmp/nix-build-<drvpath>-<counter> instead of
/tmp/nix-<pid>-<counter> for temporary build directories.  This
  increases purity a bit: many packages store the temporary build path
  in their output, causing (generally unimportant) binary differences.
2008-03-27 13:45:17 +00:00
Eelco Dolstra
98968fbb63 * Disable the don't-run-as-root sanity check because it breaks RPM
builds (which are done as root...).
2008-03-20 18:15:20 +00:00
Eelco Dolstra
2f1e2cf632 * Note that the SDF grammar isn't used. 2008-03-20 14:59:33 +00:00
Eelco Dolstra
f789ea1d09 * A transaction is unnecessary since the path cannot be
garbage-collected (it's a temporary root).
2008-03-01 21:05:33 +00:00
Eelco Dolstra
0a84137c45 * checkVarDefs: don't check in closed terms, which don't have
undefined variables by definition.  This matters for the
  implementation of "with", which does a call to checkVarDefs to see
  if the body of the with has no undefined variables.  (It can't be
  checked at parse time because you don't know which variables are in
  the "with" attribute set.)  If we check closed terms, then we check
  not just the with body but also the substituted terms, which are
  typically very large.  This is the cause of the poor nix-env
  performance on Nixpkgs lately.  It didn't happen earlier because
  "with" wasn't used very often in the past.

  This fix improves nix-env performance roughly 60x on current Nixpkgs.
  nix-env -qa is down from 29.3s to 0.5s on my laptop, and nix-env -qa
  --out-path is down from 229s to 3.39s.  Not bad for a 1-line fix :-)
2008-02-21 12:01:24 +00:00
Eelco Dolstra
0ed89c569f * Fix the parsing of
''
      '${foo}'
    ''
    
  where the antiquote should work as expected, instead of giving the
  string "'${foo}'".
2008-02-05 13:38:07 +00:00
Eelco Dolstra
66c51dc215 * nix-store --dump-db / --load-db to dump/load the Nix DB.
* nix-store --register-validity: option to supply the content hash of
  each path.
* Removed compatibility with Nix <= 0.7 stores.
2008-01-29 18:17:36 +00:00
Michael Raskin
5b5a3af983 Probably fixed __exprToString 2008-01-20 20:44:03 +00:00
Michael Raskin
5eb5c23447 Fixed exportBuildReferenceGraph 2008-01-15 04:32:08 +00:00
Eelco Dolstra
7d0f6aed59 * New primop `unsafeDiscardStringContext' to get rid of string
contexts.  Needed to prevent unnecessary dependencies when building
  the NixOS manual.
2008-01-04 14:22:49 +00:00
Eelco Dolstra
e0ca671491 * More documentation. 2007-12-31 02:52:17 +00:00
Eelco Dolstra
dedd62dd0c * More release notes. 2007-12-31 00:08:09 +00:00
Michael Raskin
2196cb67c5 Added a way to include derivation with its outputs and dependencies - exportBuildReferenceGraph 2007-12-30 09:30:56 +00:00
Eelco Dolstra
df303666bc * Use strsignal if available to give better error messages for
builders that fail due to a signal.
2007-12-14 14:49:35 +00:00
Eelco Dolstra
30e4653783 * Mac OS X 10.5 compatibility. 2007-12-13 22:06:19 +00:00
Michael Raskin
3a2833daff corrected --help message for nix-store a bit 2007-12-10 22:16:47 +00:00
Eelco Dolstra
b42ef9c054 * Syntax to escape '', ${. 2007-12-06 10:20:58 +00:00
Eelco Dolstra
6d6c68c0d2 * Added a new kind of multi-line string literal delimited by two
single quotes.  Example (from NixOS):

    job = ''
      start on network-interfaces

      start script

        rm -f /var/run/opengl-driver
        ${if videoDriver == "nvidia"        
          then "ln -sf ${nvidiaDrivers} /var/run/opengl-driver"
          else if cfg.driSupport
          then "ln -sf ${mesa} /var/run/opengl-driver"
          else ""
        }

        rm -f /var/log/slim.log

      end script
    '';

  This style has two big advantages:

  - \, ' and " aren't special, only '' and ${.  So you get a lot less
    escaping in shell scripts / configuration files in Nixpkgs/NixOS.
    The delimiter '' is rare in scripts (and can usually be written as
    "").  ${ is also fairly rare.

    Other delimiters such as <<...>>, {{...}} and <|...|> were also
    considered but this one appears to have the fewest drawbacks
    (thanks Martin).

  - Indentation is intelligently stripped so that multi-line strings
    can follow the nesting structure of the containing Nix
    expression.  E.g. in the example above 6 spaces are stripped from
    the start of each line.  This prevents unnecessary indentation in
    generated files (which sometimes even breaks things).

  See tests/lang/eval-okay-ind-string.nix for some examples.
2007-11-30 16:48:45 +00:00
Eelco Dolstra
633518628f * nix-env -e: support uninstalling by path, so that one can say
$ nix-env -e $(which firefox)

  or

    $ nix-env -e /nix/store/nywzlygrkfcgz7dfmhm5xixlx1l0m60v-pan-0.132

* nix-env -i: if an argument contains a slash anywhere, treat it as a
  path and follow it through symlinks into the Nix store.  This allows
  things like

    $ nix-build -A firefox
    $ nix-env -i ./result

* nix-env -q/-i/-e: don't complain when the `*' selector doesn't match
  anything.  In particular, `nix-env -q \*' doesn't fail anymore on an
  empty profile.
2007-11-29 16:18:24 +00:00
Eelco Dolstra
06f95dd07c * New primop `readFile' to get the contents of a file as a string. 2007-11-21 13:49:59 +00:00
Eelco Dolstra
c370755583 * Flag `--no-build-hook' to disable distributed builds.
* queryDeriver in daemon mode: don't barf if the other side returns an
  empty string (which means there is no deriver).
2007-11-16 16:15:26 +00:00
Eelco Dolstra
c05783ad67 * nix-store --import: add a flag --require-signature. 2007-11-16 16:10:27 +00:00
Eelco Dolstra
149e273023 * Don't need gc.hh. 2007-11-15 15:07:27 +00:00
Michael Raskin
6fc60e2060 Added possibility to specify garbage collection levels for store paths; so packages can share intermediate results of compilation and GC will collect it automatically while never touching tarballs, for example. 2007-11-15 03:47:12 +00:00
Eelco Dolstra
fa44e401a8 * Documented multi-user Nix. 2007-10-31 18:01:56 +00:00
Eelco Dolstra
bfea7b1f35 * Make the --prebuilt-only' / -b' option work not just for queries
but installations/upgrades as well.  So `nix-env -ub \*' will
  upgrade only those packages for which a substitute is available (or
  to be precise, it will upgrade each package to the highest version
  for which a substitute is available).
2007-10-29 14:31:45 +00:00
Eelco Dolstra
42d80d1170 * On FreeBSD, sys/mount.h needs sys/param.h. 2007-10-29 10:03:07 +00:00
Eelco Dolstra
d91cd30563 * Detect whether chroot / bind-mount support is available. 2007-10-27 16:51:55 +00:00
Eelco Dolstra
dc6f373842 * Delete the chroot directory automatically.
* Removed some debug messages.
2007-10-27 16:06:38 +00:00
Eelco Dolstra
9397cd30c8 * Support for doing builds in a chroot under Linux. The builder is
executed in a chroot that contains just the Nix store, the temporary
  build directory, and a configurable set of additional directories
  (/dev and /proc by default).  This allows a bit more purity
  enforcement: hidden build-time dependencies on directories such as
  /usr or /nix/var/nix/profiles are no longer possible.  As an added
  benefit, accidental network downloads (cf. NIXPKGS-52) are prevented
  as well (because files such as /etc/resolv.conf are not available in
  the chroot).

  However the usefulness of chroots is diminished by the fact that
  many builders depend on /bin/sh, so you need /bin in the list of
  additional directories.  (And then on non-NixOS you need /lib as
  well...)
2007-10-27 00:46:59 +00:00
Eelco Dolstra
0b4ed64d29 * "trace" primop: write the trace to standard error. 2007-10-26 18:25:50 +00:00
Eelco Dolstra
f90d85107e 2007-10-22 12:05:30 +00:00
Eelco Dolstra
14e47e9c2c * Doh. 2007-10-10 14:58:00 +00:00
Eelco Dolstra
d04291cfab * Doh! Don't change the permissions on /nix/store. 2007-10-10 13:52:38 +00:00
Eelco Dolstra
315183f194 * nix-store --optimise: flag "--dry-run" to just query what the disk
savings would be.
2007-10-10 13:43:04 +00:00
Eelco Dolstra
a8629de827 * New command `nix-store --optimise' to reduce Nix store disk space
usage by finding identical files in the store and hard-linking them
  to each other.  It typically reduces the size of the store by
  something like 25-35%.  This is what the optimise-store.pl script
  did, but the new command is faster and more correct (it's safe wrt
  garbage collection and concurrent builds).
2007-10-09 22:14:27 +00:00
Eelco Dolstra
27a0662828 * listToAttrs: the list now should consist of {name, value} attribute
sets instead of {attr, value}.  "name" is better than "attr" because
  the *combination* of the two forms the attribute.
2007-10-09 12:51:25 +00:00
Eelco Dolstra
27f57c9018 * Ignore dangling symlinks in ~/.nix-defexpr. 2007-09-18 14:01:14 +00:00
Eelco Dolstra
f3441e6122 * Pass various options to the worker so that flags like -K or -j work
in multi-user Nix (NIX-72).
* Client/worker: exchange a protocol version number for future
  compatibility.
2007-09-18 09:11:20 +00:00
Eelco Dolstra
c8ea8a09b8 * nix-env -qa: make the "-A" flag do the expected thing, namely follow
the given attribute path (just as -A does with other option)
  (NIX-83).  So you can now say

  $ nix-env -qa -A nixpkgs_unstable.gnome \*
  atk-1.12.4
  esound-0.2.36
  ...

  to see the packages in the "gnome" attribute in Nixpkgs.

  To *print* the attribute path, you should now use "--attr-path" /
  "-P" (running out of letters...).
2007-09-17 19:24:07 +00:00
Eelco Dolstra
055608227f * nix-env: allow ~/.nix-defexpr to be a directory. If it is, then the
Nix expressions in that directory are combined into an attribute set
  {file1 = import file1; file2 = import file2; ...}, i.e. each Nix
  expression is an attribute with the file name as the attribute
  name.  Also recurses into directories.

* nix-env: removed the "--import" (-I) option which set the
  ~/.nix-defexpr symlink.

* nix-channel: don't use "nix-env --import", instead symlink
  ~/.nix-defexpr/channels.  So finally nix-channel --update doesn't
  override any default Nix expressions but combines with them.

  This means that you can have (say) a local Nixpkgs SVN tree and use
  it as a default for nix-env:

  $ ln -s .../path-to-nixpkgs-tree ~/.nix-defexpr/nixpkgs_svn

  and be subscribed to channels (including Nixpkgs) at the same time.
  (If there is any ambiguity, the -A flag can be used to
  disambiguate, e.g. "nix-env -i -A nixpkgs_svn.pan".)
2007-09-17 16:08:24 +00:00
Eelco Dolstra
0d65fc08e2 * Create the Nix daemon socket in a separate directory
(/nix/var/nix/daemon-socket).  This allows access to the Nix daemon
  to be restricted by setting the mode/ownership on that directory as
  desired, e.g.

    $ chmod 770 /nix/var/nix/daemon-socket
    $ chown root.wheel /nix/var/nix/daemon-socket

  to allow only users in the wheel group to use Nix.

  Setting the ownership on a socket is much trickier, since the socket
  must be deleted and recreated every time the daemon is started
  (which would require additional Nix configuration file directives to
  specify the mode/ownership, and wouldn't support arbitrary ACLs),
  some BSD variants appear to ignore permissions on sockets, and it's
  not clear whether the umask is respected on every platform when
  creating sockets.
2007-08-30 09:50:44 +00:00
Eelco Dolstra
cb1c1004cd * When there are multiple substituters, make sure to release the
lock on the output path after trying each.  Otherwise the
  pathIsLockedByMe() test gets confused.
2007-08-28 16:22:08 +00:00
Eelco Dolstra
c970b28ba0 * Fix a race condition with parallel builds where multiple
fixed-output derivations or substitutions try to build the same
  store path at the same time.  Locking generally catches this, but
  not between multiple goals in the same process.  This happened
  especially often (actually, only) in the build farm with fetchurl
  downloads of the same file being executed on multiple machines and
  then copied back to the main machine where they would clobber each
  other (NIXBF-13).

  Solution: if a goal notices that the output path is already locked,
  then go to sleep until another goal finishes (hopefully the one
  locking the path) and try again.
2007-08-28 11:36:17 +00:00
Eelco Dolstra
e0d7e47862 * PathLocks::lockPaths: don't allow reacquiring a lock we already
hold.
2007-08-28 09:39:03 +00:00
Marc Weber
2629998e91 primop functions listToAttrs (+test), __isAttrs, __trace added
new configuration style proposal in lib/default-unstable.nix
2007-08-18 22:12:00 +00:00
Eelco Dolstra
3757ee589f * Bump the Nix database schema version number; delete the substitutes
table.
2007-08-13 11:37:39 +00:00
Eelco Dolstra
59afc1a15c * Backwards compatibility. 2007-08-13 11:15:13 +00:00
Eelco Dolstra
9e975458b4 * Get rid of the substitutes database table (NIX-47). Instead, if we
need any info on substitutable paths, we just call the substituters
  (such as download-using-manifests.pl) directly.  This means that
  it's no longer necessary for nix-pull to register substitutes or for
  nix-channel to clear them, which makes those operations much faster
  (NIX-95).  Also, we don't have to worry about keeping nix-pull
  manifests (in /nix/var/nix/manifests) and the database in sync with
  each other.

  The downside is that there is some overhead in calling an external
  program to get the substitutes info.  For instance, "nix-env -qas"
  takes a bit longer.

  Abolishing the substitutes table also makes the logic in
  local-store.cc simpler, as we don't need to store info for invalid
  paths.  On the downside, you cannot do things like "nix-store -qR"
  on a substitutable but invalid path (but nobody did that anyway).

* Never catch interrupts (the Interrupted exception).
2007-08-12 00:29:28 +00:00
Eelco Dolstra
d71cc503a6 * Don't allocate input files on the stack. 2007-08-07 15:00:13 +00:00
Eelco Dolstra
4cad125e08 * Optionally warn about packages that give an assertion failure. 2007-08-06 16:08:09 +00:00
Eelco Dolstra
6da6fbfbe9 * Properly keep packages during upgrades. 2007-07-05 20:33:04 +00:00
Eelco Dolstra
6d1a1191b0 * Support queryDeriver() in multi-user installations. 2007-06-12 16:53:44 +00:00
Eelco Dolstra
9bff7ad728 * Check against creation of GC roots in the store. Those roots don't
work, because findRoots() stops when it encounters a symlink to the
  store.  And of course the store is supposed to be read-only.
2007-06-11 11:36:22 +00:00
Eelco Dolstra
bddc83a148 * New builtin function "isFunction". You're not supposed to use it
;-)
* Channels: fix channels that are plain lists of derivations (like
  strategoxt-unstable) instead  of functions (like nixpkgs-unstable).
  This fixes the error message "error: the left-hand side of the
  function call is neither a function nor a primop (built-in
  operation) but a list".
2007-05-16 16:17:04 +00:00
Eelco Dolstra
ca00aa1171 * Allow empty argument lists in function definitions (e.g., `{}:
bla').  Also allow trailing commas (`{x, y,}: ...') as a unintented
  consequence.  Hopefully the reduce/reduce conflict won't cause any
  problems.
2007-05-15 12:14:37 +00:00
Eelco Dolstra
7046c35843 * Typo (reported by Marc Weber). 2007-05-14 12:16:41 +00:00
Eelco Dolstra
e5da9c8803 * Create the database directory if it doesn't exist. 2007-05-07 16:59:19 +00:00
Eelco Dolstra
93aefd9fc0 * Give unpacked channels more sensible names than 0, 1, ... They now
get the basename of the channel URL (e.g., nixpkgs-unstable).  The
  top-level Nix expression of the channel is now an attribute set, the
  attributes of which are the individual channels (e.g.,
  {nixpkgs_unstable = ...; strategoxt_unstable = ...}).  This makes
  attribute paths ("nix-env -qaA" and "nix-env -iA") more sensible,
  e.g., "nix-env -iA nixpkgs_unstable.subversion".
2007-05-01 23:16:38 +00:00
Eelco Dolstra
a9d15d4f43 * nix-env -i: instead of breaking package ties by version, break them
by priority and version install.  That is, if there are multiple
  packages with the same name, then pick the package with the highest
  priority, and only use the version if there are multiple packages
  with the same priority.

  This makes it possible to mark specific versions/variant in Nixpkgs
  more or less desirable than others.  A typical example would be a
  beta version of some package (e.g., "gcc-4.2.0rc1") which should not
  be installed even though it is the highest version, except when it
  is explicitly selected (e.g., "nix-env -i gcc-4.2.0rc1").

* Idem for nix-env -u, only the semantics are a bit trickier since we
  also need to take into account the priority of the currently
  installed package (we never upgrade to a lower priority, unless
  --always is given).
2007-05-01 20:33:18 +00:00
Eelco Dolstra
cbfac2fdcc * Set a terminate() handler to ensure that we leave the BDB
environment cleanly even when an exception is thrown from a
  destructor.  We still crash, but we don't take all other Nix
  processes with us.
2007-05-01 15:16:17 +00:00
Eelco Dolstra
644946feed * Make --verify more interruptable. 2007-05-01 13:21:05 +00:00
Eelco Dolstra
bd1f66453a * `nix-env -q --xml --meta' to show all meta attributes. 2007-05-01 11:30:52 +00:00
Eelco Dolstra
e20f0da22c * Doh. 2007-04-30 18:41:27 +00:00
Eelco Dolstra
89c1d2b202 * Package flag "keep" that prevents a package from being removed from
a user environment by an install or upgrade action.  This is
  particularly useful if you have a version installed that you don't
  want to upgrade (e.g., because the newer versions are broken).
  Example:

    $ nix-env -u zapping --dry-run
    (dry run; not doing anything)
    upgrading `zapping-0.9.6' to `zapping-0.10cvs6'

    $ nix-env --set-flag keep true zapping

    $ nix-env -u zapping --dry-run
    (dry run; not doing anything)

  However, "-e" will still uninstall the package.  (Maybe we should
  require the keep flag to be explicitly set to false before it can be
  uninstalled.)
2007-04-27 23:48:14 +00:00
Eelco Dolstra
b7f0f65c19 * nix-env -q now has a flag --prebuilt-only (-b<) that causes nix-env
to show only those derivations whose output is already in the Nix
  store or that can be substituted (i.e., downloaded from somewhere).
  In other words, it shows the packages that can be installed “quickly”,
  i.e., don’t need to be built from source.
2007-04-26 14:20:31 +00:00
Eelco Dolstra
5f2492eaec * New primop "throw <string>" to throw an error. This is like abort,
only thrown errors are caught by the top-level derivation evaluation
  in nix-env -qa / -i.
2007-04-16 15:03:19 +00:00
Eelco Dolstra
0a8eeea9d8 * Remove a warning. 2007-04-16 14:45:25 +00:00
Eelco Dolstra
ae7990cc88 * Work around a bug in Apple's GCC preprocessor. 2007-03-30 13:24:35 +00:00
Eelco Dolstra
17b506c0c7 * Handle ECONNRESET from the client. Also, don't abort() if there are
unexpected conditions in the SIGPOLL handler, since that messes up
  the Berkeley DB environment (which a client must never be able to
  trigger).
2007-03-28 15:46:21 +00:00
Eelco Dolstra
803cb6e3b9 * Override the setuid helper using NIX_SETUID_HELPER. 2007-03-20 22:04:25 +00:00
Eelco Dolstra
8ab229ddf2 * Terminate build hooks and substitutes with a TERM signal, not a KILL
signal.  This is necessary because those processes may have joined
  the BDB environment, so they have to be given a chance to clean up.
  (NIX-85)
2007-03-19 12:48:45 +00:00
Eelco Dolstra
b2b6cf3fc8 * Undocumented option `gc-check-reachability' to allow reachability
checking to be turned off on machines with way too many roots.
2007-03-19 09:16:47 +00:00
Eelco Dolstra
917e06bf63 * Delete the output paths before invoking the build hook. 2007-03-07 15:53:11 +00:00
Eelco Dolstra
db1973d012 * Look for the openssl program at compile time. If not found, call
openssl through $PATH at runtime.
2007-03-01 13:30:46 +00:00
Eelco Dolstra
b4a040e52b * Don't check the signature unless we have to. 2007-03-01 12:30:24 +00:00
Eelco Dolstra
2ea3bebc23 * Doh! The deriver can be empty. 2007-02-27 23:18:57 +00:00
Eelco Dolstra
044b6482c1 * Greatly reduced the amount of stack space used by the Nix expression
evaluator.  This was important because the NixOS expressions started
  to hit 2 MB default stack size on Linux.

  GCC is really dumb about stack space: it just adds up all the local
  variables and temporaries of every scope into one huge stack frame.
  This is really bad for deeply recursive functions.  For instance,
  every `throw Error(format("error message"))' causes a format object
  of a few hundred bytes to be allocated on the stack.  As a result,
  every recursive call to evalExpr2() consumed 4680 bytes.  By
  splitting evalExpr2() and by moving the exception-throwing code out
  of the main functions, evalExpr2() now only consumes 40 bytes.
  Similar for evalExpr().
2007-02-27 19:10:45 +00:00
Eelco Dolstra
adce01a8d0 * When NIX_SHOW_STATS=1, show the amount of stack space consumed by
the Nix expression evaluator.
2007-02-27 17:28:51 +00:00
Eelco Dolstra
ddde8e2f32 * Handle EINTR in select(). 2007-02-22 18:15:29 +00:00
Eelco Dolstra
7f6161ab3a * Flush cout to show progress. 2007-02-21 23:08:55 +00:00
Eelco Dolstra
0db450024d * Export/import many paths in one go. 2007-02-21 23:00:31 +00:00
Eelco Dolstra
9da367b7d5 * `nix-store -qR' and friends: print the paths sorted topologically
under the references relation.  This is useful for commands that
  want to copy paths to another Nix store in the right order.
2007-02-21 22:45:10 +00:00
Eelco Dolstra
881feb9698 * Flag --print-invalid' in nix-store --check-validity' to print out
which paths specified on the command line are invalid (i.e., don't
  barf when encountering an invalid path, just print it).  This is
  useful for build-remote.pl to figure out which paths need to be
  copied to a remote machine.  (Currently we use rsync, but that's
  rather inefficient.)
2007-02-21 17:57:59 +00:00
Eelco Dolstra
65f195f4c7 * Check that the file containing the secret key is secret. 2007-02-21 17:51:10 +00:00
Eelco Dolstra
bdadb98de8 * `nix-store --import' now also works in remote mode. The worker
always requires a signature on the archive.  This is to ensure that
  unprivileged users cannot add Trojan horses to the Nix store.
2007-02-21 17:34:02 +00:00
Eelco Dolstra
0f5da8a83c * Support exportPath() in remote mode. 2007-02-21 16:34:00 +00:00
Eelco Dolstra
dc7d594776 * importPath(): set the deriver.
* exportPath(): lock the path, use a transaction.
2007-02-21 16:23:25 +00:00
Eelco Dolstra
43c4d18c6a * nix-store --import': import an archive created by nix-store
--export' into the Nix store, and optionally check the cryptographic
  signatures against /nix/etc/nix/signing-key.pub.  (TODO: verify
  against a set of public keys.)
2007-02-21 15:45:32 +00:00
Eelco Dolstra
46e0919ced * `nix-store --export --sign': sign the Nix archive using the RSA key
in /nix/etc/nix/signing-key.sec
2007-02-21 14:31:42 +00:00
Eelco Dolstra
b824a1daee * Start of `nix-store --export' operation for serialising a store
path.  This is like `nix-store --dump', only it also dumps the
  meta-information of the store path (references, deriver).  Will add
  a `--sign' flag later to add a cryptographic signature, which we
  will use for exchanging store paths between build farm machines in a
  secure manner.
2007-02-20 23:17:20 +00:00
Eelco Dolstra
46605fb4f5 * Fix 64-bit compiler warnings. 2007-02-06 20:03:53 +00:00
Eelco Dolstra
451dbf687f * nix-env now maintains meta info (from the `meta' derivation
attribute) about installed packages in user environments.  Thus, an
  operation like `nix-env -q --description' shows useful information
  not only on available packages but also on installed packages.

* nix-env now passes the entire manifest as an argument to the Nix
  expression of the user environment builder (not just a list of
  paths), so that in particular the user environment builder has
  access to the meta attributes.
  
* New operation `--set-flag' in nix-env to change meta info of
  installed packages.  This will be useful to pass per-package
  policies to the user environment builder (e.g., how to resolve
  collision or whether to disable a package (NIX-80)) or upgrade
  policies in nix-env (e.g., that a package should be "masked", that
  is, left untouched by upgrade actions).  Example:

  $ nix-env --set-flag enabled false ghc-6.4
2007-02-02 01:52:42 +00:00
Eelco Dolstra
f52de527c7 * Doh! 2007-01-29 15:55:49 +00:00
Eelco Dolstra
b618fa6eb6 * computeStorePathForText: take the references into account when
computing the store path (NIX-77).  This is an important security
  property in multi-user Nix stores.

  Note that this changes the store paths of derivations (since the
  derivation aterms are added using addTextToStore), but not most
  outputs (unless they use builtins.toFile).
2007-01-29 15:51:37 +00:00
Eelco Dolstra
c558b1583c * Don't capitalise the primop functions. 2007-01-29 15:15:37 +00:00
Eelco Dolstra
18e6096105 * Organise primops.cc a bit better. 2007-01-29 15:11:32 +00:00
Eelco Dolstra
7349bd0176 New primitives:
* `sub' to subtract two numbers.
* `stringLength' to get the length of a string.
* `substring' to get a substring of a string.  These should be enough
  to allow most string operations to be expressed.
2007-01-29 14:23:09 +00:00
Eelco Dolstra
7dedbd896a * filterSource: pass strings to the predicate function instead of
paths.  Paths can have unexpected semantics.
2007-01-29 13:32:50 +00:00
Eelco Dolstra
84a84afb0e * Nasty: Glibc clears the TMPDIR environment variable in setuid
programs, so if a builder uses TMPDIR, then it will fail when
  executed through nix-setuid-helper.  In fact Glibc clears a whole
  bunch of variables (see sysdeps/generic/unsecvars.h in the Glibc
  sources), but only TMPDIR should matter in practice.  As a
  workaround, we reinitialise TMPDIR from NIX_BUILD_TOP.
2007-01-24 13:31:20 +00:00
Eelco Dolstra
fac63d6416 * exportReferencesGraph: work on paths within store paths as well. 2007-01-23 16:57:43 +00:00
Eelco Dolstra
71ceb1c161 * Handle multiple indirect symlinks when loading a Nix expression. 2007-01-15 14:50:25 +00:00
Eelco Dolstra
e4b0666f8e * builtins.filterSource: pass the type of the file ("regular",
"directory", "symlink") as the second argument to the filter
  predicate.
2007-01-15 08:54:51 +00:00
Eelco Dolstra
63f3ce6d9a * `nix-store --verify': revive checking the referrers table. This is
important to get garbage collection to work if there is any
  inconsistency in the database (because the referrer table is used to
  determine whether it is safe to delete a path).
* `nix-store --verify': show some progress.
2007-01-14 17:28:30 +00:00
Eelco Dolstra
8f67b35886 * Make the garbage collector more resilient to certain consistency
errors: in-use paths now cause a warning, not a fatal error.
2007-01-14 16:24:49 +00:00
Eelco Dolstra
e418976107 * Option --argstr for passing string arguments easily. (NIX-75) 2007-01-14 12:32:44 +00:00
Eelco Dolstra
afe23b5f38 * nix-pack-closure: store the top-level store paths in the closure.
* nix-unpack-closure: extract the top-level paths from the closure and
  print them on stdout.  This allows them to be installed, e.g.,
  "nix-env -i $(nix-unpack-closure)".  (NIX-64)
2007-01-13 19:50:42 +00:00
Eelco Dolstra
f25f900045 * Allow multiple --attr / -A arguments in nix-build / nix-instantiate
(NIX-74).
2007-01-13 18:25:30 +00:00
Eelco Dolstra
215505bb46 * Removed chroot support. 2007-01-13 17:54:01 +00:00
Eelco Dolstra
f23dcdd603 * Canonicalise ASTs in `nix-instantiate --eval': remove position
info, sort attribute sets.
2007-01-13 16:17:07 +00:00
Eelco Dolstra
05879db628 * Memoize strict evaluation. 2007-01-13 15:41:54 +00:00
Eelco Dolstra
5011588459 * printTermAsXML: treat derivations specially; emit an element
<derivation outPath=... drvPath=...> attrs </derivation>.  Only emit
  the attributes of any specific derivation only.  This prevents
  exponententially large XML output due to the absense of sharing.
2007-01-13 15:11:10 +00:00
Eelco Dolstra
792878af91 * Make printing an expression as XML interruptible. 2007-01-13 14:48:41 +00:00
Eelco Dolstra
11158028be * Cleanup. 2007-01-13 14:21:49 +00:00
Eelco Dolstra
1073b1780a * Remove debug message. 2006-12-13 14:29:05 +00:00
Eelco Dolstra
a3e6415ba8 * New primop builtins.filterSource, which can be used to filter files
from a source directory.  All files for which a predicate function
  returns true are copied to the store.  Typical example is to leave
  out the .svn directory:

    stdenv.mkDerivation {
      ...
      src = builtins.filterSource
        (path: baseNameOf (toString path) != ".svn")
        ./source-dir;
      # as opposed to
      #   src = ./source-dir;
    }

  This is important because the .svn directory influences the hash in
  a rather unpredictable and variable way.
2006-12-12 23:05:01 +00:00
Eelco Dolstra
b438d37558 * In dumpPath(): pass a function object that allows files to be
selectively in/excluded from the dump.
2006-12-12 21:51:02 +00:00
Eelco Dolstra
7ace29dae7 * New operation `nix-env --set' which sets a user environment to a
single derivation specified by the argument.  This is useful when we
  want to have a profile for a single derivation, such as a server
  configuration.  Then we can just say (e.g.)

  $ nix-env -p /.../server-profile -f server.nix --set -A server

  We can't do queries or upgrades on such a profile, but we can do
  rollbacks.  The advantage over -i is that we don't have to worry
  about other packages having been installed in the profile
  previously; --set gets rid of them.
2006-12-12 19:06:02 +00:00
Eelco Dolstra
1a7e88bbd9 * New built-in function `builtins.attrNames' that returns the
names of the attributes in an attribute set.
2006-12-12 16:14:31 +00:00
Eelco Dolstra
b17677462c * Use lchown() instead of chown() in canonicalisePathMetaData(). This
matters when running as root, since then we don't use the setuid
  helper (which already used lchown()).
  
* Also check for an obscure security problem on platforms that don't
  have lchown.  Then we can't change the ownership of symlinks, which
  doesn't matter *except* when the containing directory is writable by
  the owner (which is the case with the top-level Nix store directory).
2006-12-09 20:02:27 +00:00
Eelco Dolstra
5f681988f2 * Use deletePathWrapped() in more places. 2006-12-09 00:26:24 +00:00
Eelco Dolstra
fa33303146 * Goal cancellation inside the waitForInput() loop needs to be handled
very carefully, since it can invalidate iterators into the
  `children' map.
2006-12-08 18:41:48 +00:00
Eelco Dolstra
06c4929958 * Some refactoring.
* Throw more exceptions as BuildErrors instead of Errors.  This
  matters when --keep-going is turned on.  (A BuildError is caught
  and terminates the goal in question, an Error terminates the
  program.)
2006-12-08 17:26:21 +00:00
Eelco Dolstra
9dbfe242e3 * Kill a build if it has gone for more than a certain number of
seconds without producing output on stdout or stderr (NIX-65).  This
  timeout can be specified using the `--max-silent-time' option or the
  `build-max-silent-time' configuration setting.  The default is
  infinity (0).

* Fix a tricky race condition: if we kill the build user before the
  child has done its setuid() to the build user uid, then it won't be
  killed, and we'll potentially lock up in pid.wait().  So also send a
  conventional kill to the child.
2006-12-08 15:44:00 +00:00
Eelco Dolstra
d3fe6ab024 * Also for convenience, change the ownership of the build output even
in case of failure.
2006-12-08 00:19:50 +00:00
Eelco Dolstra
096194ab29 * Remove ancient terminology. 2006-12-07 23:58:36 +00:00
Eelco Dolstra
6833e8bbe8 * When keeping the temporary build directory (-K), change the owner
back to the Nix account.
2006-12-07 23:27:40 +00:00
Eelco Dolstra
e24d0201c2 * Doh! 2006-12-07 22:07:05 +00:00
Eelco Dolstra
2819eb36a4 * Be less verbose. 2006-12-07 21:43:35 +00:00
Eelco Dolstra
4ca01065c3 * Rename all those main.cc files. 2006-12-07 20:47:30 +00:00
Eelco Dolstra
d03f0d4117 * Check for lchown. 2006-12-07 18:51:11 +00:00
Eelco Dolstra
c3286ec020 * Don't count on the Pid deconstructor to kill the child process,
since if we're running a build user in non-root mode, we can't.  Let
  the setuid helper do it.
2006-12-07 17:52:58 +00:00
Eelco Dolstra
a82d80ddeb * Move setuidCleanup() to libutil. 2006-12-07 16:40:41 +00:00
Eelco Dolstra
f76fdb6d42 * If not running as root, let the setuid helper kill the build user's
processes before and after the build.
2006-12-07 16:33:31 +00:00