Commit graph

1113 commits

Author SHA1 Message Date
Eelco Dolstra
0c478d2f4d * Ignore carriage returns. 2008-11-25 02:30:35 +00:00
Eelco Dolstra
5024bde8f4 * Handle prematurely ended logfiles, i.e. make sure we emit enough
close tags.
2008-11-25 01:06:15 +00:00
Eelco Dolstra
a55113411f * Nix daemon: reload the configuration file after forking (NIX-100). 2008-11-20 12:25:11 +00:00
Eelco Dolstra
aab530e971 * Primop builtins.storePath for declaring a store path as a
dependency.  `storePath /nix/store/bla' gives exactly the same
  result as `toPath /nix/store/bla', except that the former includes
  /nix/store/bla in the dependency context of the string.

  Useful in some generated Nix expressions like nix-push, which now
  finally does the right thing wrt distributed builds.  (Previously
  the path to be packed wasn't an explicit dependency, so it wouldn't
  be copied to the remote machine.)
2008-11-19 23:26:19 +00:00
Eelco Dolstra
e13da525a7 * Files in the info directory starting with "." are temporary files
and don't indicate path validity.
2008-11-19 16:27:07 +00:00
Eelco Dolstra
5d250ad1ea * nix-store --dump-db: be more streamy. 2008-11-19 16:26:34 +00:00
Eelco Dolstra
3f4ed681c2 * Prevent zombies. Previous the SIGCHLD handler only reaped one
zombie at a time, so if multiple children died before the handler
  got to run, some of them would not be cleaned up.
2008-11-14 16:50:01 +00:00
Eelco Dolstra
6fedb7aa0f * Restore SIGPIPE to SIG_DFL when running the builder. This prevents
subtle and often hard-to-reproduce bugs where programs in pipes
  either barf with a "Broken pipe" message or not, depending on the
  exact timing conditions.  This particularly happened in GNU M4 (and
  Bison, which uses M4).
2008-11-14 15:46:45 +00:00
Eelco Dolstra
a519bb0635 * Some somewhat ad hoc mechanism to allow the build farm to monitor
build progress.
2008-11-12 11:08:27 +00:00
Eelco Dolstra
96598e7b06 * Pass the --no-build-output flag to the daemon. 2008-11-11 15:11:10 +00:00
Eelco Dolstra
4166b11a53 * Add /dev/pts to the default nix.conf. 2008-11-11 14:59:20 +00:00
Eelco Dolstra
2b7c839b4e * Typo. 2008-11-11 14:58:37 +00:00
Eelco Dolstra
709b55ee02 * Put the chroots under /nix/var/nix/chroots to reduce the risk of
disasters involving `rm -rf' on bind mounts.  Will try the
  definitive fix (per-process mounts, apparently possible via the
  CLONE_NEWNS flag in clone()) some other time.
2008-10-29 15:34:48 +00:00
Ludovic Courtès
c98ea254dc libstore: Always mount `/dev/pts' individually.
This fixes problems such as Tcl's PTY handling:

  ERROR: The system has no more ptys.  Ask your system administrator to
  create more.
2008-10-16 21:04:32 +00:00
Eelco Dolstra
9d6d50269b * Bug fix for building on some old installations (contributed by Pjotr). 2008-10-16 14:16:03 +00:00
Eelco Dolstra
fa61ee70ee * Fix `--from-profile'. 2008-09-18 09:08:54 +00:00
Eelco Dolstra
f32fef1b07 * GC option `--max-atime' that specifies an upper limit to the last
accessed time of paths that may be deleted.  Anything more recently
  used won't be deleted.  The time is specified in time_t,
  e.g. seconds since 1970-01-01 00:00:00 UTC; use `date +%s' to
  convert to time_t from the command line. 

  Example: to delete everything that hasn't been used in the last two
  months:

  $ nix-store --gc -v --max-atime $(date +%s -d "2 months ago")
2008-09-17 14:52:35 +00:00
Eelco Dolstra
4af2fdba6d * Typo. 2008-09-17 13:00:55 +00:00
Eelco Dolstra
c987061aa4 * Some refactoring. Better output with `-v' for --use-atime. 2008-09-17 12:54:07 +00:00
Eelco Dolstra
77afd97a99 * nix-store --gc / --delete: show how many store paths were deleted. 2008-09-17 12:53:33 +00:00
Eelco Dolstra
7ab68961e4 * Garbage collector: added an option `--use-atime' to delete paths in
order of ascending last access time.  This is useful in conjunction
  with --max-freed or --max-links to prefer deleting non-recently used
  garbage, which is good (especially in the build farm) since garbage
  may become live again.

  The code could easily be modified to accept other criteria for
  ordering garbage by changing the comparison operator used by the
  priority queue in collectGarbage().
2008-09-17 10:02:55 +00:00
Eelco Dolstra
7933cdc6dc * When writing the user environment manifest, filter out non-string
attributes from the meta attribute.  Not doing so caused nix-env to
  barf on the "psi" package, which has a meta.function attribute,
  the textual serialisation of which causes a gigantic string to be
  produced --- so big that it causes nix-env to run out of memory.

  Note however that "meta" really only should contain strings.  
  meta.function should be passthru.function.
2008-09-02 09:21:38 +00:00
Eelco Dolstra
7718b19389 * Explicitly set PWD to prevent problems with chroot builds. In
particular, dietlibc cannot figure out the cwd because the inode of
  the current directory doesn't appear in .. (because getdents returns
  the inode of the mount point).
2008-08-27 17:20:25 +00:00
Eelco Dolstra
9cc0da8453 * Create a /tmp with 1777 permission in the chroot. Some builders
need a writable /tmp (they don't respect $TMPDIR).
2008-08-27 16:03:03 +00:00
Eelco Dolstra
99dc3e613a * Require that __overrides is defined as a non-recursive attribute
(which means it can only be defined via "inherit"), otherwise we get
  scoping bugs, since __overrides can't be recursive (or at least, it
  would be hard).
2008-08-26 14:05:59 +00:00
Eelco Dolstra
d06be428f6 * Disable chroot builds for fixed-output derivations so that we don't
need /etc in the chroot (in particular, /etc/resolv.conf for
  fetchurl).  Not having /etc/resolv.conf in the chroot is a good
  thing, since we don't want normal derivations to download files.
2008-08-25 15:49:22 +00:00
Eelco Dolstra
abec1c0004 * Evaluate attributes in sorted order for better determinism. 2008-08-25 14:31:29 +00:00
Eelco Dolstra
c4f1c2114b * Minor simplification. 2008-08-25 14:15:56 +00:00
Eelco Dolstra
49829da8b4 * Doh. 2008-08-25 13:32:27 +00:00
Eelco Dolstra
b428adc267 * Strip off the `.nix' suffix from the attribute name for files in
~/.nix-defexpr,  otherwise the attribute cannot be selected with the
  `-A' option.  Useful if you want to stick a Nix expression directly
  in ~/.nix-defexpr.
2008-08-25 13:31:57 +00:00
Michael Raskin
b7ff182b6e Fixing an obvious typo in override code. I do not know whether it works correctly after the change, but at least it ca nbe compiled now. 2008-08-14 22:01:43 +00:00
Eelco Dolstra
ca07f3e370 * Another experimental feature: a way to truly override attributes in
a rec.  This will be very useful to allow end-user customisation of
  all-packages.nix, for instance globally overriding GCC or some other
  dependency.  The // operator doesn't cut it: you could replace the
  "gcc" attribute, but all other attributes would continue to
  reference the original value due to the substitution semantics of
  rec.

  The syntax is a bit hacky but this is to allow backwards
  compatibility.
2008-08-14 16:59:37 +00:00
Eelco Dolstra
9279174dde * Added an experimental feature suggested by Andres: ellipses ("...")
in attribute set pattern matches.  This allows defining a function
  that takes *at least* the listed attributes, while ignoring
  additional attributes.  For instance,

    {stdenv, fetchurl, fuse, ...}:
    
    stdenv.mkDerivation {
      ...
    };
    
  defines a function that requires an attribute set that contains the 
  specified attributes but ignores others.  The main advantage is that
  we can then write in all-packages.nix

    aefs = import ../bla/aefs pkgs;

  instead of

    aefs = import ../bla/aefs {
      inherit stdenv fetchurl fuse;
    };

  This saves a lot of typing (not to mention not having to update
  all-packages.nix with purely mechanical changes).  It saves as much
  typing as the "args: with args;" style, but has the advantage that
  the function arguments are properly declared (not implicit in what
  the body of the "with" uses).
2008-08-14 14:00:44 +00:00
Eelco Dolstra
db4f4a8425 * Backward compatibility check to prevent nixos-rebuild from barfing
when upgrading Nix.
2008-08-14 13:02:19 +00:00
Eelco Dolstra
1b962fc720 * @-patterns as in Haskell. For instance, in a function definition
f = args @ {x, y, z}: ...;

  `args' refers to the argument as a whole, which is further
  pattern-matched against the attribute set pattern {x, y, z}.
2008-08-14 12:53:29 +00:00
Eelco Dolstra
e818838412 * "pattern" non-terminal. 2008-08-14 10:14:34 +00:00
Eelco Dolstra
efe4b690ae * Refactoring: combine functions that take an attribute set and
functions that take a single argument (plain lambdas) into one AST
  node (Function) that contains a Pattern node describing the
  arguments.  Current patterns are single lazy arguments (VarPat) and
  matching against an attribute set (AttrsPat).

  This refactoring allows other kinds of patterns to be added easily,
  such as Haskell-style @-patterns, or list pattern matching.
2008-08-14 10:04:22 +00:00
Eelco Dolstra
c03b729319 * Increase the sleep periods a bit to make the test less likely to
fail on slow machines.  Of course it would be better if this test
  wasn't timing dependent...
2008-08-14 09:26:30 +00:00
Eelco Dolstra
5664b6d7ba * Removed the "valid values" feature. Nobody uses it anyway. 2008-08-11 13:36:40 +00:00
Eelco Dolstra
98b07466fb * Better error checking of the data from the substituters. 2008-08-05 10:57:53 +00:00
Eelco Dolstra
339c142009 * Use optimistic profile locking for nix-env operations like `-i' and
`-u'.  Instead of acquiring an exclusive lock on the profile for the
  entire duration of the operation, we just perform the operation
  optimistically (without an exclusive lock), and check at the end
  whether the profile changed while we were busy (i.e., the symlink
  target changed).  If so, the operation is restarted.  Restarting is
  generally cheap, since the build results are still in the Nix store.
  Most of the time, only the user environment has to be rebuilt.
2008-08-04 16:21:45 +00:00
Eelco Dolstra
001b3f06ec * `nix-env --set': support --dry-run. 2008-08-04 14:58:50 +00:00
Eelco Dolstra
a1d310b6b5 * `nix-store --realise': print what paths will be built/downloaded,
just like nix-env.
* `nix-store --realise': --dry-run option.
2008-08-04 13:44:46 +00:00
Eelco Dolstra
5adbb0aabe * build.cc: only use a substituter if it returns info for a path. 2008-08-04 13:15:35 +00:00
Eelco Dolstra
5b1052663a * Always show what paths we're going to build/download (as in
--dry-run).  Maybe there should be an option to turn this on/off?
2008-08-04 13:11:09 +00:00
Eelco Dolstra
c4f98941ed * nix-env --dry-run: show the total size of the substituter
downloads.
2008-08-04 12:29:04 +00:00
Eelco Dolstra
03427e76f1 * querySubstitutablePathInfo: work properly when run via the daemon.
* --dry-run: print the paths that we don't know how to build/substitute.
2008-08-04 11:44:50 +00:00
Eelco Dolstra
3c92ea399d * Make nix-env --dry-run print the paths to be substituted correctly
again.  (After the previous substituter mechanism refactoring I
  didn't update the code that obtains the references of substitutable
  paths.)  This required some refactoring: the substituter programs
  are now kept running and receive/respond to info requests via
  stdin/stdout.
2008-08-02 12:54:35 +00:00
Eelco Dolstra
fc691e1cbd * Print a better error message when a non-derivation attribute set is
coerced to a string.
2008-07-24 14:52:25 +00:00
Eelco Dolstra
989176c56e * Allow read-only access to the store (e.g., non-root users on NixOS
can do operations like "nix-store -qR <path>" even without the Nix
  daemon).
2008-07-18 15:34:46 +00:00
Eelco Dolstra
8bc591a6f0 * Use the copy-from-other-stores substituter by default. Of course,
it only does something if $NIX_OTHER_STORES (not really a good
  name...) is set.
* Do globbing on the elements of $NIX_OTHER_STORES.  E.g. you could
  set it to /mnts/*/nix or something.
* Install substituters in libexec/nix/substituters.
2008-07-18 13:05:10 +00:00
Eelco Dolstra
7cd88b1dec * Generalised the dependencyClosure primop to builtins.genericClosure,
which is hopefully more useful.
* New primops: length, mul, div.
2008-07-11 13:29:04 +00:00
Eelco Dolstra
d567baabbd * Export the nix-env derivation name parsing and version comparison
logic through the `parseDrvName' and `compareVersions' primops.
  This will allow expressions to easily check whether some dependency
  is a specific needed version or falls in some version range.  See
  tests/lang/eval-okay-versions.nix for examples.
2008-07-01 10:10:32 +00:00
Eelco Dolstra
6c8641a542 2008-06-18 19:17:05 +00:00
Eelco Dolstra
5af84139a8 * --max-freed: support values >= 4 GB. 2008-06-18 15:20:33 +00:00
Eelco Dolstra
d3aa183beb * Garbage collector: option `--max-freed' to stop after at least N
bytes have been freed, `--max-links' to stop when the Nix store
  directory has fewer than N hard links (the latter being important
  for very large Nix stores on filesystems with a 32000 subdirectories
  limit).
2008-06-18 14:20:16 +00:00
Eelco Dolstra
a8f3b02092 * `nix-store --optimise': handle files with >= 32000 hard links.
(There can easily be more than 32000 occurrences of the empty file.)
2008-06-18 14:13:00 +00:00
Eelco Dolstra
a72709afd8 * Some refactoring: put the GC options / results in separate structs.
* The garbage collector now also prints the number of blocks freed.
2008-06-18 09:34:17 +00:00
Eelco Dolstra
f351834f77 * nix-worker: clean up the temporary root for the worker processes
in /nix/var/nix/temproots.
2008-06-14 16:03:02 +00:00
Eelco Dolstra
94fd46fa1c * Note. 2008-06-14 16:02:31 +00:00
Eelco Dolstra
826b271d9a * Garbage collector: don't do a complete topological sort of the Nix
store under the reference relation, since that means that the
  garbage collector will need a long time to start deleting paths.
  Instead just delete the referrers of a path first.
2008-06-13 18:25:24 +00:00
Eelco Dolstra
30c9f909b2 * Print some progress info during the early GC stages. 2008-06-13 17:21:20 +00:00
Eelco Dolstra
b0e92f6d47 * Merged the no-bdb branch (-r10900:HEAD
https://svn.nixos.org/repos/nix/nix/branches/no-bdb).
2008-06-09 13:52:45 +00:00
Sander van der Burg
c41a3ec3a9 First attempt to update Nix SDF grammar to match the actual bison grammar 2008-06-04 14:36:46 +00:00
Eelco Dolstra
bd955e15e1 * GCC 4.3.0 (Fedora 9) compatibility fixes. Reported by Gour and
Armijn Hemel.
2008-05-21 11:17:31 +00:00
Eelco Dolstra
72034ab35d * sockaddr_un doesn't allow path names of more than 108 characters.
This isn't usually a problem, except that it causes tests to fail
  when performed in a directory with a very long path name.  So chdir
  to the socket directory and use a relative path name.
2008-04-09 05:57:01 +00:00
Eelco Dolstra
329025253d * Use /tmp/nix-build-<drvpath>-<counter> instead of
/tmp/nix-<pid>-<counter> for temporary build directories.  This
  increases purity a bit: many packages store the temporary build path
  in their output, causing (generally unimportant) binary differences.
2008-03-27 13:45:17 +00:00
Eelco Dolstra
98968fbb63 * Disable the don't-run-as-root sanity check because it breaks RPM
builds (which are done as root...).
2008-03-20 18:15:20 +00:00
Eelco Dolstra
2f1e2cf632 * Note that the SDF grammar isn't used. 2008-03-20 14:59:33 +00:00
Eelco Dolstra
f789ea1d09 * A transaction is unnecessary since the path cannot be
garbage-collected (it's a temporary root).
2008-03-01 21:05:33 +00:00
Eelco Dolstra
0a84137c45 * checkVarDefs: don't check in closed terms, which don't have
undefined variables by definition.  This matters for the
  implementation of "with", which does a call to checkVarDefs to see
  if the body of the with has no undefined variables.  (It can't be
  checked at parse time because you don't know which variables are in
  the "with" attribute set.)  If we check closed terms, then we check
  not just the with body but also the substituted terms, which are
  typically very large.  This is the cause of the poor nix-env
  performance on Nixpkgs lately.  It didn't happen earlier because
  "with" wasn't used very often in the past.

  This fix improves nix-env performance roughly 60x on current Nixpkgs.
  nix-env -qa is down from 29.3s to 0.5s on my laptop, and nix-env -qa
  --out-path is down from 229s to 3.39s.  Not bad for a 1-line fix :-)
2008-02-21 12:01:24 +00:00
Eelco Dolstra
0ed89c569f * Fix the parsing of
''
      '${foo}'
    ''
    
  where the antiquote should work as expected, instead of giving the
  string "'${foo}'".
2008-02-05 13:38:07 +00:00
Eelco Dolstra
66c51dc215 * nix-store --dump-db / --load-db to dump/load the Nix DB.
* nix-store --register-validity: option to supply the content hash of
  each path.
* Removed compatibility with Nix <= 0.7 stores.
2008-01-29 18:17:36 +00:00
Michael Raskin
5b5a3af983 Probably fixed __exprToString 2008-01-20 20:44:03 +00:00
Michael Raskin
5eb5c23447 Fixed exportBuildReferenceGraph 2008-01-15 04:32:08 +00:00
Eelco Dolstra
7d0f6aed59 * New primop `unsafeDiscardStringContext' to get rid of string
contexts.  Needed to prevent unnecessary dependencies when building
  the NixOS manual.
2008-01-04 14:22:49 +00:00
Eelco Dolstra
e0ca671491 * More documentation. 2007-12-31 02:52:17 +00:00
Eelco Dolstra
dedd62dd0c * More release notes. 2007-12-31 00:08:09 +00:00
Michael Raskin
2196cb67c5 Added a way to include derivation with its outputs and dependencies - exportBuildReferenceGraph 2007-12-30 09:30:56 +00:00
Eelco Dolstra
df303666bc * Use strsignal if available to give better error messages for
builders that fail due to a signal.
2007-12-14 14:49:35 +00:00
Eelco Dolstra
30e4653783 * Mac OS X 10.5 compatibility. 2007-12-13 22:06:19 +00:00
Michael Raskin
3a2833daff corrected --help message for nix-store a bit 2007-12-10 22:16:47 +00:00
Eelco Dolstra
b42ef9c054 * Syntax to escape '', ${. 2007-12-06 10:20:58 +00:00
Eelco Dolstra
6d6c68c0d2 * Added a new kind of multi-line string literal delimited by two
single quotes.  Example (from NixOS):

    job = ''
      start on network-interfaces

      start script

        rm -f /var/run/opengl-driver
        ${if videoDriver == "nvidia"        
          then "ln -sf ${nvidiaDrivers} /var/run/opengl-driver"
          else if cfg.driSupport
          then "ln -sf ${mesa} /var/run/opengl-driver"
          else ""
        }

        rm -f /var/log/slim.log

      end script
    '';

  This style has two big advantages:

  - \, ' and " aren't special, only '' and ${.  So you get a lot less
    escaping in shell scripts / configuration files in Nixpkgs/NixOS.
    The delimiter '' is rare in scripts (and can usually be written as
    "").  ${ is also fairly rare.

    Other delimiters such as <<...>>, {{...}} and <|...|> were also
    considered but this one appears to have the fewest drawbacks
    (thanks Martin).

  - Indentation is intelligently stripped so that multi-line strings
    can follow the nesting structure of the containing Nix
    expression.  E.g. in the example above 6 spaces are stripped from
    the start of each line.  This prevents unnecessary indentation in
    generated files (which sometimes even breaks things).

  See tests/lang/eval-okay-ind-string.nix for some examples.
2007-11-30 16:48:45 +00:00
Eelco Dolstra
633518628f * nix-env -e: support uninstalling by path, so that one can say
$ nix-env -e $(which firefox)

  or

    $ nix-env -e /nix/store/nywzlygrkfcgz7dfmhm5xixlx1l0m60v-pan-0.132

* nix-env -i: if an argument contains a slash anywhere, treat it as a
  path and follow it through symlinks into the Nix store.  This allows
  things like

    $ nix-build -A firefox
    $ nix-env -i ./result

* nix-env -q/-i/-e: don't complain when the `*' selector doesn't match
  anything.  In particular, `nix-env -q \*' doesn't fail anymore on an
  empty profile.
2007-11-29 16:18:24 +00:00
Eelco Dolstra
06f95dd07c * New primop `readFile' to get the contents of a file as a string. 2007-11-21 13:49:59 +00:00
Eelco Dolstra
c370755583 * Flag `--no-build-hook' to disable distributed builds.
* queryDeriver in daemon mode: don't barf if the other side returns an
  empty string (which means there is no deriver).
2007-11-16 16:15:26 +00:00
Eelco Dolstra
c05783ad67 * nix-store --import: add a flag --require-signature. 2007-11-16 16:10:27 +00:00
Eelco Dolstra
149e273023 * Don't need gc.hh. 2007-11-15 15:07:27 +00:00
Michael Raskin
6fc60e2060 Added possibility to specify garbage collection levels for store paths; so packages can share intermediate results of compilation and GC will collect it automatically while never touching tarballs, for example. 2007-11-15 03:47:12 +00:00
Eelco Dolstra
fa44e401a8 * Documented multi-user Nix. 2007-10-31 18:01:56 +00:00
Eelco Dolstra
bfea7b1f35 * Make the --prebuilt-only' / -b' option work not just for queries
but installations/upgrades as well.  So `nix-env -ub \*' will
  upgrade only those packages for which a substitute is available (or
  to be precise, it will upgrade each package to the highest version
  for which a substitute is available).
2007-10-29 14:31:45 +00:00
Eelco Dolstra
42d80d1170 * On FreeBSD, sys/mount.h needs sys/param.h. 2007-10-29 10:03:07 +00:00
Eelco Dolstra
d91cd30563 * Detect whether chroot / bind-mount support is available. 2007-10-27 16:51:55 +00:00
Eelco Dolstra
dc6f373842 * Delete the chroot directory automatically.
* Removed some debug messages.
2007-10-27 16:06:38 +00:00
Eelco Dolstra
9397cd30c8 * Support for doing builds in a chroot under Linux. The builder is
executed in a chroot that contains just the Nix store, the temporary
  build directory, and a configurable set of additional directories
  (/dev and /proc by default).  This allows a bit more purity
  enforcement: hidden build-time dependencies on directories such as
  /usr or /nix/var/nix/profiles are no longer possible.  As an added
  benefit, accidental network downloads (cf. NIXPKGS-52) are prevented
  as well (because files such as /etc/resolv.conf are not available in
  the chroot).

  However the usefulness of chroots is diminished by the fact that
  many builders depend on /bin/sh, so you need /bin in the list of
  additional directories.  (And then on non-NixOS you need /lib as
  well...)
2007-10-27 00:46:59 +00:00
Eelco Dolstra
0b4ed64d29 * "trace" primop: write the trace to standard error. 2007-10-26 18:25:50 +00:00
Eelco Dolstra
f90d85107e 2007-10-22 12:05:30 +00:00
Eelco Dolstra
14e47e9c2c * Doh. 2007-10-10 14:58:00 +00:00
Eelco Dolstra
d04291cfab * Doh! Don't change the permissions on /nix/store. 2007-10-10 13:52:38 +00:00