Commit graph

958 commits

Author SHA1 Message Date
Eelco Dolstra
46e0919ced * `nix-store --export --sign': sign the Nix archive using the RSA key
in /nix/etc/nix/signing-key.sec
2007-02-21 14:31:42 +00:00
Eelco Dolstra
b824a1daee * Start of `nix-store --export' operation for serialising a store
path.  This is like `nix-store --dump', only it also dumps the
  meta-information of the store path (references, deriver).  Will add
  a `--sign' flag later to add a cryptographic signature, which we
  will use for exchanging store paths between build farm machines in a
  secure manner.
2007-02-20 23:17:20 +00:00
Eelco Dolstra
46605fb4f5 * Fix 64-bit compiler warnings. 2007-02-06 20:03:53 +00:00
Eelco Dolstra
451dbf687f * nix-env now maintains meta info (from the `meta' derivation
attribute) about installed packages in user environments.  Thus, an
  operation like `nix-env -q --description' shows useful information
  not only on available packages but also on installed packages.

* nix-env now passes the entire manifest as an argument to the Nix
  expression of the user environment builder (not just a list of
  paths), so that in particular the user environment builder has
  access to the meta attributes.
  
* New operation `--set-flag' in nix-env to change meta info of
  installed packages.  This will be useful to pass per-package
  policies to the user environment builder (e.g., how to resolve
  collision or whether to disable a package (NIX-80)) or upgrade
  policies in nix-env (e.g., that a package should be "masked", that
  is, left untouched by upgrade actions).  Example:

  $ nix-env --set-flag enabled false ghc-6.4
2007-02-02 01:52:42 +00:00
Eelco Dolstra
f52de527c7 * Doh! 2007-01-29 15:55:49 +00:00
Eelco Dolstra
b618fa6eb6 * computeStorePathForText: take the references into account when
computing the store path (NIX-77).  This is an important security
  property in multi-user Nix stores.

  Note that this changes the store paths of derivations (since the
  derivation aterms are added using addTextToStore), but not most
  outputs (unless they use builtins.toFile).
2007-01-29 15:51:37 +00:00
Eelco Dolstra
c558b1583c * Don't capitalise the primop functions. 2007-01-29 15:15:37 +00:00
Eelco Dolstra
18e6096105 * Organise primops.cc a bit better. 2007-01-29 15:11:32 +00:00
Eelco Dolstra
7349bd0176 New primitives:
* `sub' to subtract two numbers.
* `stringLength' to get the length of a string.
* `substring' to get a substring of a string.  These should be enough
  to allow most string operations to be expressed.
2007-01-29 14:23:09 +00:00
Eelco Dolstra
7dedbd896a * filterSource: pass strings to the predicate function instead of
paths.  Paths can have unexpected semantics.
2007-01-29 13:32:50 +00:00
Eelco Dolstra
84a84afb0e * Nasty: Glibc clears the TMPDIR environment variable in setuid
programs, so if a builder uses TMPDIR, then it will fail when
  executed through nix-setuid-helper.  In fact Glibc clears a whole
  bunch of variables (see sysdeps/generic/unsecvars.h in the Glibc
  sources), but only TMPDIR should matter in practice.  As a
  workaround, we reinitialise TMPDIR from NIX_BUILD_TOP.
2007-01-24 13:31:20 +00:00
Eelco Dolstra
fac63d6416 * exportReferencesGraph: work on paths within store paths as well. 2007-01-23 16:57:43 +00:00
Eelco Dolstra
71ceb1c161 * Handle multiple indirect symlinks when loading a Nix expression. 2007-01-15 14:50:25 +00:00
Eelco Dolstra
e4b0666f8e * builtins.filterSource: pass the type of the file ("regular",
"directory", "symlink") as the second argument to the filter
  predicate.
2007-01-15 08:54:51 +00:00
Eelco Dolstra
63f3ce6d9a * `nix-store --verify': revive checking the referrers table. This is
important to get garbage collection to work if there is any
  inconsistency in the database (because the referrer table is used to
  determine whether it is safe to delete a path).
* `nix-store --verify': show some progress.
2007-01-14 17:28:30 +00:00
Eelco Dolstra
8f67b35886 * Make the garbage collector more resilient to certain consistency
errors: in-use paths now cause a warning, not a fatal error.
2007-01-14 16:24:49 +00:00
Eelco Dolstra
e418976107 * Option --argstr for passing string arguments easily. (NIX-75) 2007-01-14 12:32:44 +00:00
Eelco Dolstra
afe23b5f38 * nix-pack-closure: store the top-level store paths in the closure.
* nix-unpack-closure: extract the top-level paths from the closure and
  print them on stdout.  This allows them to be installed, e.g.,
  "nix-env -i $(nix-unpack-closure)".  (NIX-64)
2007-01-13 19:50:42 +00:00
Eelco Dolstra
f25f900045 * Allow multiple --attr / -A arguments in nix-build / nix-instantiate
(NIX-74).
2007-01-13 18:25:30 +00:00
Eelco Dolstra
215505bb46 * Removed chroot support. 2007-01-13 17:54:01 +00:00
Eelco Dolstra
f23dcdd603 * Canonicalise ASTs in `nix-instantiate --eval': remove position
info, sort attribute sets.
2007-01-13 16:17:07 +00:00
Eelco Dolstra
05879db628 * Memoize strict evaluation. 2007-01-13 15:41:54 +00:00
Eelco Dolstra
5011588459 * printTermAsXML: treat derivations specially; emit an element
<derivation outPath=... drvPath=...> attrs </derivation>.  Only emit
  the attributes of any specific derivation only.  This prevents
  exponententially large XML output due to the absense of sharing.
2007-01-13 15:11:10 +00:00
Eelco Dolstra
792878af91 * Make printing an expression as XML interruptible. 2007-01-13 14:48:41 +00:00
Eelco Dolstra
11158028be * Cleanup. 2007-01-13 14:21:49 +00:00
Eelco Dolstra
1073b1780a * Remove debug message. 2006-12-13 14:29:05 +00:00
Eelco Dolstra
a3e6415ba8 * New primop builtins.filterSource, which can be used to filter files
from a source directory.  All files for which a predicate function
  returns true are copied to the store.  Typical example is to leave
  out the .svn directory:

    stdenv.mkDerivation {
      ...
      src = builtins.filterSource
        (path: baseNameOf (toString path) != ".svn")
        ./source-dir;
      # as opposed to
      #   src = ./source-dir;
    }

  This is important because the .svn directory influences the hash in
  a rather unpredictable and variable way.
2006-12-12 23:05:01 +00:00
Eelco Dolstra
b438d37558 * In dumpPath(): pass a function object that allows files to be
selectively in/excluded from the dump.
2006-12-12 21:51:02 +00:00
Eelco Dolstra
7ace29dae7 * New operation `nix-env --set' which sets a user environment to a
single derivation specified by the argument.  This is useful when we
  want to have a profile for a single derivation, such as a server
  configuration.  Then we can just say (e.g.)

  $ nix-env -p /.../server-profile -f server.nix --set -A server

  We can't do queries or upgrades on such a profile, but we can do
  rollbacks.  The advantage over -i is that we don't have to worry
  about other packages having been installed in the profile
  previously; --set gets rid of them.
2006-12-12 19:06:02 +00:00
Eelco Dolstra
1a7e88bbd9 * New built-in function `builtins.attrNames' that returns the
names of the attributes in an attribute set.
2006-12-12 16:14:31 +00:00
Eelco Dolstra
b17677462c * Use lchown() instead of chown() in canonicalisePathMetaData(). This
matters when running as root, since then we don't use the setuid
  helper (which already used lchown()).
  
* Also check for an obscure security problem on platforms that don't
  have lchown.  Then we can't change the ownership of symlinks, which
  doesn't matter *except* when the containing directory is writable by
  the owner (which is the case with the top-level Nix store directory).
2006-12-09 20:02:27 +00:00
Eelco Dolstra
5f681988f2 * Use deletePathWrapped() in more places. 2006-12-09 00:26:24 +00:00
Eelco Dolstra
fa33303146 * Goal cancellation inside the waitForInput() loop needs to be handled
very carefully, since it can invalidate iterators into the
  `children' map.
2006-12-08 18:41:48 +00:00
Eelco Dolstra
06c4929958 * Some refactoring.
* Throw more exceptions as BuildErrors instead of Errors.  This
  matters when --keep-going is turned on.  (A BuildError is caught
  and terminates the goal in question, an Error terminates the
  program.)
2006-12-08 17:26:21 +00:00
Eelco Dolstra
9dbfe242e3 * Kill a build if it has gone for more than a certain number of
seconds without producing output on stdout or stderr (NIX-65).  This
  timeout can be specified using the `--max-silent-time' option or the
  `build-max-silent-time' configuration setting.  The default is
  infinity (0).

* Fix a tricky race condition: if we kill the build user before the
  child has done its setuid() to the build user uid, then it won't be
  killed, and we'll potentially lock up in pid.wait().  So also send a
  conventional kill to the child.
2006-12-08 15:44:00 +00:00
Eelco Dolstra
d3fe6ab024 * Also for convenience, change the ownership of the build output even
in case of failure.
2006-12-08 00:19:50 +00:00
Eelco Dolstra
096194ab29 * Remove ancient terminology. 2006-12-07 23:58:36 +00:00
Eelco Dolstra
6833e8bbe8 * When keeping the temporary build directory (-K), change the owner
back to the Nix account.
2006-12-07 23:27:40 +00:00
Eelco Dolstra
e24d0201c2 * Doh! 2006-12-07 22:07:05 +00:00
Eelco Dolstra
2819eb36a4 * Be less verbose. 2006-12-07 21:43:35 +00:00
Eelco Dolstra
4ca01065c3 * Rename all those main.cc files. 2006-12-07 20:47:30 +00:00
Eelco Dolstra
d03f0d4117 * Check for lchown. 2006-12-07 18:51:11 +00:00
Eelco Dolstra
c3286ec020 * Don't count on the Pid deconstructor to kill the child process,
since if we're running a build user in non-root mode, we can't.  Let
  the setuid helper do it.
2006-12-07 17:52:58 +00:00
Eelco Dolstra
a82d80ddeb * Move setuidCleanup() to libutil. 2006-12-07 16:40:41 +00:00
Eelco Dolstra
f76fdb6d42 * If not running as root, let the setuid helper kill the build user's
processes before and after the build.
2006-12-07 16:33:31 +00:00
Eelco Dolstra
ec23ecc64d * In the garbage collector, if deleting a path fails, try to fix its
ownership, then try again.
2006-12-07 15:54:52 +00:00
Eelco Dolstra
a0a43c3206 * When not running as root, call the setuid helper to change the
ownership of the build result after the build.
2006-12-07 15:18:14 +00:00
Eelco Dolstra
6a07ff1ec0 * Change the ownership of store paths to the Nix account before
deleting them using the setuid helper.
2006-12-07 14:14:35 +00:00
Eelco Dolstra
7d8cf316ee * Pass the actual build user to the setuid helper. 2006-12-07 11:27:32 +00:00
Eelco Dolstra
a45c498e4e * If Nix is not running as root, call the setuid helper to start the
builder under the desired build user.
2006-12-07 00:42:30 +00:00
Eelco Dolstra
813a7c65c9 * Sanity check. 2006-12-07 00:19:27 +00:00
Eelco Dolstra
6a8e60913a * Move killUser() to libutil so that the setuid helper can use it. 2006-12-07 00:16:07 +00:00
Eelco Dolstra
79875c5e42 * Change the ownership of the current directory to the build user. 2006-12-06 23:52:25 +00:00
Eelco Dolstra
62ab131412 * Verify that the desired target user is in the build users group (as
specified in the setuid config file).
2006-12-06 23:15:26 +00:00
Eelco Dolstra
f07ac41656 * Check that the caller is allowed to call the setuid helper. The
allowed uid is specified in a configuration file in
  /etc/nix-setuid.conf.
2006-12-06 22:45:41 +00:00
Eelco Dolstra
ef281b93c2 * Fix the safety check. 2006-12-06 20:18:29 +00:00
Eelco Dolstra
6e5ec1029a * Get rid of `build-users'. We'll just take all the members of
`build-users-group'.  This makes configuration easier: you can just
  add users in /etc/group.
2006-12-06 20:00:15 +00:00
Eelco Dolstra
751f6d2157 * nix-setuid-helper: allow running programs under a different uid. 2006-12-06 17:29:10 +00:00
Eelco Dolstra
9f0efa6611 * Start of the setuid helper (the program that performs the operations
that have to be done as root: running builders under different uids,
  changing ownership of build results, and deleting paths in the store
  with the wrong ownership).
2006-12-06 01:24:02 +00:00
Eelco Dolstra
2b558843a2 * Be less chatty. 2006-12-05 19:01:19 +00:00
Eelco Dolstra
44cad9630f * Urgh. Do setgid() before setuid(), because the semantics of setgid()
changes completely depending on whether you're root...
2006-12-05 18:28:15 +00:00
Eelco Dolstra
6f0d050324 * Tricky: child processes should not send data to the client since
that might mess up the protocol.  And besides, the socket file
  descriptor is probably closed.
2006-12-05 18:21:16 +00:00
Eelco Dolstra
4c1c37d0b6 * FreeBSD returns ESRCH when there are no processes to kill. 2006-12-05 18:07:46 +00:00
Eelco Dolstra
8d1854c3f1 * Oops! In daemon mode, we can't run as root either if build-users is empty. 2006-12-05 17:44:19 +00:00
Eelco Dolstra
99655245ae * Use an explicit handler for SIGCHLD, since SIG_IGN doesn't do the
right thing on FreeBSD 4 (it leaves zombies).
2006-12-05 17:21:42 +00:00
Eelco Dolstra
62b0497c0f * Better message. 2006-12-05 16:17:01 +00:00
Eelco Dolstra
c808e6252f * Ugly hack to handle spurious SIGPOLLs. 2006-12-05 15:36:31 +00:00
Eelco Dolstra
fd4a9db91f * Some renaming. 2006-12-05 14:15:51 +00:00
Eelco Dolstra
a9c4f66cfb * Allow unprivileged users to run the garbage collector and to do
`nix-store --delete'.  But unprivileged users are not allowed to
  ignore liveness.
* `nix-store --delete --ignore-liveness': ignore the runtime roots as
  well.
2006-12-05 02:18:46 +00:00
Eelco Dolstra
29cf434a35 * The determination of the root set should be made by the privileged
process, so forward the operation.
* Spam the user about GC misconfigurations (NIX-71).
* findRoots: skip all roots that are unreadable - the warnings with
  which we spam the user should be enough.
2006-12-05 01:31:45 +00:00
Eelco Dolstra
8623256f48 * findRoots: return a map from the symlink (outside of the store) to
the store path (inside the store).
2006-12-05 00:48:36 +00:00
Eelco Dolstra
d27a73b1a9 * In addPermRoot, check that the root that we just registered can be
found by the garbage collector.  This addresses NIX-71 and is a
  particular concern in multi-user stores.
2006-12-05 00:34:42 +00:00
Eelco Dolstra
74033a844f * Add indirect root registration to the protocol so that unprivileged
processes can register indirect roots.  Of course, there is still
  the problem that the garbage collector can only read the targets of
  the indirect roots when it's running as root...
2006-12-04 23:29:16 +00:00
Eelco Dolstra
0d40f6d7bb * Not every OS knows about SIGPOLL. 2006-12-04 22:58:44 +00:00
Eelco Dolstra
7751160e9f * Don't redirect stderr. 2006-12-04 19:10:23 +00:00
Eelco Dolstra
40c3529909 * Handle exceptions and stderr for all protocol functions.
* SIGIO -> SIGPOLL (POSIX calls it that).
* Use sigaction instead of signal to register the SIGPOLL handler.
  Sigaction is better defined, and a handler registered with signal
  appears not to interrupt fcntl(..., F_SETLKW, ...), which is bad.
2006-12-04 17:55:14 +00:00
Eelco Dolstra
0130ef88ea * Daemon mode (`nix-worker --daemon'). Clients connect to the server
via the Unix domain socket in /nix/var/nix/daemon.socket.  The
  server forks a worker process per connection.
* readString(): use the heap, not the stack.
* Some protocol fixes.
2006-12-04 17:17:13 +00:00
Eelco Dolstra
4740baf3a6 * When NIX_REMOTE=daemon, connect to /nix/var/nix/daemon.socket
instead of forking a worker.
2006-12-04 14:21:39 +00:00
Eelco Dolstra
f5f0cf423f * Refactoring. 2006-12-04 13:28:14 +00:00
Eelco Dolstra
052b6fb149 * Pass the verbosity level to the worker. 2006-12-04 13:15:29 +00:00
Eelco Dolstra
1e16d20655 * Install the worker in bindir, not libexecdir.
* Allow the worker path to be overriden through the NIX_WORKER
  environment variable.
2006-12-04 13:09:16 +00:00
Eelco Dolstra
9322b399f3 * Doh. 2006-12-03 20:41:22 +00:00
Eelco Dolstra
f4279bcde0 * Don't run setuid root when build-users is empty.
* Send startup errors to the client.
2006-12-03 16:25:19 +00:00
Eelco Dolstra
35247c4c9f * Removed `build-allow-root'.
* Added `build-users-group', the group under which builds are to be
  performed.
* Check that /nix/store has 1775 permission and is owner by the
  build-users-group.
2006-12-03 15:32:38 +00:00
Eelco Dolstra
84d6459bd5 * Use setreuid if setresuid is not available. 2006-12-03 14:32:22 +00:00
Eelco Dolstra
a9f9241054 * Handle a subtle race condition: the client closing the socket
between the last worker read/write and the enabling of the signal
  handler.
2006-12-03 03:16:27 +00:00
Eelco Dolstra
3ed9e4ad9b * Some hardcore magic to handle asynchronous client disconnects.
The problem is that when we kill the client while the worker is
  building, and the builder is not writing anything to stderr, then
  the worker never notice that the socket is closed on the other side,
  so it just continues indefinitely.  The solution is to catch SIGIO,
  which is sent when the far side of the socket closes, and simulate
  an normal interruption.  Of course, SIGIO is also sent every time
  the client sends data over the socket, so we only enable the signal
  handler when we're not expecting any data...
2006-12-03 03:03:36 +00:00
Eelco Dolstra
4251f94b32 * Use a Unix domain socket instead of pipes. 2006-12-03 02:36:44 +00:00
Eelco Dolstra
8c76df93e6 * Better error message if the worker doesn't start. 2006-12-03 02:22:04 +00:00
Eelco Dolstra
363f40022f * Pid::kill() should be interruptable. 2006-12-03 02:12:26 +00:00
Eelco Dolstra
7951c3c546 * Some hackery to propagate the worker's stderr and exceptions to the
client.
2006-12-03 02:08:13 +00:00
Eelco Dolstra
714fa24cfb * Run the worker in a separate session to prevent terminal signals
from interfering.
2006-12-03 00:52:27 +00:00
Eelco Dolstra
e25fad691a * Move addTempRoot() to the store API, and add another function
syncWithGC() to allow clients to register GC roots without needing
  write access to the global roots directory or the GC lock.
2006-12-02 16:41:36 +00:00
Eelco Dolstra
30bf547f4f * Doh. 2006-12-02 15:46:17 +00:00
Eelco Dolstra
536595b072 * Remove most of the old setuid code.
* Much simpler setuid code for the worker in slave mode.
2006-12-02 15:45:51 +00:00
Eelco Dolstra
9c9cdb06d0 * Remove SwitchToOriginalUser, we're not going to need it anymore. 2006-12-02 14:34:14 +00:00
Eelco Dolstra
8ba5d32769 * Remove queryPathHash().
* Help for nix-worker.
2006-12-02 14:27:24 +00:00
Eelco Dolstra
fcd9900d74 * Replace read-only calls to addTextToStore. 2006-12-01 21:00:39 +00:00
Eelco Dolstra
a824d58b56 * Merge addToStore and addToStoreFixed.
* addToStore now adds unconditionally, it doesn't use readOnlyMode.
  Read-only operation is up to the caller (who can call
  computeStorePathForPath).
2006-12-01 20:51:18 +00:00
Eelco Dolstra
ceb982a1be * Right name. 2006-12-01 18:02:05 +00:00