Commit graph

7417 commits

Author SHA1 Message Date
Cole Helbling
e5b70d47aa libutil: cleanup savedCwd logic
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-04-04 08:20:11 -07:00
Cole Helbling
2a45cf54e4 libutil: Properly guard self-allocating getcwd on GNU
It's a GNU extension, as pointed out by pennae.
2022-04-01 12:20:34 -07:00
Cole Helbling
7f5caaa7c0 libutil: Don't use std::filesystem
Just in case making libutil depend on std::filesystem is unacceptable,
here is the non-filesystem approach.
2022-04-01 10:24:31 -07:00
aszlig
435848cef1 libutil: Fix restoring mount namespace
I regularly pass around simple scripts by using nix-shell as the script
interpreter, eg. like this:

    #!/usr/bin/env nix-shell
    #!nix-shell -p dd_rescue coreutils bash -i bash

While this works most of the time, I recently had one occasion where it
would not and the above would result in the following:

    $ sudo ./myscript.sh
    bash: ./myscript.sh: No such file or directory

Note the "sudo" here, because this error only occurs if we're root.

The reason for the latter is because running Nix as root means that we
can directly access the store, which makes sure we use a filesystem
namespace to make the store writable. XXX - REWORD!

So when stracing the process, I stumbled on the following sequence:

    openat(AT_FDCWD, "/proc/self/ns/mnt", O_RDONLY) = 3
    unshare(CLONE_NEWNS)                            = 0
    ... later ...
    getcwd("/the/real/cwd", 4096)                   = 14
    setns(3, CLONE_NEWNS)                           = 0
    getcwd("/", 4096)                               = 2

In the whole strace output there are no calls to chdir() whatsoever, so
I decided to look into the kernel source to see what else could change
directories and found this[1]:

    /* Update the pwd and root */
    set_fs_pwd(fs, &root);
    set_fs_root(fs, &root);

The set_fs_pwd() call is roughly equivalent to a chdir() syscall and
this is called when the setns() syscall is invoked[2].

[1]: b14ffae378/fs/namespace.c (L4659)
[2]: b14ffae378/kernel/nsproxy.c (L346)
2022-04-01 09:30:52 -07:00
Eelco Dolstra
bf4895961d
Merge pull request #6344 from flox/profile_url_uri
profile!: consistent use of url/uri. create new version
2022-04-01 14:38:32 +02:00
Eelco Dolstra
fdfe737867 Fix handling of outputHash when outputHashAlgo is not specified
https://hydra.nixos.org/build/171351131
2022-04-01 12:40:49 +02:00
Eelco Dolstra
c9a29d0d92
Merge pull request #6227 from NixOS/impure-derivations-ng
Impure derivations
2022-03-31 19:58:35 +02:00
Eelco Dolstra
7537097284 Provide default values for outputHashAlgo and outputHashMode 2022-03-31 16:56:44 +02:00
Eelco Dolstra
a99af85a77 Fix macOS build 2022-03-31 16:39:18 +02:00
Eelco Dolstra
6051cc954b Rename 'pure' -> 'sandboxed' for consistency 2022-03-31 16:12:25 +02:00
Eelco Dolstra
e279fbb16a needsNetworkAccess() -> isSandboxed() 2022-03-31 16:06:40 +02:00
Eelco Dolstra
4e043c2f32 Document isPure() 2022-03-31 16:01:50 +02:00
Eelco Dolstra
d7fc33c842 Fix macOS build 2022-03-31 15:59:14 +02:00
Eelco Dolstra
18935e8b9f Support fixed-output derivations depending on impure derivations 2022-03-31 13:43:20 +02:00
Eelco Dolstra
5cd72598fe Add support for impure derivations
Impure derivations are derivations that can produce a different result
every time they're built. Example:

  stdenv.mkDerivation {
    name = "impure";
    __impure = true; # marks this derivation as impure
    outputHashAlgo = "sha256";
    outputHashMode = "recursive";
    buildCommand = "date > $out";
  };

Some important characteristics:

* This requires the 'impure-derivations' experimental feature.

* Impure derivations are not "cached". Thus, running "nix-build" on
  the example above multiple times will cause a rebuild every time.

* They are implemented similar to CA derivations, i.e. the output is
  moved to a content-addressed path in the store. The difference is
  that we don't register a realisation in the Nix database.

* Pure derivations are not allowed to depend on impure derivations. In
  the future fixed-output derivations will be allowed to depend on
  impure derivations, thus forming an "impurity barrier" in the
  dependency graph.

* When sandboxing is enabled, impure derivations can access the
  network in the same way as fixed-output derivations. In relaxed
  sandboxing mode, they can access the local filesystem.
2022-03-31 13:43:20 +02:00
Eelco Dolstra
0fe8849914
Merge pull request #6337 from danpls/fix-to-json-repl
libexpr: Throw the correct error in toJSON
2022-03-31 11:33:16 +02:00
Eelco Dolstra
28309352d9 replaceEnv(): Pass newEnv by reference 2022-03-31 10:39:53 +02:00
Tom Bereknyei
50f9f335c9 profile!: consistent use of url/uri. create new version 2022-03-30 16:35:26 -04:00
Tom Bereknyei
d77823b502 bundler: update default bundler to support new bundler API 2022-03-30 16:10:42 -04:00
Daniel Pauls
629edd43ba libutil: Change return value of addTrace to void
The return value of BaseError::addTrace(...) is never used and
error-prone as subclasses calling it will return a BaseError instead of
the subclass.
This commit changes its return value to be void.
2022-03-30 18:37:32 +02:00
Daniel Pauls
fa83b865a2 libexpr: Throw the correct error in toJSON
BaseError::addTrace(...) returns a BaseError, but we want to
throw a TypeError instead.

Fixes #6336.
2022-03-30 15:50:13 +02:00
Théophane Hufschmitt
390269ed87 Simplify the handling of the hash modulo
Rather than having four different but very similar types of hashes, make
only one, with a tag indicating whether it corresponds to a regular of
deferred derivation.

This implies a slight logical change: The original Nix+multiple-outputs
model assumed only one hash-modulo per derivation. Adding
multiple-outputs CA derivations changed this as these have one
hash-modulo per output. This change is now treating each derivation as
having one hash modulo per output.
This obviously means that we internally loose the guaranty that
all the outputs of input-addressed derivations have the same hash
modulo. But it turns out that it doesn’t matter because there’s nothing
in the code taking advantage of that fact (and it probably shouldn’t
anyways).

The upside is that it is now much easier to work with these hashes, and
we can get rid of a lot of useless `std::visit{ overloaded`.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2022-03-29 18:17:35 +02:00
Théophane Hufschmitt
2d572a250f
Merge pull request #6330 from edolstra/run-remote-store
nix {run,shell}: Print a better error message if the store is not local
2022-03-28 17:57:37 +02:00
Eelco Dolstra
b266fd53dd nix {run,shell}: Print a better error message if the store is not local
Closes #6317
2022-03-28 14:58:38 +02:00
Eelco Dolstra
057f9ee190 nix profile install: Don't use queryDerivationOutputMap()
Instead get the outputs from Installable::build(). This will also
allow 'nix profile install' to support impure derivations.

Fixes #6286.
2022-03-28 14:23:39 +02:00
Eelco Dolstra
1844172dd1
Merge pull request #6314 from edolstra/experimental-primop
Only provide builtins is the corresponding experimental feature is enabled
2022-03-25 16:16:31 +01:00
Eelco Dolstra
16cf1e6089
Merge pull request #6311 from edolstra/return-wanted-paths
Make buildPathsWithResults() only return info on wanted outputs
2022-03-25 15:44:39 +01:00
Eelco Dolstra
fc35b11a7c Fix mismatched tag warning on clang 2022-03-25 15:22:22 +01:00
Eelco Dolstra
8c363eb3eb Document getFlake
Fixes #5523.
2022-03-25 14:19:55 +01:00
Eelco Dolstra
86b05ccd54 Only provide builtin.{getFlake,fetchClosure} is the corresponding experimental feature is enabled
This allows writing fallback code like

  if builtins ? fetchClosure then
    builtins.fetchClose { ... }
  else
    builtins.storePath ...
2022-03-25 14:04:18 +01:00
Eelco Dolstra
55bc524019
Merge pull request #6051 from Ma27/fix-empty-nix-log
`nix log` should also work if the log didn't provide any output
2022-03-25 10:35:10 +01:00
Eelco Dolstra
50c229ad9a
Use wantOutput
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-03-25 08:02:49 +01:00
polykernel
cbcb69a39c nix: allow whitespace characters before command in repl
Before this change, processLine always uses the first character
as the start of the line. This cause whitespaces to matter at the
beginning of the line whereas it does not matter anywhere else.

This commit trims leading white spaces of the string line so that
subsequent operations can be performed on the string without explicitly
tracking starting and ending indices of the string.
2022-03-24 21:33:29 -04:00
Eelco Dolstra
540d7e33d8 Retry substitution after an incomplete closure only once
This avoids an infinite loop in the final test in
tests/binary-cache.sh. I think this was only not triggered previously
by accident (because we were clearing wantedOutputs in between).
2022-03-24 23:25:12 +01:00
Eelco Dolstra
fe5509df9a Only return wanted outputs 2022-03-24 23:24:48 +01:00
Eelco Dolstra
09796c0263 Random cleanup 2022-03-24 23:24:10 +01:00
Eelco Dolstra
175c78591b Random cleanup 2022-03-24 23:09:43 +01:00
Maximilian Bosch
c85467a1b6
Revert "TarArchive: Small refactoring"
This reverts commit 50a35860ee.

With this change Nix fails to open bzip2 logfiles that were created from
builds with no stdout/stderr.
2022-03-24 22:30:46 +01:00
Eelco Dolstra
f902f3c2cb Add experimental feature 'fetch-closure' 2022-03-24 21:33:33 +01:00
Eelco Dolstra
e5f7029ba4 nix store make-content-addressed: Support --from / --to 2022-03-24 21:33:33 +01:00
Eelco Dolstra
98658ae9d2 Document fetchClosure 2022-03-24 21:33:33 +01:00
Eelco Dolstra
28186b7044 Add a test for fetchClosure and 'nix store make-content-addressed' 2022-03-24 21:33:33 +01:00
Eelco Dolstra
4120930ac1 fetchClosure: Only allow some "safe" store types 2022-03-24 21:33:33 +01:00
Eelco Dolstra
7ffda0af6e fetchClosure: Skip makeContentAddressed() if toPath is already valid 2022-03-24 21:33:33 +01:00
Eelco Dolstra
5acaf13d35 Rename 'nix store make-content-addressable' to 'nix store make-content-addressed' 2022-03-24 21:33:33 +01:00
Eelco Dolstra
f18607549c Fix makeContentAddressed() on self-references
LocalStore::addToStore() since
79ae9e4558 expects a regular NAR hash,
rather than a NAR hash modulo self-references. Fixes #6300.

Also, makeContentAddressed() now rewrites the entire closure (so 'nix
store make-content-addressable' no longer needs '-r'). See #6301.
2022-03-24 21:33:33 +01:00
Eelco Dolstra
545c2d0d8c fetchClosure: Allow a path to be rewritten to CA on the fly
The advantage is that the resulting closure doesn't need to be signed,
so you don't need to configure any binary cache keys on the client.
2022-03-24 21:33:33 +01:00
Eelco Dolstra
7f6fe8ca1d Rename 2022-03-24 21:33:33 +01:00
Eelco Dolstra
41659418cf fetchClosure: Require a CA path in pure mode 2022-03-24 21:33:33 +01:00
Eelco Dolstra
f4bafc412f Add builtins.fetchClosure
This allows closures to be imported at evaluation time, without
requiring the user to configure substituters. E.g.

  builtins.fetchClosure {
    storePath = /nix/store/f89g6yi63m1ywfxj96whv5sxsm74w5ka-python3.9-sqlparse-0.4.2;
    from = "https://cache.ngi0.nixos.org";
  }
2022-03-24 21:33:33 +01:00