Commit graph

88 commits

Author SHA1 Message Date
Adam Joseph
6fc56318bf local-derivation-goal.cc: add comment re: CLONE_NEWUSER
local-derivation-goal.cc contains a comment stating that "Some distros
patch Linux to not allow unprivileged user namespaces."  Let's give a
pointer to a common version of this patch for those who want more
details about this failure mode.
2022-07-17 01:23:32 -07:00
Adam Joseph
8d35f387dc local-derivation-goal.cc: warn if failing and /proc/self/ns/user missing
This commit causes nix to `warn()` if sandbox setup has failed and
`/proc/self/ns/user` does not exist.  This is usually a sign that the
kernel was compiled without `CONFIG_USER_NS=y`, which is required for
sandboxing.
2022-07-16 19:37:27 -07:00
Adam Joseph
90830b1074 local-derivation-goal.cc: warn if failing due to max_user_namespaces==0
This commit uses `warn()` to notify the user if sandbox setup fails
with errno==EPERM and /proc/sys/user/max_user_namespaces is missing or
zero, since that is at least part of the reason why sandbox setup
failed.

Note that `echo -n 0 > /proc/sys/user/max_user_namespaces` or
equivalent at boot time has been the recommended mitigation for
several Linux LPE vulnerabilities over the past few years.  Many users
have applied this mitigation and then forgotten that they have done
so.
2022-07-16 19:30:53 -07:00
Adam Joseph
8ea3a911aa local-derivation-goal.cc: improve error messages when sandboxing fails
The failure modes for nix's sandboxing setup are pretty complicated.
When nix is unable to set up the sandbox, let's provide more detail
about what went wrong.  Specifically:

* Make sure the error message includes the word "sandbox" so the user
  knows that the failure was related to sandboxing.

* If `--option sandbox-fallback false` was provided, and removing it
  would have allowed further attempts to make progress, let the user
  know.
2022-07-16 14:56:24 -07:00
Eelco Dolstra
925b975224 Embed the sandbox shell into the statically linked 'nix' binary
With this, Nix will write a copy of the sandbox shell to /bin/sh in
the sandbox rather than bind-mounting it from the host filesystem.
This makes /bin/sh work out of the box with nix-static, i.e. you no
longer get

  /nix/store/qa36xhc5gpf42l3z1a8m1lysi40l9p7s-bootstrap-stage4-stdenv-linux/setup: ./configure: /bin/sh: bad interpreter: No such file or directory
2022-06-23 04:08:28 +02:00
Eelco Dolstra
3e87c8e62b Move json stuff out of util.cc 2022-05-04 11:22:06 +02:00
Alain Zscheile
1385b20078
Get rid of most .at calls (#6393)
Use one of `get` or `getOr` instead which will either return a null-pointer (with a nicer error message) or a default value when the key is missing.
2022-05-04 07:44:32 +02:00
Sebastian Blunt
63d9a81819 Log builder args and environment variables
Previously it only logged the builder's path, this changes it to log the
arguments at the same log level, and the environment variables at the
vomit level.

This helped me debug https://github.com/svanderburg/node2nix/issues/75
2022-04-10 21:10:37 -07:00
Eelco Dolstra
e279fbb16a needsNetworkAccess() -> isSandboxed() 2022-03-31 16:06:40 +02:00
Eelco Dolstra
d7fc33c842 Fix macOS build 2022-03-31 15:59:14 +02:00
Eelco Dolstra
5cd72598fe Add support for impure derivations
Impure derivations are derivations that can produce a different result
every time they're built. Example:

  stdenv.mkDerivation {
    name = "impure";
    __impure = true; # marks this derivation as impure
    outputHashAlgo = "sha256";
    outputHashMode = "recursive";
    buildCommand = "date > $out";
  };

Some important characteristics:

* This requires the 'impure-derivations' experimental feature.

* Impure derivations are not "cached". Thus, running "nix-build" on
  the example above multiple times will cause a rebuild every time.

* They are implemented similar to CA derivations, i.e. the output is
  moved to a content-addressed path in the store. The difference is
  that we don't register a realisation in the Nix database.

* Pure derivations are not allowed to depend on impure derivations. In
  the future fixed-output derivations will be allowed to depend on
  impure derivations, thus forming an "impurity barrier" in the
  dependency graph.

* When sandboxing is enabled, impure derivations can access the
  network in the same way as fixed-output derivations. In relaxed
  sandboxing mode, they can access the local filesystem.
2022-03-31 13:43:20 +02:00
Eelco Dolstra
50c229ad9a
Use wantOutput
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-03-25 08:02:49 +01:00
Eelco Dolstra
fe5509df9a Only return wanted outputs 2022-03-24 23:24:48 +01:00
John Ericson
a544ed7684 Generalize DerivationType in preparation for impure derivations 2022-03-18 14:59:56 +00:00
John Ericson
197feed51d Clean up DerivationOutput, and headers
1. `DerivationOutput` now as the `std::variant` as a base class. And the
   variants are given hierarchical names under `DerivationOutput`.

   In 8e0d0689be @matthewbauer and I
   didn't know a better idiom, and so we made it a field. But this sort
   of "newtype" is anoying for literals downstream.

   Since then we leaned the base class, inherit the constructors trick,
   e.g. used in `DerivedPath`. Switching to use that makes this more
   ergonomic, and consistent.

2. `store-api.hh` and `derivations.hh` are now independent.

   In bcde5456cc I swapped the dependency,
   but I now know it is better to just keep on using incomplete types as
   much as possible for faster compilation and good separation of
   concerns.
2022-03-17 22:35:53 +00:00
John Ericson
678d1c2aa0 Factor out a LogStore interface
Continue progress on #5729.

Just as I hoped, this uncovered an issue: the daemon protocol is missing
a way to query build logs. This doesn't effect `unix://`, but does
effect `ssh://`. A FIXME is left for this, so we come back to it later.
2022-03-11 13:32:16 +00:00
Eelco Dolstra
a4604f1928 Add Store::buildPathsWithResults()
This function is like buildPaths(), except that it returns a vector of
BuildResults containing the exact statuses and output paths of each
derivation / substitution. This is convenient for functions like
Installable::build(), because they then don't need to do another
series of calls to get the outputs of CA derivations. It's also a
precondition to impure derivations, where we *can't* query the output
of those derivations since they're not stored in the Nix database.

Note that PathSubstitutionGoal can now also return a BuildStatus.
2022-03-08 19:56:34 +01:00
John Ericson
6636202356 Factor out a GcStore interface
Starts progress on #5729.

The idea is that we should not have these default methods throwing
"unimplemented". This is a small step in that direction.

I kept `addTempRoot` because it is a no-op, rather than failure. Also,
as a practical matter, it is called all over the place, while doing
other tasks, so the downcasting would be annoying.

Maybe in the future I could move the "real" `addTempRoot` to `GcStore`,
and the existing usecases use a `tryAddTempRoot` wrapper to downcast or
do nothing, but I wasn't sure whether that was a good idea so with a
bias to less churn I didn't do it yet.
2022-03-03 19:01:25 +00:00
Eelco Dolstra
b91500a14e Fix clang warning 2022-02-28 15:48:26 +01:00
Eelco Dolstra
a949673a5b Fix Darwin build
Fixes #6169
2022-02-28 15:48:26 +01:00
Eelco Dolstra
df552ff53e Remove std::string alias (for real this time)
Also use std::string_view in a few more places.
2022-02-25 16:13:02 +01:00
Eelco Dolstra
2d6d9a28eb addToStoreFromDump(): Take std::string_view 2022-02-16 11:02:35 +01:00
Sebastian Ullrich
c437e1326d Fix using sandbox without user namespaces 2022-02-12 16:28:36 +01:00
Eelco Dolstra
d62a9390fc Get rid of std::shared_ptr<std::string> and ref<std::string>
These were needed back in the pre-C++11 era because we didn't have
move semantics. But now we do.
2022-01-18 11:12:30 +01:00
Eelco Dolstra
8a446aff75
Merge pull request #5898 from layus/repair-path-links
Make --repair-path also repair corrupt optimised links
2022-01-11 14:14:44 +01:00
Guillaume Maudoux
9f9f39a24b Prefer RepairFlag over bool when applicable 2022-01-11 13:38:28 +01:00
Guillaume Maudoux
e9a4abdb5d Make --repair-path also repair corrupt optimised links
There already existed a smoke test for the link content length,
but it appears that there exists some corruptions pernicious enough
to replace the file content with zeros, and keeping the same length.

--repair-path now goes as far as checking the content of the link,
making it true to its name and actually repairing the path for such
coruption cases.
2022-01-11 11:57:45 +01:00
Eelco Dolstra
19fd6e585d 'target' points to 'source' 2022-01-10 16:52:25 +01:00
Eelco Dolstra
df11e75d0e Merge branch 'recursive-nix-fix' of https://github.com/L-as/nix 2022-01-10 16:51:59 +01:00
Eelco Dolstra
46d86e06ba Simplify 2021-12-13 20:28:53 +01:00
Eelco Dolstra
9529de2eed Merge branch 'arm32-personality' of https://github.com/cleverca22/nix 2021-12-13 19:58:40 +01:00
Las Safin
8388d2c7c6
Make recursive-nix work even when not privileged
Before this, `setns` would fail when switching to the mount namespace,
since we did not have the privileges to do so when not root.

Closes #5360
2021-11-25 11:15:11 +00:00
Alex Shabalin
a18d9269a5 Fix build warnings on MacOS 2021-11-19 15:22:31 +01:00
Eelco Dolstra
6463eaca14
Merge pull request #5472 from NixOS/async-realisation-substitution
async realisation substitution
2021-11-16 12:54:20 +01:00
Eelco Dolstra
bceda30498 Typo 2021-11-12 13:41:15 +01:00
Eelco Dolstra
67179472df
Merge pull request #5494 from tweag/balsoft/allow-references-in-addPath
Allow references in addPath
2021-11-09 15:57:39 +01:00
Alexander Bantyev
0b005bc9d6
addToStore, addToStoreFromDump: refactor: pass refs by const reference
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
2021-11-09 12:24:49 +03:00
regnat
96670ed216 Expose an async interface for queryRealisation
Doesn’t change much so far because everything is still using it
synchronously, but should allow the binary cache to fetch stuff in
parallel
2021-11-03 06:51:34 +01:00
Eelco Dolstra
a7d4f3411e Merge remote-tracking branch 'origin/master' into non-blocking-gc 2021-10-28 14:56:55 +02:00
regnat
af99941279 Make experimental-features a proper type
Rather than having them plain strings scattered through the whole
codebase, create an enum containing all the known experimental features.

This means that
- Nix can now `warn` when an unkwown experimental feature is passed
  (making it much nicer to spot typos and spot deprecated features)
- It’s now easy to remove a feature altogether (once the feature isn’t
  experimental anymore or is dropped) by just removing the field for the
  enum and letting the compiler point us to all the now invalid usages
  of it.
2021-10-26 07:02:31 +02:00
Alexander Bantyev
b9234142f5
addToStore, addToStoreFromDump: add references argument
Allow to pass a set of references to be added as info to the added paths.
2021-10-23 21:30:51 +03:00
Eelco Dolstra
8614cf1334 Non-blocking garbage collector
The garbage collector no longer blocks other processes from
adding/building store paths or adding GC roots. To prevent the
collector from deleting store paths just added by another process,
processes need to connect to the garbage collector via a Unix domain
socket to register new temporary roots.
2021-10-13 12:12:44 +02:00
regnat
7466048d39 (partially) Revert "Don't copy in rethrow"
This reverts some parts of commit
8430a8f086 which was trying to rethrow
some exceptions while we weren’t in the context of a `catch` block,
causing some weird “terminate called without an active exception”
errors.

Fix #5368
2021-10-11 10:51:22 +02:00
Eelco Dolstra
c6718a9d95 Don't reset the logger in a vfork
9c766a40cb broke logging from the
daemon, because commonChildInit is called when starting the build hook
in a vfork, so it ends up resetting the parent's logger. So don't
vfork.

It might be best to get rid of vfork altogether, but that may cause
problems, e.g. when we call an external program like git from the
evaluator.
2021-10-06 13:54:59 +02:00
Eelco Dolstra
3b7f4c7d9d Add FIXME about ptsname 2021-10-05 11:04:46 +02:00
Eelco Dolstra
ef34fd0656 scanForReferences(): Use a StorePathSet 2021-10-04 13:47:38 +02:00
Sergei Trofimovich
621aa65325 local-derivation-goal.cc: downgrade "warning: rewriting hashes in..." down to debug
Before the changes when building the whole system with
`contentAddressedByDefault = true;` we get many noninformative messages:

    $ nix build -f nixos system --keep-going
    ...
    warning: rewriting hashes in '/nix/store/...-clang-11.1.0.drv.chroot/nix/store/...-11.1.0'; cross fingers
    warning: rewriting hashes in '/nix/store/...-clang-11.1.0.drv.chroot/nix/store/...-11.1.0-dev'; cross fingers
    warning: rewriting hashes in '/nix/store/...-clang-11.1.0.drv.chroot/nix/store/...-11.1.0-python'; cross fingers
    error: 2 dependencies of derivation '/nix/store/...-hub-2.14.2.drv' failed to build
    warning: rewriting hashes in '/nix/store/...-subversion-1.14.1.drv.chroot/nix/store/...-subversion-1.14.1-dev'; cross fingers
    warning: rewriting hashes in '/nix/store/...-subversion-1.14.1.drv.chroot/nix/store/...-subversion-1.14.1-man'; cross fingers
    ...

Let's downgrade these messages down to debug().
2021-10-01 17:41:44 +01:00
John Ericson
242f9bf3dc std::visit by reference
I had started the trend of doing `std::visit` by value (because a type
error once mislead me into thinking that was the only form that
existed). While the optomizer in principle should be able to deal with
extra coppying or extra indirection once the lambdas inlined, sticking
with by reference is the conventional default. I hope this might even
improve performance.
2021-09-30 21:35:09 +00:00
Eelco Dolstra
6b5efeafd1
Merge pull request #5295 from Ma27/bmcheck-fod-mismatch
build: also throw hash-mismatch errors if `buildMode == bmCheck`
2021-09-29 10:13:45 +02:00
Maximilian Bosch
bb1a851bcf
build: also throw hash-mismatch errors if buildMode == bmCheck
This actually bit me quite recently in `nixpkgs` because I assumed that
`nix-build --check` would also error out if hashes don't match anymore[1]
and so I wrongly assumed that I couldn't reproduce the mismatch error.

The fix is rather simple, during the output registration a so-called
`delayedException` is instantiated e.g. if a FOD hash-mismatch occurs.
However, in case of `nix-build --check` (or `--rebuild` in case of `nix
build`), the code-path where this exception is thrown will never be
reached.

By adding that check to the if-clause that causes an early exit in case
of `bmCheck`, the issue is gone. Also added a (previously failing)
test-case to demonstrate the problem.

[1] https://github.com/NixOS/nixpkgs/pull/139238, the underlying issue
    was that `nix-prefetch-git` returns different hashes than `fetchgit`
    because the latter one fetches submodules by default.
2021-09-27 15:44:39 +02:00