max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2024-11-30 01:26:15 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge branch 'path-info' into ca-drv-exotic

Browse source
This commit is contained in:
John Ericson 2021-10-01 17:25:22 +00:00
commit edf67e1508
24 changed files with 638 additions and 106 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

504
doc/manual/src/release-notes/rl-2.4.md
View file

@ -1,8 +1,498 @@
# Release 2.4 (202X-XX-XX)
# Release 2.4 (2021-10-XX)
- It is now an error to modify the `plugin-files` setting via a
command-line flag that appears after the first non-flag argument
to any command, including a subcommand to `nix`. For example,
`nix-instantiate default.nix --plugin-files ""` must now become
`nix-instantiate --plugin-files "" default.nix`.
- Plugins that add new `nix` subcommands are now actually respected.
This is the first release in more than two years and is the result of
more than 2800 commits from 195 contributors since release 2.3.
## Highlights
* Nix's **error messages** have been improved a lot. For instance,
evaluation errors now point out the location of the error:
```
$ nix build
error: undefined variable 'bzip3'
at /nix/store/449lv242z0zsgwv95a8124xi11sp419f-source/flake.nix:88:13:
87| [ curl
88| bzip3 xz brotli editline
| ^
89| openssl sqlite
```
* The **`nix` command** has seen a lot of work and is now almost at
feature parity with the old command-line interface (the `nix-*`
commands). It aims to be [more modern, consistent and pleasant to
use](../contributing/cli-guideline.md) than the old CLI. It is still
marked as experimental but its interface should not change much
anymore in future releases.
* **Flakes** are a new format to package Nix-based projects in a more
discoverable, composable, consistent and reproducible way. A flake
is just a repository or tarball containing a file named `flake.nix`
that specifies dependencies on other flakes and returns any Nix
assets such as packages, Nixpkgs overlays, NixOS modules or CI
tests. The new `nix` CLI is primarily based around flakes; for
example, a command like `nix run nixpkgs#hello` runs the `hello`
application from the `nixpkgs` flake.
Flakes are currently marked as experimental. For an introduction,
see [this blog
post](https://www.tweag.io/blog/2020-05-25-flakes/). For detailed
information about flake syntax and semantics, see the [`nix flake`
manual page](../command-ref/new-cli/nix3-flake.md).
* Nix's store can now be **content-addressed**, meaning that the hash
component of a store path is the hash of the path's
contents. Previously Nix could only build **input-addressed** store
paths, where the hash is computed from the derivation dependency
graph. Content-addressing allows deduplication, early cutoff in
build systems, and unprivileged closure copying. This is still [an
experimental
feature](https://discourse.nixos.org/t/content-addressed-nix-call-for-testers/12881).
* The Nix manual has been converted into Markdown, making it easier to
contribute. In addition, every `nix` subcommand now has a manual
page, documenting every option.
* A new setting that allows **experimental features** to be enabled
selectively. This allows us to merge unstable features into Nix more
quickly and do more frequent releases.
## Other features
* There are many new `nix` subcommands:
- `nix develop` is intended to replace `nix-shell`. It has a number
of new features:
* It automatically sets the output environment variables (such as
`$out`) to writable locations (such as `./outputs/out`).
* It can store the environment in a profile. This is useful for
offline work.
* It can run specific phases directly. For instance, `nix develop
--build` runs `buildPhase`.
- It allows dependencies in the Nix store to be "redirected" to
arbitrary directories using the `--redirect` flag. This is
useful if you want to hack on a package *and* some of its
dependencies at the same time.
- `nix print-dev-env` prints the environment variables and bash
functions defined by a derivation. This is useful for users of
other shells than bash (especially with `--json`).
- `nix shell` was previously named `nix run` and is intended to
replace `nix-shell -p`, but without the `stdenv` overhead. It
simply starts a shell where some packages have been added to
`$PATH`.
- `nix run` (not to be confused with the old subcommand that has
been renamed to `nix shell`) runs an "app", a flake output that
specifies a command to run, or an eponymous program from a
package. For example, `nix run nixpkgs#hello` runs the `hello`
program from the `hello` package in `nixpkgs`.
- `nix flake` is the container for flake-related operations, such as
creating a new flake, querying the contents of a flake or updating
flake lock files.
- `nix registry` allows you to query and update the flake registry,
which maps identifiers such as `nixpkgs` to concrete flake URLs.
- `nix profile` is intended to replace `nix-env`. Its main advantage
is that it keeps track of the provenance of installed packages
(e.g. exactly which flake version a package came from). It also
has some helpful subcommands:
* `nix profile history` shows what packages were added, upgraded
or removed between each version of a profile.
* `nix profile diff-closures` shows the changes between the
closures of each version of a profile. This allows you to
discover the addition or removal of dependencies or size
changes.
**Warning**: after a profile has been updated using `nix profile`,
it is no longer usable with `nix-env`.
- `nix store diff-closures` shows the differences between the
closures of two store paths in terms of the versions and sizes of
dependencies in the closures.
- `nix store make-content-addressable` rewrites an arbitrary closure
to make it content-addressed. Such paths can be copied into other
stores without requiring signatures.
- `nix bundle` uses the [`nix-bundle`
program](https://github.com/matthewbauer/nix-bundle) to convert a
closure into a self-extracting executable.
- Various other replacements for the old CLI, e.g. `nix store gc`,
`nix store delete`, `nix store repair`, `nix nar dump-path`, `nix
store prefetch-file`, `nix store prefetch-tarball`, `nix key` and
`nix daemon`.
* Nix now has an **evaluation cache** for flake outputs. For example,
a second invocation of the command `nix run nixpkgs#firefox` will
not need to evaluate the `firefox` attribute because it's already in
the evaluation cache. This is made possible by the hermetic
evaluation model of flakes.
* The new `--offline` flag disables substituters and causes all
locally cached tarballs and repositories to be considered
up-to-date.
* The new `--refresh` flag causes all locally cached tarballs and
repositories to be considered out-of-date.
* Many `nix` subcommands now have a `--json` option to produce
machine-readable output.
* `nix repl` has a new `:doc` command to show documentation about
builtin functions (e.g. `:doc builtins.map`).
* Binary cache stores now have an option `index-debug-info` to create
an index of DWARF debuginfo files for use by
[`dwarffs`](https://github.com/edolstra/dwarffs).
* To support flakes, Nix now has an extensible mechanism for fetching
source trees. Currently it has the following backends:
* Git repositories
* Mercurial repositories
* GitHub and GitLab repositories (an optimisation for faster
fetching than Git)
* Tarballs
* Arbitrary directories
The fetcher infrastructure is exposed via flake input specifications
and via the `fetchTree` built-in.
* **Languages changes**: the only new language feature is that you can
now have antiquotations in paths, e.g. `./${foo}` instead of `./. +
foo`.
* **New built-in functions**:
- `builtins.fetchTree` allows fetching a source tree using any
backends supported by the fetcher infrastructure. It subsumes the
functionality of existing built-ins like `fetchGit`,
`fetchMercurial` and `fetchTarball`.
- `builtins.getFlake` fetches a flake and returns its output
attributes. This function should not be used inside flakes! Use
flake inputs instead.
- `builtins.floor` and `builtins.ceil` round a floating-point number
down and up, respectively.
* Experimental support for recursive Nix. This means that Nix
derivations can now call Nix to build other derivations. This is not
in a stable state yet and not well
[documented](https://github.com/NixOS/nix/commit/c4d7c76b641d82b2696fef73ce0ac160043c18da).
* The new experimental feature `no-url-literals` disables URL
literals. This helps to implement [RFC
45](https://github.com/NixOS/rfcs/pull/45).
* Nix now uses `libarchive` to decompress and unpack tarballs and zip
files, so `tar` is no longer required.
* The priority of substituters can now be overridden using the
`priority` substituter setting (e.g. `--substituters
'http://cache.nixos.org?priority=100 daemon?priority=10'`).
* `nix edit` now supports non-derivation attributes, e.g. `nix edit
.#nixosConfigurations.bla`.
* The `nix` command now provides command line completion for `bash`,
`zsh` and `fish`. Since the support for getting completions is built
into `nix`, it's easy to add support for other shells.
* The new `--log-format` flag selects what Nix's output looks like. It
defaults to a terse progress indicator. There is a new
`internal-json` output format for use by other programs.
* `nix eval` has a new `--apply` flag that applies a function to the
evaluation result.
* `nix eval` has a new `--write-to` flag that allows it to write a
nested attribute set of string leaves to a corresponding directory
tree.
* Memory improvements: many operations that add paths to the store or
copy paths between stores now run in constant memory.
* Many `nix` commands now support the flag `--derivation` to operate
on a `.drv` file itself instead of its outputs.
* There is a new store called `dummy://` that does not support
building or adding paths. This is useful if you want to use the Nix
evaluator but don't have a Nix store.
* The `ssh-ng://` store now allows substituting paths on the remote,
as `ssh://` already did.
* When auto-calling a function with an ellipsis, all arguments are now
passed.
* New `nix-shell` features:
- It preserves the `PS1` environment variable if
`NIX_SHELL_PRESERVE_PROMPT` is set.
- With `-p`, it passes any `--arg`s as Nixpkgs arguments.
- Support for structured attributes.
* `nix-prefetch-url` has a new `--executable` flag.
* On `x86_64` systems, [`x86_64` microarchitecture
levels](https://lwn.net/Articles/844831/) are mapped to additional
system types (e.g. `x86_64-v1-linux`).
* The new `--eval-store` flag allows you to use a different store for
evaluation than for building or storing the build result. This is
primarily useful when you want to query whether something exists in
a read-only store, such as a binary cache:
```
# nix path-info --json --store https://cache.nixos.org \
--eval-store auto nixpkgs#hello
```
(Here `auto` indicates the local store.)
* The Nix daemon has a new low-latency mechanism for copying
closures. This is useful when building on remote stores such as
`ssh-ng://`.
* Plugins can now register `nix` subcommands.
## Incompatible changes
* The `nix` command is now marked as an experimental feature. This
means that you need to add
> experimental-features = nix-command
to your `nix.conf` if you want to use it, or pass
`--extra-experimental-features nix-command` on the command line.
* The old `nix run` has been renamed to `nix shell` (and there is a
new `nix run` that does something else, as described above).
* It is now an error to modify the `plugin-files` setting via a
command-line flag that appears after the first non-flag argument to
any command, including a subcommand to `nix`. For example,
`nix-instantiate default.nix --plugin-files ""` must now become
`nix-instantiate --plugin-files "" default.nix`.
* We no longer release source tarballs. If you want to build from
source, please build from the tags in the Git repository.
## Contributors
This release has contributions from
Adam Höse,
Albert Safin,
Alex Kovar,
Alex Zero,
Alexander Bantyev,
Alexandre Esteves,
Alyssa Ross,
Anatole Lucet,
Anders Kaseorg,
Andreas Rammhold,
Antoine Eiche,
Antoine Martin,
Arnout Engelen,
Arthur Gautier,
aszlig,
Ben Burdette,
Benjamin Hipple,
Bernardo Meurer,
Björn Gohla,
Bjørn Forsman,
Bob van der Linden,
Brian Leung,
Brian McKenna,
Brian Wignall,
Bruce Toll,
Bryan Richter,
Calle Rosenquist,
Calvin Loncaric,
Carlo Nucera,
Carlos D'Agostino,
Chaz Schlarp,
Christian Höppner,
Christian Kampka,
Chua Hou,
Chuck,
Cole Helbling,
Daiderd Jordan,
Dan Callahan,
Dani,
Daniel Fitzpatrick,
Danila Fedorin,
Daniël de Kok,
Danny Bautista,
DavHau,
David McFarland,
Dima,
Domen Kožar,
Dominik Schrempf,
Dominique Martinet,
dramforever,
Dustin DeWeese,
edef,
Eelco Dolstra,
Emilio Karakey,
Emily,
Eric Culp,
Ersin Akinci,
Fabian Möller,
Farid Zakaria,
Federico Pellegrin,
Finn Behrens,
Florian Franzen,
Félix Baylac-Jacqué,
Gabriel Gonzalez,
Geoff Reedy,
Georges Dubus,
Graham Christensen,
Greg Hale,
Greg Price,
Gregor Kleen,
Gregory Hale,
Griffin Smith,
Guillaume Bouchard,
Harald van Dijk,
illustris,
Ivan Zvonimir Horvat,
Jade,
Jake Waksbaum,
jakobrs,
James Ottaway,
Jan Tojnar,
Janne Heß,
Jaroslavas Pocepko,
Jarrett Keifer,
Jeremy Schlatter,
Joachim Breitner,
Joe Hermaszewski,
Joe Pea,
John Ericson,
Jonathan Ringer,
Josef Kemetmüller,
Joseph Lucas,
Jude Taylor,
Julian Stecklina,
Julien Tanguy,
Jörg Thalheim,
Kai Wohlfahrt,
keke,
Keshav Kini,
Kevin Quick,
Kevin Stock,
Kjetil Orbekk,
Krzysztof Gogolewski,
kvtb,
Lars Mühmel,
Leonhard Markert,
Lily Ballard,
Linus Heckemann,
Lorenzo Manacorda,
Lucas Desgouilles,
Lucas Franceschino,
Lucas Hoffmann,
Luke Granger-Brown,
Madeline Haraj,
Marwan Aljubeh,
Mat Marini,
Mateusz Piotrowski,
Matthew Bauer,
Matthew Kenigsberg,
Mauricio Scheffer,
Maximilian Bosch,
Michael Adler,
Michael Bishop,
Michael Fellinger,
Michael Forney,
Michael Reilly,
mlatus,
Mykola Orliuk,
Nathan van Doorn,
Naïm Favier,
ng0,
Nick Van den Broeck,
Nicolas Stig124 Formichella,
Niels Egberts,
Niklas Hambüchen,
Nikola Knezevic,
oxalica,
p01arst0rm,
Pamplemousse,
Patrick Hilhorst,
Paul Opiyo,
Pavol Rusnak,
Peter Kolloch,
Philipp Bartsch,
Philipp Middendorf,
Piotr Szubiakowski,
Profpatsch,
Puck Meerburg,
Ricardo M. Correia,
Rickard Nilsson,
Robert Hensing,
Robin Gloster,
Rodrigo,
Rok Garbas,
Ronnie Ebrin,
Rovanion Luckey,
Ryan Burns,
Ryan Mulligan,
Ryne Everett,
Sam Doshi,
Sam Lidder,
Samir Talwar,
Samuel Dionne-Riel,
Sebastian Ullrich,
Sergei Trofimovich,
Sevan Janiyan,
Shao Cheng,
Shea Levy,
Silvan Mosberger,
Stefan Frijters,
Stefan Jaax,
sternenseemann,
Steven Shaw,
Stéphan Kochen,
SuperSandro2000,
Suraj Barkale,
Taeer Bar-Yam,
Thomas Churchman,
Théophane Hufschmitt,
Timothy DeHerrera,
Timothy Klim,
Tobias Möst,
Tobias Pflug,
Tom Bereknyei,
Travis A. Everett,
Ujjwal Jain,
Vladimír Čunát,
Wil Taylor,
Will Dietz,
Yaroslav Bolyukin,
Yestin L. Harrison,
YI,
Yorick van Pelt,
Yuriy Taraday and
zimbatm.

4
src/libcmd/command.cc
View file

@ -203,10 +203,10 @@ void MixProfile::updateProfile(const BuiltPaths & buildables)
for (auto & buildable : buildables) {
std::visit(overloaded {
[&](BuiltPath::Opaque bo) {
[&](const BuiltPath::Opaque & bo) {
result.push_back(bo.path);
},
[&](BuiltPath::Built bfd) {
[&](const BuiltPath::Built & bfd) {
for (auto & output : bfd.outputs) {
result.push_back(output.second);
}

14
src/libcmd/installables.cc
View file

@ -697,13 +697,13 @@ std::shared_ptr<Installable> SourceExprCommand::parseInstallable(
BuiltPaths getBuiltPaths(ref<Store> evalStore, ref<Store> store, const DerivedPaths & hopefullyBuiltPaths)
{
BuiltPaths res;
for (auto & b : hopefullyBuiltPaths)
for (const auto & b : hopefullyBuiltPaths)
std::visit(
overloaded{
[&](DerivedPath::Opaque bo) {
[&](const DerivedPath::Opaque & bo) {
res.push_back(BuiltPath::Opaque{bo.path});
},
[&](DerivedPath::Built bfd) {
[&](const DerivedPath::Built & bfd) {
OutputPathMap outputs;
auto drv = evalStore->readDerivation(bfd.drvPath);
auto outputHashes = staticOutputHashes(*evalStore, drv); // FIXME: expensive
@ -823,10 +823,10 @@ StorePathSet toDerivations(
{
StorePathSet drvPaths;
for (auto & i : installables)
for (auto & b : i->toDerivedPaths())
for (const auto & i : installables)
for (const auto & b : i->toDerivedPaths())
std::visit(overloaded {
[&](DerivedPath::Opaque bo) {
[&](const DerivedPath::Opaque & bo) {
if (!useDeriver)
throw Error("argument '%s' did not evaluate to a derivation", i->what());
auto derivers = store->queryValidDerivers(bo.path);
@ -835,7 +835,7 @@ StorePathSet toDerivations(
// FIXME: use all derivers?
drvPaths.insert(*derivers.begin());
},
[&](DerivedPath::Built bfd) {
[&](const DerivedPath::Built & bfd) {
drvPaths.insert(bfd.drvPath);
},
}, b.raw());

6
src/libexpr/primops.cc
View file

@ -1178,7 +1178,7 @@ static void prim_derivationStrict(EvalState & state, const Pos & pos, Value * *
// hash per output.
auto hashModulo = hashDerivationModulo(*state.store, Derivation(drv), true);
std::visit(overloaded {
[&](Hash h) {
[&](Hash & h) {
for (auto & i : outputs) {
auto outPath = state.store->makeOutputPath(i, h, drvName);
drv.env[i] = state.store->printStorePath(outPath);
@ -1190,11 +1190,11 @@ static void prim_derivationStrict(EvalState & state, const Pos & pos, Value * *
});
}
},
[&](CaOutputHashes) {
[&](CaOutputHashes &) {
// Shouldn't happen as the toplevel derivation is not CA.
assert(false);
},
[&](DeferredHash _) {
[&](DeferredHash &) {
for (auto & i : outputs) {
drv.outputs.insert_or_assign(i,
DerivationOutput {

6
src/libstore/build/entry-points.cc
View file

@ -11,12 +11,12 @@ void Store::buildPaths(const std::vector<DerivedPath> & reqs, BuildMode buildMod
Worker worker(*this, evalStore ? *evalStore : *this);
Goals goals;
for (auto & br : reqs) {
for (const auto & br : reqs) {
std::visit(overloaded {
[&](DerivedPath::Built bfd) {
[&](const DerivedPath::Built & bfd) {
goals.insert(worker.makeDerivationGoal(bfd.drvPath, bfd.outputs, buildMode));
},
[&](DerivedPath::Opaque bo) {
[&](const DerivedPath::Opaque & bo) {
goals.insert(worker.makePathSubstitutionGoal(bo.path, buildMode == bmRepair ? Repair : NoRepair));
},
}, br.raw());

20
src/libstore/build/local-derivation-goal.cc
View file

@ -1094,10 +1094,10 @@ void LocalDerivationGoal::writeStructuredAttrs()
static StorePath pathPartOfReq(const DerivedPath & req)
{
return std::visit(overloaded {
[&](DerivedPath::Opaque bo) {
[&](const DerivedPath::Opaque & bo) {
return bo.path;
},
[&](DerivedPath::Built bfd) {
[&](const DerivedPath::Built & bfd) {
return bfd.drvPath;
},
}, req.raw());
@ -2155,8 +2155,8 @@ void LocalDerivationGoal::registerOutputs()
/* Since we'll use the already installed versions of these, we
can treat them as leaves and ignore any references they
have. */
[&](AlreadyRegistered _) { return StringSet {}; },
[&](PerhapsNeedToRegister refs) {
[&](const AlreadyRegistered &) { return StringSet {}; },
[&](const PerhapsNeedToRegister & refs) {
StringSet referencedOutputs;
/* FIXME build inverted map up front so no quadratic waste here */
for (auto & r : refs.refs)
@ -2192,11 +2192,11 @@ void LocalDerivationGoal::registerOutputs()
};
std::optional<StorePathSet> referencesOpt = std::visit(overloaded {
[&](AlreadyRegistered skippedFinalPath) -> std::optional<StorePathSet> {
[&](const AlreadyRegistered & skippedFinalPath) -> std::optional<StorePathSet> {
finish(skippedFinalPath.path);
return std::nullopt;
},
[&](PerhapsNeedToRegister r) -> std::optional<StorePathSet> {
[&](const PerhapsNeedToRegister & r) -> std::optional<StorePathSet> {
return r.refs;
},
}, outputReferencesIfUnregistered.at(outputName));
@ -2262,10 +2262,10 @@ void LocalDerivationGoal::registerOutputs()
std::string oldHashPart { scratchPath.hashPart() };
HashModuloSink caSink { outputHash.hashType, oldHashPart };
std::visit(overloaded {
[&](TextHashMethod _) {
[&](const TextHashMethod &) {
readFile(actualPath, caSink);
},
[&](FileIngestionMethod m2) {
[&](const FileIngestionMethod & m2) {
switch (m2) {
case FileIngestionMethod::Recursive:
dumpPath(actualPath, caSink);
@ -2312,7 +2312,7 @@ void LocalDerivationGoal::registerOutputs()
};
ValidPathInfo newInfo = std::visit(overloaded {
[&](DerivationOutputInputAddressed output) {
[&](const DerivationOutputInputAddressed & output) {
/* input-addressed case */
auto requiredFinalPath = output.path;
/* Preemptively add rewrite rule for final hash, as that is
@ -2328,7 +2328,7 @@ void LocalDerivationGoal::registerOutputs()
static_cast<PathReferences<StorePath> &>(newInfo0) = rewriteRefs();
return newInfo0;
},
[&](DerivationOutputCAFixed dof) {
[&](const DerivationOutputCAFixed & dof) {
auto wanted = getContentAddressHash(dof.ca);
auto newInfo0 = newInfoFromCA(DerivationOutputCAFloating {

26
src/libstore/content-address.cc
View file

@ -23,7 +23,7 @@ std::string makeFileIngestionPrefix(FileIngestionMethod m)
std::string makeContentAddressingPrefix(ContentAddressMethod m) {
return std::visit(overloaded {
[](TextHashMethod _) -> std::string { return "text:"; },
[](TextHashMethod) -> std::string { return "text:"; },
[](FileIngestionMethod m2) {
/* Not prefixed for back compat with things that couldn't produce text before. */
return makeFileIngestionPrefix(m2);
@ -52,11 +52,11 @@ std::string makeFixedOutputCA(FileIngestionMethod method, const Hash & hash)
std::string renderContentAddress(ContentAddress ca)
{
return std::visit(overloaded {
[](TextHash th) {
[](TextHash & th) {
return "text:"
+ th.hash.to_string(Base32, true);
},
[](FixedOutputHash fsh) {
[](FixedOutputHash & fsh) {
return "fixed:"
+ makeFileIngestionPrefix(fsh.method)
+ fsh.hash.to_string(Base32, true);
@ -128,12 +128,12 @@ ContentAddress parseContentAddress(std::string_view rawCa) {
auto hashType = hashType_; // work around clang bug
return std::visit(overloaded {
[&](TextHashMethod _) {
[&](TextHashMethod &) {
return ContentAddress(TextHash {
.hash = Hash::parseNonSRIUnprefixed(rest, hashType)
});
},
[&](FileIngestionMethod fim) {
[&](FileIngestionMethod & fim) {
return ContentAddress(FixedOutputHash {
.method = fim,
.hash = Hash::parseNonSRIUnprefixed(rest, hashType),
@ -185,10 +185,10 @@ ContentAddressWithReferences contentAddressFromMethodHashAndRefs(
ContentAddressMethod getContentAddressMethod(const ContentAddressWithReferences & ca)
{
return std::visit(overloaded {
[](TextInfo th) -> ContentAddressMethod {
[](const TextInfo & th) -> ContentAddressMethod {
return TextHashMethod {};
},
[](FixedOutputInfo fsh) -> ContentAddressMethod {
[](const FixedOutputInfo & fsh) -> ContentAddressMethod {
return fsh.method;
},
}, ca);
@ -197,10 +197,10 @@ ContentAddressMethod getContentAddressMethod(const ContentAddressWithReferences
Hash getContentAddressHash(const ContentAddress & ca)
{
return std::visit(overloaded {
[](TextHash th) {
[](const TextHash & th) {
return th.hash;
},
[](FixedOutputHash fsh) {
[](const FixedOutputHash & fsh) {
return fsh.hash;
},
}, ca);
@ -208,10 +208,10 @@ Hash getContentAddressHash(const ContentAddress & ca)
ContentAddressWithReferences caWithoutRefs(const ContentAddress & ca) {
return std::visit(overloaded {
[&](TextHash h) -> ContentAddressWithReferences {
[&](const TextHash & h) -> ContentAddressWithReferences {
return TextInfo { h, {}};
},
[&](FixedOutputHash h) -> ContentAddressWithReferences {
[&](const FixedOutputHash & h) -> ContentAddressWithReferences {
return FixedOutputInfo { h, {}};
},
}, ca);
@ -220,10 +220,10 @@ ContentAddressWithReferences caWithoutRefs(const ContentAddress & ca) {
Hash getContentAddressHash(const ContentAddressWithReferences & ca)
{
return std::visit(overloaded {
[](TextInfo th) {
[](const TextInfo & th) {
return th.hash;
},
[](FixedOutputInfo fsh) {
[](const FixedOutputInfo & fsh) {
return fsh.hash;
},
}, ca);

4
src/libstore/daemon.cc
View file

@ -396,7 +396,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
FramedSource source(from);
// TODO this is essentially RemoteStore::addCAToStore. Move it up to Store.
return std::visit(overloaded {
[&](TextHashMethod _) {
[&](TextHashMethod &) {
if (hashType != htSHA256)
throw UnimplementedError("Only SHA-256 is supported for adding text-hashed data, but '%1' was given",
printHashType(hashType));
@ -405,7 +405,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
auto path = store->addTextToStore(name, contents, refs, repair);
return store->queryPathInfo(path);
},
[&](FileIngestionMethod fim) {
[&](FileIngestionMethod & fim) {
if (!refs.empty())
throw UnimplementedError("cannot yet have refs with flat or nar-hashed data");
auto path = store->addToStoreFromDump(source, name, fim, hashType, repair);

44
src/libstore/derivations.cc
View file

@ -11,18 +11,18 @@ namespace nix {
std::optional<StorePath> DerivationOutput::path(const Store & store, std::string_view drvName, std::string_view outputName) const
{
return std::visit(overloaded {
[](DerivationOutputInputAddressed doi) -> std::optional<StorePath> {
[](const DerivationOutputInputAddressed & doi) -> std::optional<StorePath> {
return { doi.path };
},
[&](DerivationOutputCAFixed dof) -> std::optional<StorePath> {
[&](const DerivationOutputCAFixed & dof) -> std::optional<StorePath> {
return {
dof.path(store, drvName, outputName)
};
},
[](DerivationOutputCAFloating dof) -> std::optional<StorePath> {
[](const DerivationOutputCAFloating & dof) -> std::optional<StorePath> {
return std::nullopt;
},
[](DerivationOutputDeferred) -> std::optional<StorePath> {
[](const DerivationOutputDeferred &) -> std::optional<StorePath> {
return std::nullopt;
},
}, output);
@ -330,22 +330,22 @@ string Derivation::unparse(const Store & store, bool maskOutputs,
if (first) first = false; else s += ',';
s += '('; printUnquotedString(s, i.first);
std::visit(overloaded {
[&](DerivationOutputInputAddressed doi) {
[&](const DerivationOutputInputAddressed & doi) {
s += ','; printUnquotedString(s, maskOutputs ? "" : store.printStorePath(doi.path));
s += ','; printUnquotedString(s, "");
s += ','; printUnquotedString(s, "");
},
[&](DerivationOutputCAFixed dof) {
[&](const DerivationOutputCAFixed & dof) {
s += ','; printUnquotedString(s, maskOutputs ? "" : store.printStorePath(dof.path(store, name, i.first)));
s += ','; printUnquotedString(s, printMethodAlgo(dof.ca));
s += ','; printUnquotedString(s, getContentAddressHash(dof.ca).to_string(Base16, false));
},
[&](DerivationOutputCAFloating dof) {
[&](const DerivationOutputCAFloating & dof) {
s += ','; printUnquotedString(s, "");
s += ','; printUnquotedString(s, makeContentAddressingPrefix(dof.method) + printHashType(dof.hashType));
s += ','; printUnquotedString(s, "");
},
[&](DerivationOutputDeferred) {
[&](const DerivationOutputDeferred &) {
s += ','; printUnquotedString(s, "");
s += ','; printUnquotedString(s, "");
s += ','; printUnquotedString(s, "");
@ -418,13 +418,13 @@ DerivationType BasicDerivation::type() const
std::optional<HashType> floatingHashType;
for (auto & i : outputs) {
std::visit(overloaded {
[&](DerivationOutputInputAddressed _) {
[&](const DerivationOutputInputAddressed &) {
inputAddressedOutputs.insert(i.first);
},
[&](DerivationOutputCAFixed _) {
[&](const DerivationOutputCAFixed &) {
fixedCAOutputs.insert(i.first);
},
[&](DerivationOutputCAFloating dof) {
[&](const DerivationOutputCAFloating & dof) {
floatingCAOutputs.insert(i.first);
if (!floatingHashType) {
floatingHashType = dof.hashType;
@ -433,7 +433,7 @@ DerivationType BasicDerivation::type() const
throw Error("All floating outputs must use the same hash type");
}
},
[&](DerivationOutputDeferred _) {
[&](const DerivationOutputDeferred &) {
deferredIAOutputs.insert(i.first);
},
}, i.second.output);
@ -536,15 +536,15 @@ DrvHashModulo hashDerivationModulo(Store & store, const Derivation & drv, bool m
const auto & res = pathDerivationModulo(store, i.first);
std::visit(overloaded {
// Regular non-CA derivation, replace derivation
[&](Hash drvHash) {
[&](const Hash & drvHash) {
inputs2.insert_or_assign(drvHash.to_string(Base16, false), i.second);
},
[&](DeferredHash deferredHash) {
[&](const DeferredHash & deferredHash) {
isDeferred = true;
inputs2.insert_or_assign(deferredHash.hash.to_string(Base16, false), i.second);
},
// CA derivation's output hashes
[&](CaOutputHashes outputHashes) {
[&](const CaOutputHashes & outputHashes) {
std::set<std::string> justOut = { "out" };
for (auto & output : i.second) {
/* Put each one in with a single "out" output.. */
@ -570,17 +570,17 @@ std::map<std::string, Hash> staticOutputHashes(Store & store, const Derivation &
{
std::map<std::string, Hash> res;
std::visit(overloaded {
[&](Hash drvHash) {
[&](const Hash & drvHash) {
for (auto & outputName : drv.outputNames()) {
res.insert({outputName, drvHash});
}
},
[&](DeferredHash deferredHash) {
[&](const DeferredHash & deferredHash) {
for (auto & outputName : drv.outputNames()) {
res.insert({outputName, deferredHash.hash});
}
},
[&](CaOutputHashes outputHashes) {
[&](const CaOutputHashes & outputHashes) {
res = outputHashes;
},
}, hashDerivationModulo(store, drv, true));
@ -664,22 +664,22 @@ void writeDerivation(Sink & out, const Store & store, const BasicDerivation & dr
for (auto & i : drv.outputs) {
out << i.first;
std::visit(overloaded {
[&](DerivationOutputInputAddressed doi) {
[&](const DerivationOutputInputAddressed & doi) {
out << store.printStorePath(doi.path)
<< ""
<< "";
},
[&](DerivationOutputCAFixed dof) {
[&](const DerivationOutputCAFixed & dof) {
out << store.printStorePath(dof.path(store, drv.name, i.first))
<< printMethodAlgo(dof.ca)
<< getContentAddressHash(dof.ca).to_string(Base16, false);
},
[&](DerivationOutputCAFloating dof) {
[&](const DerivationOutputCAFloating & dof) {
out << ""
<< (makeContentAddressingPrefix(dof.method) + printHashType(dof.hashType))
<< "";
},
[&](DerivationOutputDeferred) {
[&](const DerivationOutputDeferred &) {
out << ""
<< ""
<< "";

8
src/libstore/derived-path.cc
View file

@ -24,8 +24,8 @@ StorePathSet BuiltPath::outPaths() const
{
return std::visit(
overloaded{
[](BuiltPath::Opaque p) { return StorePathSet{p.path}; },
[](BuiltPath::Built b) {
[](const BuiltPath::Opaque & p) { return StorePathSet{p.path}; },
[](const BuiltPath::Built & b) {
StorePathSet res;
for (auto & [_, path] : b.outputs)
res.insert(path);
@ -94,8 +94,8 @@ RealisedPath::Set BuiltPath::toRealisedPaths(Store & store) const
RealisedPath::Set res;
std::visit(
overloaded{
[&](BuiltPath::Opaque p) { res.insert(p.path); },
[&](BuiltPath::Built p) {
[&](const BuiltPath::Opaque & p) { res.insert(p.path); },
[&](const BuiltPath::Built & p) {
auto drvHashes =
staticOutputHashes(store, store.readDerivation(p.drvPath));
for (auto& [outputName, outputPath] : p.outputs) {

8
src/libstore/legacy-ssh-store.cc
View file

@ -247,6 +247,10 @@ private:
conn.to
<< settings.buildRepeat
<< settings.enforceDeterminism;
if (GET_PROTOCOL_MINOR(conn.remoteVersion) >= 7) {
conn.to << ((int) settings.keepFailed);
}
}
public:
@ -289,10 +293,10 @@ public:
for (auto & p : drvPaths) {
auto sOrDrvPath = StorePathWithOutputs::tryFromDerivedPath(p);
std::visit(overloaded {
[&](StorePathWithOutputs s) {
[&](const StorePathWithOutputs & s) {
ss.push_back(s.to_string(*this));
},
[&](StorePath drvPath) {
[&](const StorePath & drvPath) {
throw Error("wanted to fetch '%s' but the legacy ssh protocol doesn't support merely substituting drv files via the build paths command. It would build them instead. Try using ssh-ng://", printStorePath(drvPath));
},
}, sOrDrvPath);

8
src/libstore/local-store.cc
View file

@ -681,7 +681,7 @@ void LocalStore::checkDerivationOutputs(const StorePath & drvPath, const Derivat
std::optional<Hash> h;
for (auto & i : drv.outputs) {
std::visit(overloaded {
[&](DerivationOutputInputAddressed doia) {
[&](const DerivationOutputInputAddressed & doia) {
if (!h) {
// somewhat expensive so we do lazily
auto temp = hashDerivationModulo(*this, drv, true);
@ -693,14 +693,14 @@ void LocalStore::checkDerivationOutputs(const StorePath & drvPath, const Derivat
printStorePath(drvPath), printStorePath(doia.path), printStorePath(recomputed));
envHasRightPath(doia.path, i.first);
},
[&](DerivationOutputCAFixed dof) {
[&](const DerivationOutputCAFixed & dof) {
auto path = dof.path(*this, drvName, i.first);
envHasRightPath(path, i.first);
},
[&](DerivationOutputCAFloating _) {
[&](const DerivationOutputCAFloating &) {
/* Nothing to check */
},
[&](DerivationOutputDeferred) {
[&](const DerivationOutputDeferred &) {
},
}, i.second.output);
}

8
src/libstore/misc.cc
View file

@ -88,12 +88,12 @@ std::optional<ContentAddress> getDerivationCA(const BasicDerivation & drv)
return std::nullopt;
if (auto dof = std::get_if<DerivationOutputCAFixed>(&out->second.output)) {
return std::visit(overloaded {
[&](TextInfo ti) -> std::optional<ContentAddress> {
[&](const TextInfo & ti) -> std::optional<ContentAddress> {
if (!ti.references.empty())
return std::nullopt;
return static_cast<TextHash>(ti);
},
[&](FixedOutputInfo fi) -> std::optional<ContentAddress> {
[&](const FixedOutputInfo & fi) -> std::optional<ContentAddress> {
if (fi.references != PathReferences<StorePath> {})
return std::nullopt;
return static_cast<FixedOutputHash>(fi);
@ -177,7 +177,7 @@ void Store::queryMissing(const std::vector<DerivedPath> & targets,
}
std::visit(overloaded {
[&](DerivedPath::Built bfd) {
[&](const DerivedPath::Built & bfd) {
if (!isValidPath(bfd.drvPath)) {
// FIXME: we could try to substitute the derivation.
auto state(state_.lock());
@ -210,7 +210,7 @@ void Store::queryMissing(const std::vector<DerivedPath> & targets,
mustBuildDrv(bfd.drvPath, *drv);
},
[&](DerivedPath::Opaque bo) {
[&](const DerivedPath::Opaque & bo) {
if (isValidPath(bo.path)) return;

4
src/libstore/path-with-outputs.cc
View file

@ -31,14 +31,14 @@ std::vector<DerivedPath> toDerivedPaths(const std::vector<StorePathWithOutputs>
std::variant<StorePathWithOutputs, StorePath> StorePathWithOutputs::tryFromDerivedPath(const DerivedPath & p)
{
return std::visit(overloaded {
[&](DerivedPath::Opaque bo) -> std::variant<StorePathWithOutputs, StorePath> {
[&](const DerivedPath::Opaque & bo) -> std::variant<StorePathWithOutputs, StorePath> {
if (bo.path.isDerivation()) {
// drv path gets interpreted as "build", not "get drv file itself"
return bo.path;
}
return StorePathWithOutputs { bo.path };
},
[&](DerivedPath::Built bfd) -> std::variant<StorePathWithOutputs, StorePath> {
[&](const DerivedPath::Built & bfd) -> std::variant<StorePathWithOutputs, StorePath> {
return StorePathWithOutputs { bfd.drvPath, bfd.outputs };
},
}, p.raw());

8
src/libstore/remote-store.cc
View file

@ -530,7 +530,7 @@ ref<const ValidPathInfo> RemoteStore::addCAToStore(
if (repair) throw Error("repairing is not supported when building through the Nix daemon protocol < 1.25");
std::visit(overloaded {
[&](TextHashMethod thm) -> void {
[&](const TextHashMethod & thm) -> void {
if (hashType != htSHA256)
throw UnimplementedError("Only SHA-256 is supported for adding text-hashed data, but '%1' was given",
printHashType(hashType));
@ -539,7 +539,7 @@ ref<const ValidPathInfo> RemoteStore::addCAToStore(
worker_proto::write(*this, conn->to, references);
conn.processStderr();
},
[&](FileIngestionMethod fim) -> void {
[&](const FileIngestionMethod & fim) -> void {
conn->to
<< wopAddToStore
<< name
@ -710,10 +710,10 @@ static void writeDerivedPaths(RemoteStore & store, ConnectionHandle & conn, cons
for (auto & p : reqs) {
auto sOrDrvPath = StorePathWithOutputs::tryFromDerivedPath(p);
std::visit(overloaded {
[&](StorePathWithOutputs s) {
[&](const StorePathWithOutputs & s) {
ss.push_back(s.to_string(store));
},
[&](StorePath drvPath) {
[&](const StorePath & drvPath) {
throw Error("trying to request '%s', but daemon protocol %d.%d is too old (< 1.29) to request a derivation file",
store.printStorePath(drvPath),
GET_PROTOCOL_MAJOR(conn->daemonVersion),

2
src/libstore/serve-protocol.hh
View file

@ -5,7 +5,7 @@ namespace nix {
#define SERVE_MAGIC_1 0x390c9deb
#define SERVE_MAGIC_2 0x5452eecb
#define SERVE_PROTOCOL_VERSION (2 << 8 | 6)
#define SERVE_PROTOCOL_VERSION (2 << 8 | 7)
#define GET_PROTOCOL_MAJOR(x) ((x) & 0xff00)
#define GET_PROTOCOL_MINOR(x) ((x) & 0x00ff)

12
src/libstore/store-api.cc
View file

@ -208,10 +208,10 @@ StorePath Store::makeFixedOutputPathFromCA(const StorePathDescriptor & desc) con
{
// New template
return std::visit(overloaded {
[&](TextInfo ti) {
[&](const TextInfo & ti) {
return makeTextPath(desc.name, ti);
},
[&](FixedOutputInfo foi) {
[&](const FixedOutputInfo & foi) {
return makeFixedOutputPath(desc.name, foi);
}
}, desc.info);
@ -1150,13 +1150,13 @@ std::optional<StorePathDescriptor> ValidPathInfo::fullStorePathDescriptorOpt() c
return StorePathDescriptor {
.name = std::string { path.name() },
.info = std::visit(overloaded {
[&](TextHash th) {
[&](const TextHash & th) {
TextInfo info { th };
assert(!hasSelfReference);
info.references = references;
return ContentAddressWithReferences { info };
},
[&](FixedOutputHash foh) {
[&](const FixedOutputHash & foh) {
FixedOutputInfo info { foh };
info.references = static_cast<PathReferences<StorePath>>(*this);
return ContentAddressWithReferences { info };
@ -1218,11 +1218,11 @@ ValidPathInfo::ValidPathInfo(
, narHash(narHash)
{
std::visit(overloaded {
[this](TextInfo ti) {
[this](const TextInfo & ti) {
this->references = ti.references;
this->ca = TextHash { std::move(ti) };
},
[this](FixedOutputInfo foi) {
[this](const FixedOutputInfo & foi) {
*(static_cast<PathReferences<StorePath> *>(this)) = foi.references;
this->ca = FixedOutputHash { (FixedOutputHash) std::move(foi) };
},

3
src/nix-store/nix-store.cc
View file

@ -807,6 +807,9 @@ static void opServe(Strings opFlags, Strings opArgs)
settings.enforceDeterminism = readInt(in);
settings.runDiffHook = true;
}
if (GET_PROTOCOL_MINOR(clientVersion) >= 7) {
settings.keepFailed = (bool) readInt(in);
}
settings.printRepeatedBuilds = false;
};

4
src/nix/build.cc
View file

@ -66,12 +66,12 @@ struct CmdBuild : InstallablesCommand, MixDryRun, MixJSON, MixProfile
for (const auto & [_i, buildable] : enumerate(buildables)) {
auto i = _i;
std::visit(overloaded {
[&](BuiltPath::Opaque bo) {
[&](const BuiltPath::Opaque & bo) {
std::string symlink = outLink;
if (i) symlink += fmt("-%d", i);
store2->addPermRoot(bo.path, absPath(symlink));
},
[&](BuiltPath::Built bfd) {
[&](const BuiltPath::Built & bfd) {
for (auto & output : bfd.outputs) {
std::string symlink = outLink;
if (i) symlink += fmt("-%d", i);

4
src/nix/log.cc
View file

@ -35,10 +35,10 @@ struct CmdLog : InstallableCommand
RunPager pager;
for (auto & sub : subs) {
auto log = std::visit(overloaded {
[&](DerivedPath::Opaque bo) {
[&](const DerivedPath::Opaque & bo) {
return sub->getBuildLog(bo.path);
},
[&](DerivedPath::Built bfd) {
[&](const DerivedPath::Built & bfd) {
return sub->getBuildLog(bfd.drvPath);
},
}, b.raw());

4
src/nix/profile.cc
View file

@ -269,11 +269,11 @@ struct CmdProfileInstall : InstallablesCommand, MixDefaultProfile
ProfileElement element;
std::visit(overloaded {
[&](BuiltPath::Opaque bo) {
[&](const BuiltPath::Opaque & bo) {
pathsToBuild.push_back(bo);
element.storePaths.insert(bo.path);
},
[&](BuiltPath::Built bfd) {
[&](const BuiltPath::Built & bfd) {
// TODO: Why are we querying if we know the output
// names already? Is it just to figure out what the
// default one is?

8
src/nix/show-derivation.cc
View file

@ -65,19 +65,19 @@ struct CmdShowDerivation : InstallablesCommand
auto & outputName = _outputName; // work around clang bug
auto outputObj { outputsObj.object(outputName) };
std::visit(overloaded {
[&](DerivationOutputInputAddressed doi) {
[&](const DerivationOutputInputAddressed & doi) {
outputObj.attr("path", store->printStorePath(doi.path));
},
[&](DerivationOutputCAFixed dof) {
[&](const DerivationOutputCAFixed & dof) {
outputObj.attr("path", store->printStorePath(dof.path(*store, drv.name, outputName)));
outputObj.attr("hashAlgo", printMethodAlgo(dof.ca));
outputObj.attr("hash", getContentAddressHash(dof.ca).to_string(Base16, false));
// FIXME print refs?
},
[&](DerivationOutputCAFloating dof) {
[&](const DerivationOutputCAFloating & dof) {
outputObj.attr("hashAlgo", makeContentAddressingPrefix(dof.method) + printHashType(dof.hashType));
},
[&](DerivationOutputDeferred) {},
[&](const DerivationOutputDeferred &) {},
}, output.output);
}
}

13
tests/build-remote.sh
View file

@ -53,3 +53,16 @@ nix path-info --store $TEST_ROOT/machine3 --all \
| grep -v builder-build-remote-input-1.sh \
| grep -v builder-build-remote-input-2.sh \
| grep builder-build-remote-input-3.sh
# Behavior of keep-failed
out="$(nix-build 2>&1 failing.nix \
--builders "$(join_by '; ' "${builders[@]}")" \
--keep-failed \
--store $TEST_ROOT/machine0 \
-j0 \
--arg busybox $busybox)" || true
[[ "$out" =~ .*"note: keeping build directory".* ]]
build_dir="$(grep "note: keeping build" <<< "$out" | sed -E "s/^(.*)note: keeping build directory '(.*)'(.*)$/\2/")"
[[ "foo" = $(<"$build_dir"/bar) ]]

22
tests/failing.nix Normal file
View file

@ -0,0 +1,22 @@
{ busybox }:
with import ./config.nix;
let
mkDerivation = args:
derivation ({
inherit system;
builder = busybox;
args = ["sh" "-e" args.builder or (builtins.toFile "builder-${args.name}.sh" "if [ -e .attrs.sh ]; then source .attrs.sh; fi; eval \"$buildCommand\"")];
} // removeAttrs args ["builder" "meta"])
// { meta = args.meta or {}; };
in
{
failing = mkDerivation {
name = "failing";
buildCommand = ''
echo foo > bar
exit 1
'';
};
}