max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2025-02-07 10:47:18 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge remote-tracking branch 'nixos/master'

Browse source
This commit is contained in:
Max Headroom 2023-08-24 23:49:00 +02:00
commit 0007178284
58 changed files with 977 additions and 618 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.github/workflows/backport.yml vendored
View file

@ -21,7 +21,7 @@ jobs:
fetch-depth: 0 fetch-depth: 0
- name: Create backport PRs - name: Create backport PRs
# should be kept in sync with `version` # should be kept in sync with `version`
uses: zeebe-io/backport-action@v1.3.1 uses: zeebe-io/backport-action@v1.4.0
with: with:
# Config README: https://github.com/zeebe-io/backport-action#backport-action # Config README: https://github.com/zeebe-io/backport-action#backport-action
github_token: ${{ secrets.GITHUB_TOKEN }} github_token: ${{ secrets.GITHUB_TOKEN }}

3
.gitignore vendored
View file

@ -139,3 +139,6 @@ nix-rust/target
result result
.vscode/ .vscode/
# clangd and possibly more
.cache/

2
doc/manual/src/advanced-topics/advanced-topics.md
View file

@ -1 +1 @@
This section lists advanced topics related to builds and builds performance

367
doc/manual/src/glossary.md
View file

@ -1,236 +1,269 @@
# Glossary # Glossary
- [derivation]{#gloss-derivation}\ - [derivation]{#gloss-derivation}
A description of a build task. The result of a derivation is a
store object. Derivations are typically specified in Nix expressions
using the [`derivation` primitive](./language/derivations.md). These are
translated into low-level *store derivations* (implicitly by
`nix-env` and `nix-build`, or explicitly by `nix-instantiate`).
[derivation]: #gloss-derivation A description of a build task. The result of a derivation is a
store object. Derivations are typically specified in Nix expressions
using the [`derivation` primitive](./language/derivations.md). These are
translated into low-level *store derivations* (implicitly by
`nix-env` and `nix-build`, or explicitly by `nix-instantiate`).
- [store derivation]{#gloss-store-derivation}\ [derivation]: #gloss-derivation
A [derivation] represented as a `.drv` file in the [store].
It has a [store path], like any [store object].
Example: `/nix/store/g946hcz4c8mdvq2g8vxx42z51qb71rvp-git-2.38.1.drv` - [store derivation]{#gloss-store-derivation}
See [`nix derivation show`](./command-ref/new-cli/nix3-derivation-show.md) (experimental) for displaying the contents of store derivations. A [derivation] represented as a `.drv` file in the [store].
It has a [store path], like any [store object].
[store derivation]: #gloss-store-derivation Example: `/nix/store/g946hcz4c8mdvq2g8vxx42z51qb71rvp-git-2.38.1.drv`
- [instantiate]{#gloss-instantiate}, instantiation\ See [`nix derivation show`](./command-ref/new-cli/nix3-derivation-show.md) (experimental) for displaying the contents of store derivations.
Translate a [derivation] into a [store derivation].
See [`nix-instantiate`](./command-ref/nix-instantiate.md). [store derivation]: #gloss-store-derivation
[instantiate]: #gloss-instantiate - [instantiate]{#gloss-instantiate}, instantiation
- [realise]{#gloss-realise}, realisation\ Translate a [derivation] into a [store derivation].
Ensure a [store path] is [valid][validity].
This means either running the `builder` executable as specified in the corresponding [derivation] or fetching a pre-built [store object] from a [substituter]. See [`nix-instantiate`](./command-ref/nix-instantiate.md).
See [`nix-build`](./command-ref/nix-build.md) and [`nix-store --realise`](@docroot@/command-ref/nix-store/realise.md). [instantiate]: #gloss-instantiate
See [`nix build`](./command-ref/new-cli/nix3-build.md) (experimental). - [realise]{#gloss-realise}, realisation
[realise]: #gloss-realise Ensure a [store path] is [valid][validity].
- [content-addressed derivation]{#gloss-content-addressed-derivation}\ This means either running the `builder` executable as specified in the corresponding [derivation] or fetching a pre-built [store object] from a [substituter].
A derivation which has the
[`__contentAddressed`](./language/advanced-attributes.md#adv-attr-__contentAddressed)
attribute set to `true`.
- [fixed-output derivation]{#gloss-fixed-output-derivation}\ See [`nix-build`](./command-ref/nix-build.md) and [`nix-store --realise`](@docroot@/command-ref/nix-store/realise.md).
A derivation which includes the
[`outputHash`](./language/advanced-attributes.md#adv-attr-outputHash) attribute.
- [store]{#gloss-store}\ See [`nix build`](./command-ref/new-cli/nix3-build.md) (experimental).
The location in the file system where store objects live. Typically
`/nix/store`.
From the perspective of the location where Nix is [realise]: #gloss-realise
invoked, the Nix store can be referred to
as a "_local_" or a "_remote_" one:
+ A [local store]{#gloss-local-store} exists on the filesystem of - [content-addressed derivation]{#gloss-content-addressed-derivation}
the machine where Nix is invoked. You can use other
local stores by passing the `--store` flag to the
`nix` command. Local stores can be used for building derivations.
+ A *remote store* exists anywhere other than the A derivation which has the
local filesystem. One example is the `/nix/store` [`__contentAddressed`](./language/advanced-attributes.md#adv-attr-__contentAddressed)
directory on another machine, accessed via `ssh` or attribute set to `true`.
served by the `nix-serve` Perl script.
[store]: #gloss-store - [fixed-output derivation]{#gloss-fixed-output-derivation}
[local store]: #gloss-local-store
- [chroot store]{#gloss-chroot-store}\ A derivation which includes the
A [local store] whose canonical path is anything other than `/nix/store`. [`outputHash`](./language/advanced-attributes.md#adv-attr-outputHash) attribute.
- [binary cache]{#gloss-binary-cache}\ - [store]{#gloss-store}
A *binary cache* is a Nix store which uses a different format: its
metadata and signatures are kept in `.narinfo` files rather than in a
[Nix database]. This different format simplifies serving store objects
over the network, but cannot host builds. Examples of binary caches
include S3 buckets and the [NixOS binary cache](https://cache.nixos.org).
- [store path]{#gloss-store-path}\ The location in the file system where store objects live. Typically
The location of a [store object] in the file system, i.e., an `/nix/store`.
immediate child of the Nix store directory.
Example: `/nix/store/a040m110amc4h71lds2jmr8qrkj2jhxd-git-2.38.1` From the perspective of the location where Nix is
invoked, the Nix store can be referred to
as a "_local_" or a "_remote_" one:
[store path]: #gloss-store-path + A [local store]{#gloss-local-store} exists on the filesystem of
the machine where Nix is invoked. You can use other
local stores by passing the `--store` flag to the
`nix` command. Local stores can be used for building derivations.
- [file system object]{#gloss-store-object}\ + A *remote store* exists anywhere other than the
The Nix data model for representing simplified file system data. local filesystem. One example is the `/nix/store`
directory on another machine, accessed via `ssh` or
served by the `nix-serve` Perl script.
See [File System Object](@docroot@/architecture/file-system-object.md) for details. [store]: #gloss-store
[local store]: #gloss-local-store
[file system object]: #gloss-file-system-object - [chroot store]{#gloss-chroot-store}
- [store object]{#gloss-store-object}\ A [local store] whose canonical path is anything other than `/nix/store`.
A store object consists of a [file system object], [reference]s to other store objects, and other metadata. - [binary cache]{#gloss-binary-cache}
It can be referred to by a [store path].
[store object]: #gloss-store-object A *binary cache* is a Nix store which uses a different format: its
metadata and signatures are kept in `.narinfo` files rather than in a
[Nix database]. This different format simplifies serving store objects
over the network, but cannot host builds. Examples of binary caches
include S3 buckets and the [NixOS binary cache](https://cache.nixos.org).
- [input-addressed store object]{#gloss-input-addressed-store-object}\ - [store path]{#gloss-store-path}
A store object produced by building a
non-[content-addressed](#gloss-content-addressed-derivation),
non-[fixed-output](#gloss-fixed-output-derivation)
derivation.
- [output-addressed store object]{#gloss-output-addressed-store-object}\ The location of a [store object] in the file system, i.e., an
A [store object] whose [store path] is determined by its contents. immediate child of the Nix store directory.
This includes derivations, the outputs of [content-addressed derivations](#gloss-content-addressed-derivation), and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation).
- [substitute]{#gloss-substitute}\ Example: `/nix/store/a040m110amc4h71lds2jmr8qrkj2jhxd-git-2.38.1`
A substitute is a command invocation stored in the [Nix database] that
describes how to build a store object, bypassing the normal build
mechanism (i.e., derivations). Typically, the substitute builds the
store object by downloading a pre-built version of the store object
from some server.
- [substituter]{#gloss-substituter}\ [store path]: #gloss-store-path
An additional [store]{#gloss-store} from which Nix can obtain store objects instead of building them.
Often the substituter is a [binary cache](#gloss-binary-cache), but any store can serve as substituter.
See the [`substituters` configuration option](./command-ref/conf-file.md#conf-substituters) for details. - [file system object]{#gloss-store-object}
[substituter]: #gloss-substituter The Nix data model for representing simplified file system data.
- [purity]{#gloss-purity}\ See [File System Object](@docroot@/architecture/file-system-object.md) for details.
The assumption that equal Nix derivations when run always produce
the same output. This cannot be guaranteed in general (e.g., a
builder can rely on external inputs such as the network or the
system time) but the Nix model assumes it.
- [Nix database]{#gloss-nix-database}\ [file system object]: #gloss-file-system-object
An SQlite database to track [reference]s between [store object]s.
This is an implementation detail of the [local store].
Default location: `/nix/var/nix/db`. - [store object]{#gloss-store-object}
[Nix database]: #gloss-nix-database
- [Nix expression]{#gloss-nix-expression}\ A store object consists of a [file system object], [reference]s to other store objects, and other metadata.
A high-level description of software packages and compositions It can be referred to by a [store path].
thereof. Deploying software using Nix entails writing Nix
expressions for your packages. Nix expressions are translated to
derivations that are stored in the Nix store. These derivations can
then be built.
- [reference]{#gloss-reference}\ [store object]: #gloss-store-object
A [store object] `O` is said to have a *reference* to a store object `P` if a [store path] to `P` appears in the contents of `O`.
Store objects can refer to both other store objects and themselves. - [input-addressed store object]{#gloss-input-addressed-store-object}
References from a store object to itself are called *self-references*.
References other than a self-reference must not form a cycle.
[reference]: #gloss-reference A store object produced by building a
non-[content-addressed](#gloss-content-addressed-derivation),
non-[fixed-output](#gloss-fixed-output-derivation)
derivation.
- [reachable]{#gloss-reachable}\ - [output-addressed store object]{#gloss-output-addressed-store-object}
A store path `Q` is reachable from another store path `P` if `Q`
is in the *closure* of the *references* relation.
- [closure]{#gloss-closure}\ A [store object] whose [store path] is determined by its contents.
The closure of a store path is the set of store paths that are This includes derivations, the outputs of [content-addressed derivations](#gloss-content-addressed-derivation), and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation).
directly or indirectly “reachable” from that store path; that is,
it’s the closure of the path under the *references* relation. For
a package, the closure of its derivation is equivalent to the
build-time dependencies, while the closure of its output path is
equivalent to its runtime dependencies. For correct deployment it
is necessary to deploy whole closures, since otherwise at runtime
files could be missing. The command `nix-store --query --requisites ` prints out
closures of store paths.
As an example, if the [store object] at path `P` contains a [reference] - [substitute]{#gloss-substitute}
to a store object at path `Q`, then `Q` is in the closure of `P`. Further, if `Q`
references `R` then `R` is also in the closure of `P`.
[closure]: #gloss-closure A substitute is a command invocation stored in the [Nix database] that
describes how to build a store object, bypassing the normal build
mechanism (i.e., derivations). Typically, the substitute builds the
store object by downloading a pre-built version of the store object
from some server.
- [output path]{#gloss-output-path}\ - [substituter]{#gloss-substituter}
A [store path] produced by a [derivation].
[output path]: #gloss-output-path An additional [store]{#gloss-store} from which Nix can obtain store objects instead of building them.
Often the substituter is a [binary cache](#gloss-binary-cache), but any store can serve as substituter.
- [deriver]{#gloss-deriver}\ See the [`substituters` configuration option](./command-ref/conf-file.md#conf-substituters) for details.
The [store derivation] that produced an [output path].
- [validity]{#gloss-validity}\ [substituter]: #gloss-substituter
A store path is valid if all [store object]s in its [closure] can be read from the [store].
For a [local store], this means: - [purity]{#gloss-purity}
- The store path leads to an existing [store object] in that [store].
- The store path is listed in the [Nix database] as being valid.
- All paths in the store path's [closure] are valid.
[validity]: #gloss-validity The assumption that equal Nix derivations when run always produce
the same output. This cannot be guaranteed in general (e.g., a
builder can rely on external inputs such as the network or the
system time) but the Nix model assumes it.
- [user environment]{#gloss-user-env}\ - [Nix database]{#gloss-nix-database}
An automatically generated store object that consists of a set of
symlinks to “active” applications, i.e., other store paths. These
are generated automatically by
[`nix-env`](./command-ref/nix-env.md). See *profiles*.
- [profile]{#gloss-profile}\ An SQlite database to track [reference]s between [store object]s.
A symlink to the current *user environment* of a user, e.g., This is an implementation detail of the [local store].
`/nix/var/nix/profiles/default`.
- [installable]{#gloss-installable}\ Default location: `/nix/var/nix/db`.
Something that can be realised in the Nix store.
See [installables](./command-ref/new-cli/nix.md#installables) for [`nix` commands](./command-ref/new-cli/nix.md) (experimental) for details. [Nix database]: #gloss-nix-database
- [NAR]{#gloss-nar}\ - [Nix expression]{#gloss-nix-expression}
A *N*ix *AR*chive. This is a serialisation of a path in the Nix
store. It can contain regular files, directories and symbolic
links. NARs are generated and unpacked using `nix-store --dump`
and `nix-store --restore`.
- [`∅`]{#gloss-emtpy-set}\ A high-level description of software packages and compositions
The empty set symbol. In the context of profile history, this denotes a package is not present in a particular version of the profile. thereof. Deploying software using Nix entails writing Nix
expressions for your packages. Nix expressions are translated to
derivations that are stored in the Nix store. These derivations can
then be built.
- [`ε`]{#gloss-epsilon}\ - [reference]{#gloss-reference}
The epsilon symbol. In the context of a package, this means the version is empty. More precisely, the derivation does not have a version attribute.
- [string interpolation]{#gloss-string-interpolation}\ A [store object] `O` is said to have a *reference* to a store object `P` if a [store path] to `P` appears in the contents of `O`.
Expanding expressions enclosed in `${ }` within a [string], [path], or [attribute name].
See [String interpolation](./language/string-interpolation.md) for details. Store objects can refer to both other store objects and themselves.
References from a store object to itself are called *self-references*.
References other than a self-reference must not form a cycle.
[string]: ./language/values.md#type-string [reference]: #gloss-reference
[path]: ./language/values.md#type-path
[attribute name]: ./language/values.md#attribute-set
- [experimental feature]{#gloss-experimental-feature}\ - [reachable]{#gloss-reachable}
Not yet stabilized functionality guarded by named experimental feature flags.
These flags are enabled or disabled with the [`experimental-features`](./command-ref/conf-file.html#conf-experimental-features) setting.
See the contribution guide on the [purpose and lifecycle of experimental feaures](@docroot@/contributing/experimental-features.md). A store path `Q` is reachable from another store path `P` if `Q`
is in the *closure* of the *references* relation.
- [closure]{#gloss-closure}
The closure of a store path is the set of store paths that are
directly or indirectly “reachable” from that store path; that is,
it’s the closure of the path under the *references* relation. For
a package, the closure of its derivation is equivalent to the
build-time dependencies, while the closure of its output path is
equivalent to its runtime dependencies. For correct deployment it
is necessary to deploy whole closures, since otherwise at runtime
files could be missing. The command `nix-store --query --requisites ` prints out
closures of store paths.
As an example, if the [store object] at path `P` contains a [reference]
to a store object at path `Q`, then `Q` is in the closure of `P`. Further, if `Q`
references `R` then `R` is also in the closure of `P`.
[closure]: #gloss-closure
- [output path]{#gloss-output-path}
A [store path] produced by a [derivation].
[output path]: #gloss-output-path
- [deriver]{#gloss-deriver}
The [store derivation] that produced an [output path].
- [validity]{#gloss-validity}
A store path is valid if all [store object]s in its [closure] can be read from the [store].
For a [local store], this means:
- The store path leads to an existing [store object] in that [store].
- The store path is listed in the [Nix database] as being valid.
- All paths in the store path's [closure] are valid.
[validity]: #gloss-validity
- [user environment]{#gloss-user-env}
An automatically generated store object that consists of a set of
symlinks to “active” applications, i.e., other store paths. These
are generated automatically by
[`nix-env`](./command-ref/nix-env.md). See *profiles*.
- [profile]{#gloss-profile}
A symlink to the current *user environment* of a user, e.g.,
`/nix/var/nix/profiles/default`.
- [installable]{#gloss-installable}
Something that can be realised in the Nix store.
See [installables](./command-ref/new-cli/nix.md#installables) for [`nix` commands](./command-ref/new-cli/nix.md) (experimental) for details.
- [NAR]{#gloss-nar}
A *N*ix *AR*chive. This is a serialisation of a path in the Nix
store. It can contain regular files, directories and symbolic
links. NARs are generated and unpacked using `nix-store --dump`
and `nix-store --restore`.
- [`∅`]{#gloss-emtpy-set}
The empty set symbol. In the context of profile history, this denotes a package is not present in a particular version of the profile.
- [`ε`]{#gloss-epsilon}
The epsilon symbol. In the context of a package, this means the version is empty. More precisely, the derivation does not have a version attribute.
- [string interpolation]{#gloss-string-interpolation}
Expanding expressions enclosed in `${ }` within a [string], [path], or [attribute name].
See [String interpolation](./language/string-interpolation.md) for details.
[string]: ./language/values.md#type-string
[path]: ./language/values.md#type-path
[attribute name]: ./language/values.md#attribute-set
- [experimental feature]{#gloss-experimental-feature}
Not yet stabilized functionality guarded by named experimental feature flags.
These flags are enabled or disabled with the [`experimental-features`](./command-ref/conf-file.html#conf-experimental-features) setting.
See the contribution guide on the [purpose and lifecycle of experimental feaures](@docroot@/contributing/experimental-features.md).

3
doc/manual/src/release-notes/rl-next.md
View file

@ -19,3 +19,6 @@
- The JSON output for derived paths with are store paths is now a string, not an object with a single `path` field. - The JSON output for derived paths with are store paths is now a string, not an object with a single `path` field.
This only affects `nix-build --json` when "building" non-derivation things like fetched sources, which is a no-op. This only affects `nix-build --json` when "building" non-derivation things like fetched sources, which is a no-op.
- Introduce a new [`outputOf`](@docroot@/language/builtins.md#builtins-outputOf) builtin.
It is part of the [`dynamic-derivations`](@docroot@/contributing/experimental-features.md#xp-feature-dynamic-derivations) experimental feature.

69
flake.nix
View file

@ -19,10 +19,12 @@
then "" then ""
else "pre${builtins.substring 0 8 (self.lastModifiedDate or self.lastModified or "19700101")}_${self.shortRev or "dirty"}"; else "pre${builtins.substring 0 8 (self.lastModifiedDate or self.lastModified or "19700101")}_${self.shortRev or "dirty"}";
linux32BitSystems = [ "i686-linux" ];
linux64BitSystems = [ "x86_64-linux" "aarch64-linux" ]; linux64BitSystems = [ "x86_64-linux" "aarch64-linux" ];
linuxSystems = linux64BitSystems ++ [ "i686-linux" ]; linuxSystems = linux32BitSystems ++ linux64BitSystems;
systems = linuxSystems ++ [ "x86_64-darwin" "aarch64-darwin" ]; darwinSystems = [ "x86_64-darwin" "aarch64-darwin" ];
systems = linuxSystems ++ darwinSystems;
crossSystems = [ "armv6l-linux" "armv7l-linux" ]; crossSystems = [ "armv6l-linux" "armv7l-linux" ];
stdenvs = [ "gccStdenv" "clangStdenv" "clang11Stdenv" "stdenv" "libcxxStdenv" "ccacheStdenv" ]; stdenvs = [ "gccStdenv" "clangStdenv" "clang11Stdenv" "stdenv" "libcxxStdenv" "ccacheStdenv" ];
@ -40,6 +42,52 @@
}) })
stdenvs); stdenvs);
# Experimental fileset library: https://github.com/NixOS/nixpkgs/pull/222981
# Not an "idiomatic" flake input because:
# - Propagation to dependent locks: https://github.com/NixOS/nix/issues/7730
# - Subflake would download redundant and huge parent flake
# - No git tree hash support: https://github.com/NixOS/nix/issues/6044
inherit (import (builtins.fetchTarball { url = "https://github.com/NixOS/nix/archive/1bdcd7fc8a6a40b2e805bad759b36e64e911036b.tar.gz"; sha256 = "sha256:14ljlpdsp4x7h1fkhbmc4bd3vsqnx8zdql4h3037wh09ad6a0893"; }))
fileset;
baseFiles =
# .gitignore has already been processed, so any changes in it are irrelevant
# at this point. It is not represented verbatim for test purposes because
# that would interfere with repo semantics.
fileset.fileFilter (f: f.name != ".gitignore") ./.;
nixSrc = fileset.toSource {
root = ./.;
fileset = fileset.intersect baseFiles (
fileset.difference
(fileset.unions [
./.version
./boehmgc-coroutine-sp-fallback.diff
./bootstrap.sh
./configure.ac
./doc
./local.mk
./m4
./Makefile
./Makefile.config.in
./misc
./mk
./precompiled-headers.h
./src
./tests
./COPYING
./scripts/local.mk
(fileset.fileFilter (f: lib.strings.hasPrefix "nix-profile" f.name) ./scripts)
# TODO: do we really need README.md? It doesn't seem used in the build.
./README.md
])
(fileset.unions [
# Removed file sets
./tests/nixos
./tests/installer
])
);
};
# Memoize nixpkgs for different platforms for efficiency. # Memoize nixpkgs for different platforms for efficiency.
nixpkgsFor = forAllSystems nixpkgsFor = forAllSystems
@ -209,7 +257,7 @@
"-${client.version}-against-${daemon.version}"; "-${client.version}-against-${daemon.version}";
inherit version; inherit version;
src = self; src = nixSrc;
VERSION_SUFFIX = versionSuffix; VERSION_SUFFIX = versionSuffix;
@ -320,18 +368,11 @@
}; };
let let
canRunInstalled = currentStdenv.buildPlatform.canExecute currentStdenv.hostPlatform; canRunInstalled = currentStdenv.buildPlatform.canExecute currentStdenv.hostPlatform;
sourceByRegexInverted = rxs: origSrc: final.lib.cleanSourceWith {
filter = (path: type:
let relPath = final.lib.removePrefix (toString origSrc + "/") (toString path);
in ! lib.any (re: builtins.match re relPath != null) rxs);
src = origSrc;
};
in currentStdenv.mkDerivation (finalAttrs: { in currentStdenv.mkDerivation (finalAttrs: {
name = "nix-super-${version}"; name = "nix-super-${version}";
inherit version; inherit version;
src = sourceByRegexInverted [ "tests/nixos/.*" "tests/installer/.*" ] self; src = nixSrc;
VERSION_SUFFIX = versionSuffix; VERSION_SUFFIX = versionSuffix;
outputs = [ "out" "dev" "doc" ]; outputs = [ "out" "dev" "doc" ];
@ -529,7 +570,7 @@
releaseTools.coverageAnalysis { releaseTools.coverageAnalysis {
name = "nix-super-coverage-${version}"; name = "nix-super-coverage-${version}";
src = self; src = nixSrc;
configureFlags = testConfigureFlags; configureFlags = testConfigureFlags;
@ -557,7 +598,7 @@
pname = "nix-internal-api-docs"; pname = "nix-internal-api-docs";
inherit version; inherit version;
src = self; src = nixSrc;
configureFlags = testConfigureFlags ++ internalApiDocsConfigureFlags; configureFlags = testConfigureFlags ++ internalApiDocsConfigureFlags;

5
src/libcmd/installable-attr-path.cc
View file

@ -80,7 +80,7 @@ DerivedPathsWithInfo InstallableAttrPath::toDerivedPaths()
[&](const ExtendedOutputsSpec::Explicit & e) -> OutputsSpec { [&](const ExtendedOutputsSpec::Explicit & e) -> OutputsSpec {
return e; return e;
}, },
}, extendedOutputsSpec.raw()); }, extendedOutputsSpec.raw);
auto [iter, didInsert] = byDrvPath.emplace(*drvPath, newOutputs); auto [iter, didInsert] = byDrvPath.emplace(*drvPath, newOutputs);
@ -96,6 +96,7 @@ DerivedPathsWithInfo InstallableAttrPath::toDerivedPaths()
.outputs = outputs, .outputs = outputs,
}, },
.info = make_ref<ExtraPathInfoValue>(ExtraPathInfoValue::Value { .info = make_ref<ExtraPathInfoValue>(ExtraPathInfoValue::Value {
.extendedOutputsSpec = outputs,
/* FIXME: reconsider backwards compatibility above /* FIXME: reconsider backwards compatibility above
so we can fill in this info. */ so we can fill in this info. */
}), }),
@ -114,7 +115,7 @@ InstallableAttrPath InstallableAttrPath::parse(
return { return {
state, cmd, v, state, cmd, v,
prefix == "." ? "" : std::string { prefix }, prefix == "." ? "" : std::string { prefix },
extendedOutputsSpec std::move(extendedOutputsSpec),
}; };
} }

2
src/libcmd/installable-derived-path.cc
View file

@ -55,7 +55,7 @@ InstallableDerivedPath InstallableDerivedPath::parse(
.outputs = outputSpec, .outputs = outputSpec,
}; };
}, },
}, extendedOutputsSpec.raw()); }, extendedOutputsSpec.raw);
return InstallableDerivedPath { return InstallableDerivedPath {
store, store,
std::move(derivedPath), std::move(derivedPath),

2
src/libcmd/installable-flake.cc
View file

@ -141,7 +141,7 @@ DerivedPathsWithInfo InstallableFlake::toDerivedPaths()
[&](const ExtendedOutputsSpec::Explicit & e) -> OutputsSpec { [&](const ExtendedOutputsSpec::Explicit & e) -> OutputsSpec {
return e; return e;
}, },
}, extendedOutputsSpec.raw()), }, extendedOutputsSpec.raw),
}, },
.info = make_ref<ExtraPathInfoFlake>( .info = make_ref<ExtraPathInfoFlake>(
ExtraPathInfoValue::Value { ExtraPathInfoValue::Value {

6
src/libcmd/installables.cc
View file

@ -583,7 +583,7 @@ Installables SourceExprCommand::parseInstallables(
for (auto & s : ss) { for (auto & s : ss) {
auto [prefix, extendedOutputsSpec] = ExtendedOutputsSpec::parse(s); auto [prefix, extendedOutputsSpec] = ExtendedOutputsSpec::parse(s);
auto installableAttr = make_ref<InstallableAttrPath>(InstallableAttrPath::parse( auto installableAttr = make_ref<InstallableAttrPath>(InstallableAttrPath::parse(
state, *this, vFile, prefix, extendedOutputsSpec state, *this, vFile, std::move(prefix), std::move(extendedOutputsSpec)
)); ));
if (doModifyInstallable) { if (doModifyInstallable) {
result.push_back( result.push_back(
@ -611,7 +611,7 @@ Installables SourceExprCommand::parseInstallables(
if (prefix.find('/') != std::string::npos) { if (prefix.find('/') != std::string::npos) {
try { try {
result.push_back(make_ref<InstallableDerivedPath>( result.push_back(make_ref<InstallableDerivedPath>(
InstallableDerivedPath::parse(store, prefix, extendedOutputsSpec))); InstallableDerivedPath::parse(store, prefix, extendedOutputsSpec.raw)));
continue; continue;
} catch (BadStorePath &) { } catch (BadStorePath &) {
} catch (...) { } catch (...) {
@ -636,7 +636,7 @@ Installables SourceExprCommand::parseInstallables(
state, state,
std::move(flakeRef), std::move(flakeRef),
fragment, fragment,
extendedOutputsSpec, std::move(extendedOutputsSpec),
getDefaultFlakeAttrPaths(), getDefaultFlakeAttrPaths(),
getDefaultFlakeAttrPathPrefixes(), getDefaultFlakeAttrPathPrefixes(),
lockFlags); lockFlags);

2
src/libcmd/modify-installables.cc
View file

@ -74,7 +74,7 @@ ref<Installable> SourceExprCommand::modifyInstallable (
} }
return return
make_ref<InstallableAttrPath>(InstallableAttrPath::parse( make_ref<InstallableAttrPath>(InstallableAttrPath::parse(
state, *this, vRes, prefix, extendedOutputsSpec state, *this, vRes, std::move(prefix), std::move(extendedOutputsSpec)
)); ));
} }

2
src/libexpr/eval-cache.cc
View file

@ -604,7 +604,7 @@ string_t AttrCursor::getStringWithContext()
[&](const NixStringContextElem::Opaque & o) -> const StorePath & { [&](const NixStringContextElem::Opaque & o) -> const StorePath & {
return o.path; return o.path;
}, },
}, c.raw()); }, c.raw);
if (!root->state.store->isValidPath(path)) { if (!root->state.store->isValidPath(path)) {
valid = false; valid = false;
break; break;

109
src/libexpr/eval.cc
View file

@ -527,9 +527,9 @@ EvalState::EvalState(
/* Initialise the Nix expression search path. */ /* Initialise the Nix expression search path. */
if (!evalSettings.pureEval) { if (!evalSettings.pureEval) {
for (auto & i : _searchPath.elements) for (auto & i : _searchPath.elements)
addToSearchPath(SearchPath::Elem {i}); searchPath.elements.emplace_back(SearchPath::Elem {i});
for (auto & i : evalSettings.nixPath.get()) for (auto & i : evalSettings.nixPath.get())
addToSearchPath(SearchPath::Elem::parse(i)); searchPath.elements.emplace_back(SearchPath::Elem::parse(i));
} }
if (evalSettings.restrictEval || evalSettings.pureEval) { if (evalSettings.restrictEval || evalSettings.pureEval) {
@ -1027,24 +1027,67 @@ void EvalState::mkStorePathString(const StorePath & p, Value & v)
} }
std::string EvalState::mkOutputStringRaw(
const SingleDerivedPath::Built & b,
std::optional<StorePath> optStaticOutputPath,
const ExperimentalFeatureSettings & xpSettings)
{
/* In practice, this is testing for the case of CA derivations, or
dynamic derivations. */
return optStaticOutputPath
? store->printStorePath(*std::move(optStaticOutputPath))
/* Downstream we would substitute this for an actual path once
we build the floating CA derivation */
: DownstreamPlaceholder::fromSingleDerivedPathBuilt(b, xpSettings).render();
}
void EvalState::mkOutputString( void EvalState::mkOutputString(
Value & value, Value & value,
const StorePath & drvPath, const SingleDerivedPath::Built & b,
const std::string outputName, std::optional<StorePath> optStaticOutputPath,
std::optional<StorePath> optOutputPath,
const ExperimentalFeatureSettings & xpSettings) const ExperimentalFeatureSettings & xpSettings)
{ {
value.mkString( value.mkString(
optOutputPath mkOutputStringRaw(b, optStaticOutputPath, xpSettings),
? store->printStorePath(*std::move(optOutputPath)) NixStringContext { b });
/* Downstream we would substitute this for an actual path once }
we build the floating CA derivation */
: DownstreamPlaceholder::unknownCaOutput(drvPath, outputName, xpSettings).render(),
std::string EvalState::mkSingleDerivedPathStringRaw(
const SingleDerivedPath & p)
{
return std::visit(overloaded {
[&](const SingleDerivedPath::Opaque & o) {
return store->printStorePath(o.path);
},
[&](const SingleDerivedPath::Built & b) {
auto optStaticOutputPath = std::visit(overloaded {
[&](const SingleDerivedPath::Opaque & o) {
auto drv = store->readDerivation(o.path);
auto i = drv.outputs.find(b.output);
if (i == drv.outputs.end())
throw Error("derivation '%s' does not have output '%s'", b.drvPath->to_string(*store), b.output);
return i->second.path(*store, drv.name, b.output);
},
[&](const SingleDerivedPath::Built & o) -> std::optional<StorePath> {
return std::nullopt;
},
}, b.drvPath->raw());
return mkOutputStringRaw(b, optStaticOutputPath);
}
}, p.raw());
}
void EvalState::mkSingleDerivedPathString(
const SingleDerivedPath & p,
Value & v)
{
v.mkString(
mkSingleDerivedPathStringRaw(p),
NixStringContext { NixStringContext {
NixStringContextElem::Built { std::visit([](auto && v) -> NixStringContextElem { return v; }, p),
.drvPath = makeConstantStorePathRef(drvPath),
.output = outputName,
}
}); });
} }
@ -2321,7 +2364,7 @@ std::pair<SingleDerivedPath, std::string_view> EvalState::coerceToSingleDerivedP
[&](NixStringContextElem::Built && b) -> SingleDerivedPath { [&](NixStringContextElem::Built && b) -> SingleDerivedPath {
return std::move(b); return std::move(b);
}, },
}, ((NixStringContextElem &&) *context.begin()).raw()); }, ((NixStringContextElem &&) *context.begin()).raw);
return { return {
std::move(derivedPath), std::move(derivedPath),
std::move(s), std::move(s),
@ -2333,39 +2376,25 @@ SingleDerivedPath EvalState::coerceToSingleDerivedPath(const PosIdx pos, Value &
{ {
auto [derivedPath, s_] = coerceToSingleDerivedPathUnchecked(pos, v, errorCtx); auto [derivedPath, s_] = coerceToSingleDerivedPathUnchecked(pos, v, errorCtx);
auto s = s_; auto s = s_;
std::visit(overloaded { auto sExpected = mkSingleDerivedPathStringRaw(derivedPath);
[&](const SingleDerivedPath::Opaque & o) { if (s != sExpected) {
auto sExpected = store->printStorePath(o.path); /* `std::visit` is used here just to provide a more precise
if (s != sExpected) error message. */
std::visit(overloaded {
[&](const SingleDerivedPath::Opaque & o) {
error( error(
"path string '%s' has context with the different path '%s'", "path string '%s' has context with the different path '%s'",
s, sExpected) s, sExpected)
.withTrace(pos, errorCtx).debugThrow<EvalError>(); .withTrace(pos, errorCtx).debugThrow<EvalError>();
}, },
[&](const SingleDerivedPath::Built & b) { [&](const SingleDerivedPath::Built & b) {
auto sExpected = std::visit(overloaded {
[&](const SingleDerivedPath::Opaque & o) {
auto drv = store->readDerivation(o.path);
auto i = drv.outputs.find(b.output);
if (i == drv.outputs.end())
throw Error("derivation '%s' does not have output '%s'", b.drvPath->to_string(*store), b.output);
auto optOutputPath = i->second.path(*store, drv.name, b.output);
// This is testing for the case of CA derivations
return optOutputPath
? store->printStorePath(*optOutputPath)
: DownstreamPlaceholder::fromSingleDerivedPathBuilt(b).render();
},
[&](const SingleDerivedPath::Built & o) {
return DownstreamPlaceholder::fromSingleDerivedPathBuilt(b).render();
},
}, b.drvPath->raw());
if (s != sExpected)
error( error(
"string '%s' has context with the output '%s' from derivation '%s', but the string is not the right placeholder for this derivation output. It should be '%s'", "string '%s' has context with the output '%s' from derivation '%s', but the string is not the right placeholder for this derivation output. It should be '%s'",
s, b.output, b.drvPath->to_string(*store), sExpected) s, b.output, b.drvPath->to_string(*store), sExpected)
.withTrace(pos, errorCtx).debugThrow<EvalError>(); .withTrace(pos, errorCtx).debugThrow<EvalError>();
} }
}, derivedPath.raw()); }, derivedPath.raw());
}
return derivedPath; return derivedPath;
} }

57
src/libexpr/eval.hh
View file

@ -341,8 +341,6 @@ public:
std::shared_ptr<Store> buildStore = nullptr); std::shared_ptr<Store> buildStore = nullptr);
~EvalState(); ~EvalState();
void addToSearchPath(SearchPath::Elem && elem);
SearchPath getSearchPath() { return searchPath; } SearchPath getSearchPath() { return searchPath; }
/** /**
@ -668,37 +666,46 @@ public:
/** /**
* Create a string representing a store path. * Create a string representing a store path.
* *
* The string is the printed store path with a context containing a single * The string is the printed store path with a context containing a
* `NixStringContextElem::Opaque` element of that store path. * single `NixStringContextElem::Opaque` element of that store path.
*/ */
void mkStorePathString(const StorePath & storePath, Value & v); void mkStorePathString(const StorePath & storePath, Value & v);
/** /**
* Create a string representing a `DerivedPath::Built`. * Create a string representing a `SingleDerivedPath::Built`.
* *
* The string is the printed store path with a context containing a single * The string is the printed store path with a context containing a
* `NixStringContextElem::Built` element of the drv path and output name. * single `NixStringContextElem::Built` element of the drv path and
* output name.
* *
* @param value Value we are settings * @param value Value we are settings
* *
* @param drvPath Path the drv whose output we are making a string for * @param b the drv whose output we are making a string for, and the
* output
* *
* @param outputName Name of the output * @param optStaticOutputPath Optional output path for that string.
* * Must be passed if and only if output store object is
* @param optOutputPath Optional output path for that string. Must * input-addressed or fixed output. Will be printed to form string
* be passed if and only if output store object is input-addressed. * if passed, otherwise a placeholder will be used (see
* Will be printed to form string if passed, otherwise a placeholder * `DownstreamPlaceholder`).
* will be used (see `DownstreamPlaceholder`).
* *
* @param xpSettings Stop-gap to avoid globals during unit tests. * @param xpSettings Stop-gap to avoid globals during unit tests.
*/ */
void mkOutputString( void mkOutputString(
Value & value, Value & value,
const StorePath & drvPath, const SingleDerivedPath::Built & b,
const std::string outputName, std::optional<StorePath> optStaticOutputPath,
std::optional<StorePath> optOutputPath,
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings); const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
/**
* Create a string representing a `SingleDerivedPath`.
*
* A combination of `mkStorePathString` and `mkOutputString`.
*/
void mkSingleDerivedPathString(
const SingleDerivedPath & p,
Value & v);
void concatLists(Value & v, size_t nrLists, Value * * lists, const PosIdx pos, std::string_view errorCtx); void concatLists(Value & v, size_t nrLists, Value * * lists, const PosIdx pos, std::string_view errorCtx);
/** /**
@ -714,6 +721,22 @@ public:
private: private:
/**
* Like `mkOutputString` but just creates a raw string, not an
* string Value, which would also have a string context.
*/
std::string mkOutputStringRaw(
const SingleDerivedPath::Built & b,
std::optional<StorePath> optStaticOutputPath,
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
/**
* Like `mkSingleDerivedPathStringRaw` but just creates a raw string
* Value, which would also have a string context.
*/
std::string mkSingleDerivedPathStringRaw(
const SingleDerivedPath & p);
unsigned long nrEnvs = 0; unsigned long nrEnvs = 0;
unsigned long nrValuesInEnvs = 0; unsigned long nrValuesInEnvs = 0;
unsigned long nrValues = 0; unsigned long nrValues = 0;

2
src/libexpr/flake/flakeref.cc
View file

@ -246,7 +246,7 @@ std::tuple<FlakeRef, std::string, ExtendedOutputsSpec> parseFlakeRefWithFragment
{ {
auto [prefix, extendedOutputsSpec] = ExtendedOutputsSpec::parse(url); auto [prefix, extendedOutputsSpec] = ExtendedOutputsSpec::parse(url);
auto [flakeRef, fragment] = parseFlakeRefWithFragment(std::string { prefix }, baseDir, allowMissing, isFlake); auto [flakeRef, fragment] = parseFlakeRefWithFragment(std::string { prefix }, baseDir, allowMissing, isFlake);
return {std::move(flakeRef), fragment, extendedOutputsSpec}; return {std::move(flakeRef), fragment, std::move(extendedOutputsSpec)};
} }
} }

6
src/libexpr/parser.y
View file

@ -738,12 +738,6 @@ Expr * EvalState::parseStdin()
} }
void EvalState::addToSearchPath(SearchPath::Elem && elem)
{
searchPath.elements.emplace_back(std::move(elem));
}
SourcePath EvalState::findFile(const std::string_view path) SourcePath EvalState::findFile(const std::string_view path)
{ {
return findFile(searchPath, path); return findFile(searchPath, path);

59
src/libexpr/primops.cc
View file

@ -69,7 +69,7 @@ StringMap EvalState::realiseContext(const NixStringContext & context)
res.insert_or_assign(ctxS, ctxS); res.insert_or_assign(ctxS, ctxS);
ensureValid(d.drvPath); ensureValid(d.drvPath);
}, },
}, c.raw()); }, c.raw);
} }
if (drvs.empty()) return {}; if (drvs.empty()) return {};
@ -156,8 +156,10 @@ static void mkOutputString(
{ {
state.mkOutputString( state.mkOutputString(
attrs.alloc(o.first), attrs.alloc(o.first),
drvPath, SingleDerivedPath::Built {
o.first, .drvPath = makeConstantStorePathRef(drvPath),
.output = o.first,
},
o.second.path(*state.store, Derivation::nameFromPath(drvPath), o.first)); o.second.path(*state.store, Derivation::nameFromPath(drvPath), o.first));
} }
@ -1263,7 +1265,7 @@ drvName, Bindings * attrs, Value & v)
[&](const NixStringContextElem::Opaque & o) { [&](const NixStringContextElem::Opaque & o) {
drv.inputSrcs.insert(o.path); drv.inputSrcs.insert(o.path);
}, },
}, c.raw()); }, c.raw);
} }
/* Do we have all required attributes? */ /* Do we have all required attributes? */
@ -1332,13 +1334,13 @@ drvName, Bindings * attrs, Value & v)
if (isImpure) if (isImpure)
drv.outputs.insert_or_assign(i, drv.outputs.insert_or_assign(i,
DerivationOutput::Impure { DerivationOutput::Impure {
.method = method.raw, .method = method,
.hashType = ht, .hashType = ht,
}); });
else else
drv.outputs.insert_or_assign(i, drv.outputs.insert_or_assign(i,
DerivationOutput::CAFloating { DerivationOutput::CAFloating {
.method = method.raw, .method = method,
.hashType = ht, .hashType = ht,
}); });
} }
@ -1371,7 +1373,7 @@ drvName, Bindings * attrs, Value & v)
drv.env[i] = state.store->printStorePath(outPath); drv.env[i] = state.store->printStorePath(outPath);
drv.outputs.insert_or_assign( drv.outputs.insert_or_assign(
i, i,
DerivationOutputInputAddressed { DerivationOutput::InputAddressed {
.path = std::move(outPath), .path = std::move(outPath),
}); });
} }
@ -1379,7 +1381,7 @@ drvName, Bindings * attrs, Value & v)
; ;
case DrvHash::Kind::Deferred: case DrvHash::Kind::Deferred:
for (auto & i : outputs) { for (auto & i : outputs) {
drv.outputs.insert_or_assign(i, DerivationOutputDeferred {}); drv.outputs.insert_or_assign(i, DerivationOutput::Deferred {});
} }
} }
} }
@ -1836,6 +1838,45 @@ static RegisterPrimOp primop_readDir({
.fun = prim_readDir, .fun = prim_readDir,
}); });
/* Extend single element string context with another output. */
static void prim_outputOf(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
SingleDerivedPath drvPath = state.coerceToSingleDerivedPath(pos, *args[0], "while evaluating the first argument to builtins.outputOf");
std::string_view outputName = state.forceStringNoCtx(*args[1], pos, "while evaluating the second argument to builtins.outputOf");
state.mkSingleDerivedPathString(
SingleDerivedPath::Built {
.drvPath = make_ref<SingleDerivedPath>(drvPath),
.output = std::string { outputName },
},
v);
}
static RegisterPrimOp primop_outputOf({
.name = "__outputOf",
.args = {"derivation-reference", "output-name"},
.doc = R"(
Return the output path of a derivation, literally or using a placeholder if needed.
If the derivation has a statically-known output path (i.e. the derivation output is input-addressed, or fixed content-addresed), the output path will just be returned.
But if the derivation is content-addressed or if the derivation is itself not-statically produced (i.e. is the output of another derivation), a placeholder will be returned instead.
*`derivation reference`* must be a string that may contain a regular store path to a derivation, or may be a placeholder reference. If the derivation is produced by a derivation, you must explicitly select `drv.outPath`.
This primop can be chained arbitrarily deeply.
For instance,
```nix
builtins.outputOf
(builtins.outputOf myDrv "out)
"out"
```
will return a placeholder for the output of the output of `myDrv`.
This primop corresponds to the `^` sigil for derivable paths, e.g. as part of installable syntax on the command line.
)",
.fun = prim_outputOf,
.experimentalFeature = Xp::DynamicDerivations,
});
/************************************************************* /*************************************************************
* Creating files * Creating files
@ -2013,7 +2054,7 @@ static void prim_toFile(EvalState & state, const PosIdx pos, Value * * args, Val
StorePathSet refs; StorePathSet refs;
for (auto c : context) { for (auto c : context) {
if (auto p = std::get_if<NixStringContextElem::Opaque>(&c)) if (auto p = std::get_if<NixStringContextElem::Opaque>(&c.raw))
refs.insert(p->path); refs.insert(p->path);
else else
state.debugThrowLastTrace(EvalError({ state.debugThrowLastTrace(EvalError({

6
src/libexpr/primops/context.cc
View file

@ -51,13 +51,13 @@ static void prim_unsafeDiscardOutputDependency(EvalState & state, const PosIdx p
NixStringContext context2; NixStringContext context2;
for (auto && c : context) { for (auto && c : context) {
if (auto * ptr = std::get_if<NixStringContextElem::DrvDeep>(&c)) { if (auto * ptr = std::get_if<NixStringContextElem::DrvDeep>(&c.raw)) {
context2.emplace(NixStringContextElem::Opaque { context2.emplace(NixStringContextElem::Opaque {
.path = ptr->drvPath .path = ptr->drvPath
}); });
} else { } else {
/* Can reuse original item */ /* Can reuse original item */
context2.emplace(std::move(c)); context2.emplace(std::move(c).raw);
} }
} }
@ -114,7 +114,7 @@ static void prim_getContext(EvalState & state, const PosIdx pos, Value * * args,
[&](NixStringContextElem::Opaque && o) { [&](NixStringContextElem::Opaque && o) {
contextInfos[std::move(o.path)].path = true; contextInfos[std::move(o.path)].path = true;
}, },
}, ((NixStringContextElem &&) i).raw()); }, ((NixStringContextElem &&) i).raw);
} }
auto attrs = state.buildBindings(contextInfos.size()); auto attrs = state.buildBindings(contextInfos.size());

2
src/libexpr/search-path.cc
View file

@ -10,7 +10,7 @@ std::optional<std::string_view> SearchPath::Prefix::suffixIfPotentialMatch(
/* Non-empty prefix and suffix must be separated by a /, or the /* Non-empty prefix and suffix must be separated by a /, or the
prefix is not a valid path prefix. */ prefix is not a valid path prefix. */
bool needSeparator = n > 0 && (path.size() - n) > 0; bool needSeparator = n > 0 && n < path.size();
if (needSeparator && path[n] != '/') { if (needSeparator && path[n] != '/') {
return std::nullopt; return std::nullopt;

20
src/libexpr/tests/derived-path.cc
View file

@ -34,8 +34,8 @@ RC_GTEST_FIXTURE_PROP(
RC_GTEST_FIXTURE_PROP( RC_GTEST_FIXTURE_PROP(
DerivedPathExpressionTest, DerivedPathExpressionTest,
prop_built_path_placeholder_round_trip, prop_derived_path_built_placeholder_round_trip,
(const StorePath & drvPath, const StorePathName & outputName)) (const SingleDerivedPath::Built & b))
{ {
/** /**
* We set these in tests rather than the regular globals so we don't have * We set these in tests rather than the regular globals so we don't have
@ -45,27 +45,19 @@ RC_GTEST_FIXTURE_PROP(
mockXpSettings.set("experimental-features", "ca-derivations"); mockXpSettings.set("experimental-features", "ca-derivations");
auto * v = state.allocValue(); auto * v = state.allocValue();
state.mkOutputString(*v, drvPath, outputName.name, std::nullopt, mockXpSettings); state.mkOutputString(*v, b, std::nullopt, mockXpSettings);
auto [d, _] = state.coerceToSingleDerivedPathUnchecked(noPos, *v, ""); auto [d, _] = state.coerceToSingleDerivedPathUnchecked(noPos, *v, "");
SingleDerivedPath::Built b {
.drvPath = makeConstantStorePathRef(drvPath),
.output = outputName.name,
};
RC_ASSERT(SingleDerivedPath { b } == d); RC_ASSERT(SingleDerivedPath { b } == d);
} }
RC_GTEST_FIXTURE_PROP( RC_GTEST_FIXTURE_PROP(
DerivedPathExpressionTest, DerivedPathExpressionTest,
prop_built_path_out_path_round_trip, prop_derived_path_built_out_path_round_trip,
(const StorePath & drvPath, const StorePathName & outputName, const StorePath & outPath)) (const SingleDerivedPath::Built & b, const StorePath & outPath))
{ {
auto * v = state.allocValue(); auto * v = state.allocValue();
state.mkOutputString(*v, drvPath, outputName.name, outPath); state.mkOutputString(*v, b, outPath);
auto [d, _] = state.coerceToSingleDerivedPathUnchecked(noPos, *v, ""); auto [d, _] = state.coerceToSingleDerivedPathUnchecked(noPos, *v, "");
SingleDerivedPath::Built b {
.drvPath = makeConstantStorePathRef(drvPath),
.output = outputName.name,
};
RC_ASSERT(SingleDerivedPath { b } == d); RC_ASSERT(SingleDerivedPath { b } == d);
} }

22
src/libexpr/tests/flakeref.cc Normal file
View file

@ -0,0 +1,22 @@
#include <gtest/gtest.h>
#include "flake/flakeref.hh"
namespace nix {
/* ----------- tests for flake/flakeref.hh --------------------------------------------------*/
/* ----------------------------------------------------------------------------
* to_string
* --------------------------------------------------------------------------*/
TEST(to_string, doesntReencodeUrl) {
auto s = "http://localhost:8181/test/+3d.tar.gz";
auto flakeref = parseFlakeRef(s);
auto parsed = flakeref.to_string();
auto expected = "http://localhost:8181/test/%2B3d.tar.gz";
ASSERT_EQ(parsed, expected);
}
}

8
src/libexpr/tests/value/context.cc
View file

@ -47,7 +47,7 @@ TEST(NixStringContextElemTest, slash_invalid) {
TEST(NixStringContextElemTest, opaque) { TEST(NixStringContextElemTest, opaque) {
std::string_view opaque = "g1w7hy3qg1w7hy3qg1w7hy3qg1w7hy3q-x"; std::string_view opaque = "g1w7hy3qg1w7hy3qg1w7hy3qg1w7hy3q-x";
auto elem = NixStringContextElem::parse(opaque); auto elem = NixStringContextElem::parse(opaque);
auto * p = std::get_if<NixStringContextElem::Opaque>(&elem); auto * p = std::get_if<NixStringContextElem::Opaque>(&elem.raw);
ASSERT_TRUE(p); ASSERT_TRUE(p);
ASSERT_EQ(p->path, StorePath { opaque }); ASSERT_EQ(p->path, StorePath { opaque });
ASSERT_EQ(elem.to_string(), opaque); ASSERT_EQ(elem.to_string(), opaque);
@ -60,7 +60,7 @@ TEST(NixStringContextElemTest, opaque) {
TEST(NixStringContextElemTest, drvDeep) { TEST(NixStringContextElemTest, drvDeep) {
std::string_view drvDeep = "=g1w7hy3qg1w7hy3qg1w7hy3qg1w7hy3q-x.drv"; std::string_view drvDeep = "=g1w7hy3qg1w7hy3qg1w7hy3qg1w7hy3q-x.drv";
auto elem = NixStringContextElem::parse(drvDeep); auto elem = NixStringContextElem::parse(drvDeep);
auto * p = std::get_if<NixStringContextElem::DrvDeep>(&elem); auto * p = std::get_if<NixStringContextElem::DrvDeep>(&elem.raw);
ASSERT_TRUE(p); ASSERT_TRUE(p);
ASSERT_EQ(p->drvPath, StorePath { drvDeep.substr(1) }); ASSERT_EQ(p->drvPath, StorePath { drvDeep.substr(1) });
ASSERT_EQ(elem.to_string(), drvDeep); ASSERT_EQ(elem.to_string(), drvDeep);
@ -73,7 +73,7 @@ TEST(NixStringContextElemTest, drvDeep) {
TEST(NixStringContextElemTest, built_opaque) { TEST(NixStringContextElemTest, built_opaque) {
std::string_view built = "!foo!g1w7hy3qg1w7hy3qg1w7hy3qg1w7hy3q-x.drv"; std::string_view built = "!foo!g1w7hy3qg1w7hy3qg1w7hy3qg1w7hy3q-x.drv";
auto elem = NixStringContextElem::parse(built); auto elem = NixStringContextElem::parse(built);
auto * p = std::get_if<NixStringContextElem::Built>(&elem); auto * p = std::get_if<NixStringContextElem::Built>(&elem.raw);
ASSERT_TRUE(p); ASSERT_TRUE(p);
ASSERT_EQ(p->output, "foo"); ASSERT_EQ(p->output, "foo");
ASSERT_EQ(*p->drvPath, ((SingleDerivedPath) SingleDerivedPath::Opaque { ASSERT_EQ(*p->drvPath, ((SingleDerivedPath) SingleDerivedPath::Opaque {
@ -96,7 +96,7 @@ TEST(NixStringContextElemTest, built_built) {
std::string_view built = "!foo!bar!g1w7hy3qg1w7hy3qg1w7hy3qg1w7hy3q-x.drv"; std::string_view built = "!foo!bar!g1w7hy3qg1w7hy3qg1w7hy3qg1w7hy3q-x.drv";
auto elem = NixStringContextElem::parse(built, mockXpSettings); auto elem = NixStringContextElem::parse(built, mockXpSettings);
auto * p = std::get_if<NixStringContextElem::Built>(&elem); auto * p = std::get_if<NixStringContextElem::Built>(&elem.raw);
ASSERT_TRUE(p); ASSERT_TRUE(p);
ASSERT_EQ(p->output, "foo"); ASSERT_EQ(p->output, "foo");
auto * drvPath = std::get_if<SingleDerivedPath::Built>(&*p->drvPath); auto * drvPath = std::get_if<SingleDerivedPath::Built>(&*p->drvPath);

2
src/libexpr/value/context.cc
View file

@ -99,7 +99,7 @@ std::string NixStringContextElem::to_string() const
res += '='; res += '=';
res += d.drvPath.to_string(); res += d.drvPath.to_string();
}, },
}, raw()); }, raw);
return res; return res;
} }

82
src/libexpr/value/context.hh
View file

@ -4,8 +4,7 @@
#include "util.hh" #include "util.hh"
#include "comparator.hh" #include "comparator.hh"
#include "derived-path.hh" #include "derived-path.hh"
#include "variant-wrapper.hh"
#include <variant>
#include <nlohmann/json_fwd.hpp> #include <nlohmann/json_fwd.hpp>
@ -26,58 +25,47 @@ public:
} }
}; };
/** struct NixStringContextElem {
* Plain opaque path to some store object. /**
* * Plain opaque path to some store object.
* Encoded as just the path: ‘<path>’. *
*/ * Encoded as just the path: ‘<path>’.
typedef SingleDerivedPath::Opaque NixStringContextElem_Opaque; */
using Opaque = SingleDerivedPath::Opaque;
/** /**
* Path to a derivation and its entire build closure. * Path to a derivation and its entire build closure.
* *
* The path doesn't just refer to derivation itself and its closure, but * The path doesn't just refer to derivation itself and its closure, but
* also all outputs of all derivations in that closure (including the * also all outputs of all derivations in that closure (including the
* root derivation). * root derivation).
* *
* Encoded in the form ‘=<drvPath>’. * Encoded in the form ‘=<drvPath>’.
*/ */
struct NixStringContextElem_DrvDeep { struct DrvDeep {
StorePath drvPath; StorePath drvPath;
GENERATE_CMP(NixStringContextElem_DrvDeep, me->drvPath); GENERATE_CMP(DrvDeep, me->drvPath);
}; };
/** /**
* Derivation output. * Derivation output.
* *
* Encoded in the form ‘!<output>!<drvPath>’. * Encoded in the form ‘!<output>!<drvPath>’.
*/ */
typedef SingleDerivedPath::Built NixStringContextElem_Built; using Built = SingleDerivedPath::Built;
using _NixStringContextElem_Raw = std::variant< using Raw = std::variant<
NixStringContextElem_Opaque, Opaque,
NixStringContextElem_DrvDeep, DrvDeep,
NixStringContextElem_Built Built
>; >;
struct NixStringContextElem : _NixStringContextElem_Raw { Raw raw;
using Raw = _NixStringContextElem_Raw;
using Raw::Raw;
using Opaque = NixStringContextElem_Opaque; GENERATE_CMP(NixStringContextElem, me->raw);
using DrvDeep = NixStringContextElem_DrvDeep;
using Built = NixStringContextElem_Built;
inline const Raw & raw() const & { MAKE_WRAPPER_CONSTRUCTOR(NixStringContextElem);
return static_cast<const Raw &>(*this);
}
inline Raw & raw() & {
return static_cast<Raw &>(*this);
}
inline Raw && raw() && {
return static_cast<Raw &&>(*this);
}
/** /**
* Decode a context string, one of: * Decode a context string, one of:

3
src/libmain/progress-bar.cc
View file

@ -108,7 +108,8 @@ public:
stop(); stop();
} }
void stop() override /* Called by destructor, can't be overridden */
void stop() override final
{ {
{ {
auto state(state_.lock()); auto state(state_.lock());

7
src/libstore/build/derivation-goal.cc
View file

@ -521,7 +521,7 @@ void DerivationGoal::inputsRealised()
[&](const DerivationType::Impure &) { [&](const DerivationType::Impure &) {
return true; return true;
} }
}, drvType.raw()); }, drvType.raw);
if (resolveDrv && !fullDrv.inputDrvs.empty()) { if (resolveDrv && !fullDrv.inputDrvs.empty()) {
experimentalFeatureSettings.require(Xp::CaDerivations); experimentalFeatureSettings.require(Xp::CaDerivations);
@ -996,10 +996,11 @@ void DerivationGoal::buildDone()
} }
else { else {
assert(derivationType);
st = st =
dynamic_cast<NotDeterministic*>(&e) ? BuildResult::NotDeterministic : dynamic_cast<NotDeterministic*>(&e) ? BuildResult::NotDeterministic :
statusOk(status) ? BuildResult::OutputRejected : statusOk(status) ? BuildResult::OutputRejected :
!derivationType.isSandboxed() || diskFull ? BuildResult::TransientFailure : !derivationType->isSandboxed() || diskFull ? BuildResult::TransientFailure :
BuildResult::PermanentFailure; BuildResult::PermanentFailure;
} }
@ -1358,7 +1359,7 @@ std::pair<bool, SingleDrvOutputs> DerivationGoal::checkPathValidity()
[&](const OutputsSpec::Names & names) { [&](const OutputsSpec::Names & names) {
return static_cast<StringSet>(names); return static_cast<StringSet>(names);
}, },
}, wantedOutputs.raw()); }, wantedOutputs.raw);
SingleDrvOutputs validOutputs; SingleDrvOutputs validOutputs;
for (auto & i : queryPartialDerivationOutputMap()) { for (auto & i : queryPartialDerivationOutputMap()) {

2
src/libstore/build/derivation-goal.hh
View file

@ -184,7 +184,7 @@ struct DerivationGoal : public Goal
/** /**
* The sort of derivation we are building. * The sort of derivation we are building.
*/ */
DerivationType derivationType; std::optional<DerivationType> derivationType;
typedef void (DerivationGoal::*GoalState)(); typedef void (DerivationGoal::*GoalState)();
GoalState state; GoalState state;

4
src/libstore/build/goal.hh
View file

@ -162,6 +162,10 @@ public:
virtual void cleanup() { } virtual void cleanup() { }
/**
* @brief Hint for the scheduler, which concurrency limit applies.
* @see JobCategory
*/
virtual JobCategory jobCategory() = 0; virtual JobCategory jobCategory() = 0;
}; };

18
src/libstore/build/local-derivation-goal.cc
View file

@ -178,6 +178,8 @@ void LocalDerivationGoal::tryLocalBuild()
return; return;
} }
assert(derivationType);
/* Are we doing a chroot build? */ /* Are we doing a chroot build? */
{ {
auto noChroot = parsedDrv->getBoolAttr("__noChroot"); auto noChroot = parsedDrv->getBoolAttr("__noChroot");
@ -195,7 +197,7 @@ void LocalDerivationGoal::tryLocalBuild()
else if (settings.sandboxMode == smDisabled) else if (settings.sandboxMode == smDisabled)
useChroot = false; useChroot = false;
else if (settings.sandboxMode == smRelaxed) else if (settings.sandboxMode == smRelaxed)
useChroot = derivationType.isSandboxed() && !noChroot; useChroot = derivationType->isSandboxed() && !noChroot;
} }
auto & localStore = getLocalStore(); auto & localStore = getLocalStore();
@ -689,7 +691,7 @@ void LocalDerivationGoal::startBuilder()
"nogroup:x:65534:\n", sandboxGid())); "nogroup:x:65534:\n", sandboxGid()));
/* Create /etc/hosts with localhost entry. */ /* Create /etc/hosts with localhost entry. */
if (derivationType.isSandboxed()) if (derivationType->isSandboxed())
writeFile(chrootRootDir + "/etc/hosts", "127.0.0.1 localhost\n::1 localhost\n"); writeFile(chrootRootDir + "/etc/hosts", "127.0.0.1 localhost\n::1 localhost\n");
/* Make the closure of the inputs available in the chroot, /* Make the closure of the inputs available in the chroot,
@ -893,7 +895,7 @@ void LocalDerivationGoal::startBuilder()
us. us.
*/ */
if (derivationType.isSandboxed()) if (derivationType->isSandboxed())
privateNetwork = true; privateNetwork = true;
userNamespaceSync.create(); userNamespaceSync.create();
@ -1121,7 +1123,7 @@ void LocalDerivationGoal::initEnv()
derivation, tell the builder, so that for instance `fetchurl' derivation, tell the builder, so that for instance `fetchurl'
can skip checking the output. On older Nixes, this environment can skip checking the output. On older Nixes, this environment
variable won't be set, so `fetchurl' will do the check. */ variable won't be set, so `fetchurl' will do the check. */
if (derivationType.isFixed()) env["NIX_OUTPUT_CHECKED"] = "1"; if (derivationType->isFixed()) env["NIX_OUTPUT_CHECKED"] = "1";
/* *Only* if this is a fixed-output derivation, propagate the /* *Only* if this is a fixed-output derivation, propagate the
values of the environment variables specified in the values of the environment variables specified in the
@ -1132,7 +1134,7 @@ void LocalDerivationGoal::initEnv()
to the builder is generally impure, but the output of to the builder is generally impure, but the output of
fixed-output derivations is by definition pure (since we fixed-output derivations is by definition pure (since we
already know the cryptographic hash of the output). */ already know the cryptographic hash of the output). */
if (!derivationType.isSandboxed()) { if (!derivationType->isSandboxed()) {
for (auto & i : parsedDrv->getStringsAttr("impureEnvVars").value_or(Strings())) for (auto & i : parsedDrv->getStringsAttr("impureEnvVars").value_or(Strings()))
env[i] = getEnv(i).value_or(""); env[i] = getEnv(i).value_or("");
} }
@ -1797,7 +1799,7 @@ void LocalDerivationGoal::runChild()
/* Fixed-output derivations typically need to access the /* Fixed-output derivations typically need to access the
network, so give them access to /etc/resolv.conf and so network, so give them access to /etc/resolv.conf and so
on. */ on. */
if (!derivationType.isSandboxed()) { if (!derivationType->isSandboxed()) {
// Only use nss functions to resolve hosts and // Only use nss functions to resolve hosts and
// services. Don’t use it for anything else that may // services. Don’t use it for anything else that may
// be configured for this system. This limits the // be configured for this system. This limits the
@ -2048,7 +2050,7 @@ void LocalDerivationGoal::runChild()
#include "sandbox-defaults.sb" #include "sandbox-defaults.sb"
; ;
if (!derivationType.isSandboxed()) if (!derivationType->isSandboxed())
sandboxProfile += sandboxProfile +=
#include "sandbox-network.sb" #include "sandbox-network.sb"
; ;
@ -2599,7 +2601,7 @@ SingleDrvOutputs LocalDerivationGoal::registerOutputs()
}); });
}, },
}, output->raw()); }, output->raw);
/* FIXME: set proper permissions in restorePath() so /* FIXME: set proper permissions in restorePath() so
we don't have to do another traversal. */ we don't have to do another traversal. */

4
src/libstore/build/local-derivation-goal.hh
View file

@ -272,8 +272,10 @@ struct LocalDerivationGoal : public DerivationGoal
/** /**
* Forcibly kill the child process, if any. * Forcibly kill the child process, if any.
*
* Called by destructor, can't be overridden
*/ */
void killChild() override; void killChild() override final;
/** /**
* Kill any processes running under the build user UID or in the * Kill any processes running under the build user UID or in the

3
src/libstore/build/substitution-goal.hh
View file

@ -114,7 +114,8 @@ public:
void handleChildOutput(int fd, std::string_view data) override; void handleChildOutput(int fd, std::string_view data) override;
void handleEOF(int fd) override; void handleEOF(int fd) override;
void cleanup() override; /* Called by destructor, can't be overridden */
void cleanup() override final;
JobCategory jobCategory() override { return JobCategory::Substitution; }; JobCategory jobCategory() override { return JobCategory::Substitution; };
}; };

5
src/libstore/build/worker.cc
View file

@ -268,7 +268,10 @@ void Worker::run(const Goals & _topGoals)
for (auto & i : _topGoals) { for (auto & i : _topGoals) {
topGoals.insert(i); topGoals.insert(i);
if (auto goal = dynamic_cast<DerivationGoal *>(i.get())) { if (auto goal = dynamic_cast<DerivationGoal *>(i.get())) {
topPaths.push_back(DerivedPath::Built{makeConstantStorePathRef(goal->drvPath), goal->wantedOutputs}); topPaths.push_back(DerivedPath::Built {
.drvPath = makeConstantStorePathRef(goal->drvPath),
.outputs = goal->wantedOutputs,
});
} else if (auto goal = dynamic_cast<PathSubstitutionGoal *>(i.get())) { } else if (auto goal = dynamic_cast<PathSubstitutionGoal *>(i.get())) {
topPaths.push_back(DerivedPath::Opaque{goal->storePath}); topPaths.push_back(DerivedPath::Opaque{goal->storePath});
} }

2
src/libstore/content-address.cc
View file

@ -115,7 +115,7 @@ ContentAddress ContentAddress::parse(std::string_view rawCa)
auto [caMethod, hashType] = parseContentAddressMethodPrefix(rest); auto [caMethod, hashType] = parseContentAddressMethodPrefix(rest);
return ContentAddress { return ContentAddress {
.method = std::move(caMethod).raw, .method = std::move(caMethod),
.hash = Hash::parseNonSRIUnprefixed(rest, hashType), .hash = Hash::parseNonSRIUnprefixed(rest, hashType),
}; };
} }

12
src/libstore/content-address.hh
View file

@ -5,6 +5,7 @@
#include "hash.hh" #include "hash.hh"
#include "path.hh" #include "path.hh"
#include "comparator.hh" #include "comparator.hh"
#include "variant-wrapper.hh"
namespace nix { namespace nix {
@ -71,11 +72,7 @@ struct ContentAddressMethod
GENERATE_CMP(ContentAddressMethod, me->raw); GENERATE_CMP(ContentAddressMethod, me->raw);
/* The moral equivalent of `using Raw::Raw;` */ MAKE_WRAPPER_CONSTRUCTOR(ContentAddressMethod);
ContentAddressMethod(auto &&... arg)
: raw(std::forward<decltype(arg)>(arg)...)
{ }
/** /**
* Parse the prefix tag which indicates how the files * Parse the prefix tag which indicates how the files
@ -252,10 +249,7 @@ struct ContentAddressWithReferences
GENERATE_CMP(ContentAddressWithReferences, me->raw); GENERATE_CMP(ContentAddressWithReferences, me->raw);
/* The moral equivalent of `using Raw::Raw;` */ MAKE_WRAPPER_CONSTRUCTOR(ContentAddressWithReferences);
ContentAddressWithReferences(auto &&... arg)
: raw(std::forward<decltype(arg)>(arg)...)
{ }
/** /**
* Create a `ContentAddressWithReferences` from a mere * Create a `ContentAddressWithReferences` from a mere

36
src/libstore/derivations.cc
View file

@ -32,7 +32,7 @@ std::optional<StorePath> DerivationOutput::path(const Store & store, std::string
[](const DerivationOutput::Impure &) -> std::optional<StorePath> { [](const DerivationOutput::Impure &) -> std::optional<StorePath> {
return std::nullopt; return std::nullopt;
}, },
}, raw()); }, raw);
} }
@ -60,7 +60,7 @@ bool DerivationType::isCA() const
[](const Impure &) { [](const Impure &) {
return true; return true;
}, },
}, raw()); }, raw);
} }
bool DerivationType::isFixed() const bool DerivationType::isFixed() const
@ -75,7 +75,7 @@ bool DerivationType::isFixed() const
[](const Impure &) { [](const Impure &) {
return false; return false;
}, },
}, raw()); }, raw);
} }
bool DerivationType::hasKnownOutputPaths() const bool DerivationType::hasKnownOutputPaths() const
@ -90,7 +90,7 @@ bool DerivationType::hasKnownOutputPaths() const
[](const Impure &) { [](const Impure &) {
return false; return false;
}, },
}, raw()); }, raw);
} }
@ -106,7 +106,7 @@ bool DerivationType::isSandboxed() const
[](const Impure &) { [](const Impure &) {
return false; return false;
}, },
}, raw()); }, raw);
} }
@ -122,7 +122,7 @@ bool DerivationType::isPure() const
[](const Impure &) { [](const Impure &) {
return false; return false;
}, },
}, raw()); }, raw);
} }
@ -408,13 +408,13 @@ std::string Derivation::unparse(const Store & store, bool maskOutputs,
s += ','; printUnquotedString(s, ""); s += ','; printUnquotedString(s, "");
s += ','; printUnquotedString(s, ""); s += ','; printUnquotedString(s, "");
}, },
[&](const DerivationOutputImpure & doi) { [&](const DerivationOutput::Impure & doi) {
// FIXME // FIXME
s += ','; printUnquotedString(s, ""); s += ','; printUnquotedString(s, "");
s += ','; printUnquotedString(s, doi.method.renderPrefix() + printHashType(doi.hashType)); s += ','; printUnquotedString(s, doi.method.renderPrefix() + printHashType(doi.hashType));
s += ','; printUnquotedString(s, "impure"); s += ','; printUnquotedString(s, "impure");
} }
}, i.second.raw()); }, i.second.raw);
s += ')'; s += ')';
} }
@ -509,7 +509,7 @@ DerivationType BasicDerivation::type() const
[&](const DerivationOutput::Impure &) { [&](const DerivationOutput::Impure &) {
impureOutputs.insert(i.first); impureOutputs.insert(i.first);
}, },
}, i.second.raw()); }, i.second.raw);
} }
if (inputAddressedOutputs.empty() if (inputAddressedOutputs.empty()
@ -626,7 +626,7 @@ DrvHash hashDerivationModulo(Store & store, const Derivation & drv, bool maskOut
if (type.isFixed()) { if (type.isFixed()) {
std::map<std::string, Hash> outputHashes; std::map<std::string, Hash> outputHashes;
for (const auto & i : drv.outputs) { for (const auto & i : drv.outputs) {
auto & dof = std::get<DerivationOutput::CAFixed>(i.second.raw()); auto & dof = std::get<DerivationOutput::CAFixed>(i.second.raw);
auto hash = hashString(htSHA256, "fixed:out:" auto hash = hashString(htSHA256, "fixed:out:"
+ dof.ca.printMethodAlgo() + ":" + dof.ca.printMethodAlgo() + ":"
+ dof.ca.hash.to_string(Base16, false) + ":" + dof.ca.hash.to_string(Base16, false) + ":"
@ -663,7 +663,7 @@ DrvHash hashDerivationModulo(Store & store, const Derivation & drv, bool maskOut
[](const DerivationType::Impure &) -> DrvHash::Kind { [](const DerivationType::Impure &) -> DrvHash::Kind {
assert(false); assert(false);
} }
}, drv.type().raw()); }, drv.type().raw);
std::map<std::string, StringSet> inputs2; std::map<std::string, StringSet> inputs2;
for (auto & [drvPath, inputOutputs0] : drv.inputDrvs) { for (auto & [drvPath, inputOutputs0] : drv.inputDrvs) {
@ -720,10 +720,10 @@ StringSet BasicDerivation::outputNames() const
DerivationOutputsAndOptPaths BasicDerivation::outputsAndOptPaths(const Store & store) const DerivationOutputsAndOptPaths BasicDerivation::outputsAndOptPaths(const Store & store) const
{ {
DerivationOutputsAndOptPaths outsAndOptPaths; DerivationOutputsAndOptPaths outsAndOptPaths;
for (auto output : outputs) for (auto & [outputName, output] : outputs)
outsAndOptPaths.insert(std::make_pair( outsAndOptPaths.insert(std::make_pair(
output.first, outputName,
std::make_pair(output.second, output.second.path(store, name, output.first)) std::make_pair(output, output.path(store, name, outputName))
) )
); );
return outsAndOptPaths; return outsAndOptPaths;
@ -798,7 +798,7 @@ void writeDerivation(Sink & out, const Store & store, const BasicDerivation & dr
<< (doi.method.renderPrefix() + printHashType(doi.hashType)) << (doi.method.renderPrefix() + printHashType(doi.hashType))
<< "impure"; << "impure";
}, },
}, i.second.raw()); }, i.second.raw);
} }
WorkerProto::write(store, WorkerProto::write(store,
WorkerProto::WriteConn { .to = out }, WorkerProto::WriteConn { .to = out },
@ -840,7 +840,7 @@ static void rewriteDerivation(Store & store, BasicDerivation & drv, const String
auto hashModulo = hashDerivationModulo(store, Derivation(drv), true); auto hashModulo = hashDerivationModulo(store, Derivation(drv), true);
for (auto & [outputName, output] : drv.outputs) { for (auto & [outputName, output] : drv.outputs) {
if (std::holds_alternative<DerivationOutput::Deferred>(output.raw())) { if (std::holds_alternative<DerivationOutput::Deferred>(output.raw)) {
auto h = get(hashModulo.hashes, outputName); auto h = get(hashModulo.hashes, outputName);
if (!h) if (!h)
throw Error("derivation '%s' output '%s' has no hash (derivations.cc/rewriteDerivation)", throw Error("derivation '%s' output '%s' has no hash (derivations.cc/rewriteDerivation)",
@ -955,7 +955,7 @@ void Derivation::checkInvariants(Store & store, const StorePath & drvPath) const
[&](const DerivationOutput::Impure &) { [&](const DerivationOutput::Impure &) {
/* Nothing to check */ /* Nothing to check */
}, },
}, i.second.raw()); }, i.second.raw);
} }
} }
@ -984,7 +984,7 @@ nlohmann::json DerivationOutput::toJSON(
res["hashAlgo"] = doi.method.renderPrefix() + printHashType(doi.hashType); res["hashAlgo"] = doi.method.renderPrefix() + printHashType(doi.hashType);
res["impure"] = true; res["impure"] = true;
}, },
}, raw()); }, raw);
return res; return res;
} }

311
src/libstore/derivations.hh
View file

@ -9,6 +9,7 @@
#include "derived-path.hh" #include "derived-path.hh"
#include "sync.hh" #include "sync.hh"
#include "comparator.hh" #include "comparator.hh"
#include "variant-wrapper.hh"
#include <map> #include <map>
#include <variant> #include <variant>
@ -20,108 +21,110 @@ class Store;
/* Abstract syntax of derivations. */ /* Abstract syntax of derivations. */
/**
* The traditional non-fixed-output derivation type.
*/
struct DerivationOutputInputAddressed
{
StorePath path;
GENERATE_CMP(DerivationOutputInputAddressed, me->path);
};
/**
* Fixed-output derivations, whose output paths are content
* addressed according to that fixed output.
*/
struct DerivationOutputCAFixed
{
/**
* Method and hash used for expected hash computation.
*
* References are not allowed by fiat.
*/
ContentAddress ca;
/**
* Return the \ref StorePath "store path" corresponding to this output
*
* @param drvName The name of the derivation this is an output of, without the `.drv`.
* @param outputName The name of this output.
*/
StorePath path(const Store & store, std::string_view drvName, std::string_view outputName) const;
GENERATE_CMP(DerivationOutputCAFixed, me->ca);
};
/**
* Floating-output derivations, whose output paths are content
* addressed, but not fixed, and so are dynamically calculated from
* whatever the output ends up being.
* */
struct DerivationOutputCAFloating
{
/**
* How the file system objects will be serialized for hashing
*/
ContentAddressMethod method;
/**
* How the serialization will be hashed
*/
HashType hashType;
GENERATE_CMP(DerivationOutputCAFloating, me->method, me->hashType);
};
/**
* Input-addressed output which depends on a (CA) derivation whose hash
* isn't known yet.
*/
struct DerivationOutputDeferred {
GENERATE_CMP(DerivationOutputDeferred);
};
/**
* Impure output which is moved to a content-addressed location (like
* CAFloating) but isn't registered as a realization.
*/
struct DerivationOutputImpure
{
/**
* How the file system objects will be serialized for hashing
*/
ContentAddressMethod method;
/**
* How the serialization will be hashed
*/
HashType hashType;
GENERATE_CMP(DerivationOutputImpure, me->method, me->hashType);
};
typedef std::variant<
DerivationOutputInputAddressed,
DerivationOutputCAFixed,
DerivationOutputCAFloating,
DerivationOutputDeferred,
DerivationOutputImpure
> _DerivationOutputRaw;
/** /**
* A single output of a BasicDerivation (and Derivation). * A single output of a BasicDerivation (and Derivation).
*/ */
struct DerivationOutput : _DerivationOutputRaw struct DerivationOutput
{ {
using Raw = _DerivationOutputRaw; /**
using Raw::Raw; * The traditional non-fixed-output derivation type.
*/
struct InputAddressed
{
StorePath path;
using InputAddressed = DerivationOutputInputAddressed; GENERATE_CMP(InputAddressed, me->path);
using CAFixed = DerivationOutputCAFixed; };
using CAFloating = DerivationOutputCAFloating;
using Deferred = DerivationOutputDeferred; /**
using Impure = DerivationOutputImpure; * Fixed-output derivations, whose output paths are content
* addressed according to that fixed output.
*/
struct CAFixed
{
/**
* Method and hash used for expected hash computation.
*
* References are not allowed by fiat.
*/
ContentAddress ca;
/**
* Return the \ref StorePath "store path" corresponding to this output
*
* @param drvName The name of the derivation this is an output of, without the `.drv`.
* @param outputName The name of this output.
*/
StorePath path(const Store & store, std::string_view drvName, std::string_view outputName) const;
GENERATE_CMP(CAFixed, me->ca);
};
/**
* Floating-output derivations, whose output paths are content
* addressed, but not fixed, and so are dynamically calculated from
* whatever the output ends up being.
* */
struct CAFloating
{
/**
* How the file system objects will be serialized for hashing
*/
ContentAddressMethod method;
/**
* How the serialization will be hashed
*/
HashType hashType;
GENERATE_CMP(CAFloating, me->method, me->hashType);
};
/**
* Input-addressed output which depends on a (CA) derivation whose hash
* isn't known yet.
*/
struct Deferred {
GENERATE_CMP(Deferred);
};
/**
* Impure output which is moved to a content-addressed location (like
* CAFloating) but isn't registered as a realization.
*/
struct Impure
{
/**
* How the file system objects will be serialized for hashing
*/
ContentAddressMethod method;
/**
* How the serialization will be hashed
*/
HashType hashType;
GENERATE_CMP(Impure, me->method, me->hashType);
};
typedef std::variant<
InputAddressed,
CAFixed,
CAFloating,
Deferred,
Impure
> Raw;
Raw raw;
GENERATE_CMP(DerivationOutput, me->raw);
MAKE_WRAPPER_CONSTRUCTOR(DerivationOutput);
/**
* Force choosing a variant
*/
DerivationOutput() = delete;
/** /**
* \note when you use this function you should make sure that you're * \note when you use this function you should make sure that you're
@ -131,10 +134,6 @@ struct DerivationOutput : _DerivationOutputRaw
*/ */
std::optional<StorePath> path(const Store & store, std::string_view drvName, std::string_view outputName) const; std::optional<StorePath> path(const Store & store, std::string_view drvName, std::string_view outputName) const;
inline const Raw & raw() const {
return static_cast<const Raw &>(*this);
}
nlohmann::json toJSON( nlohmann::json toJSON(
const Store & store, const Store & store,
std::string_view drvName, std::string_view drvName,
@ -167,61 +166,71 @@ typedef std::map<std::string, std::pair<DerivationOutput, std::optional<StorePat
*/ */
typedef std::map<StorePath, StringSet> DerivationInputs; typedef std::map<StorePath, StringSet> DerivationInputs;
/** struct DerivationType {
* Input-addressed derivation types
*/
struct DerivationType_InputAddressed {
/** /**
* True iff the derivation type can't be determined statically, * Input-addressed derivation types
* for instance because it (transitively) depends on a content-addressed
* derivation.
*/
bool deferred;
};
/**
* Content-addressed derivation types
*/
struct DerivationType_ContentAddressed {
/**
* Whether the derivation should be built safely inside a sandbox.
*/ */
bool sandboxed; struct InputAddressed {
/**
* True iff the derivation type can't be determined statically,
* for instance because it (transitively) depends on a content-addressed
* derivation.
*/
bool deferred;
GENERATE_CMP(InputAddressed, me->deferred);
};
/** /**
* Whether the derivation's outputs' content-addresses are "fixed" * Content-addressed derivation types
* or "floating.
*
* - Fixed: content-addresses are written down as part of the
* derivation itself. If the outputs don't end up matching the
* build fails.
*
* - Floating: content-addresses are not written down, we do not
* know them until we perform the build.
*/ */
bool fixed; struct ContentAddressed {
}; /**
* Whether the derivation should be built safely inside a sandbox.
*/
bool sandboxed;
/**
* Whether the derivation's outputs' content-addresses are "fixed"
* or "floating".
*
* - Fixed: content-addresses are written down as part of the
* derivation itself. If the outputs don't end up matching the
* build fails.
*
* - Floating: content-addresses are not written down, we do not
* know them until we perform the build.
*/
bool fixed;
/** GENERATE_CMP(ContentAddressed, me->sandboxed, me->fixed);
* Impure derivation type };
*
* This is similar at buil-time to the content addressed, not standboxed, not fixed
* type, but has some restrictions on its usage.
*/
struct DerivationType_Impure {
};
typedef std::variant< /**
DerivationType_InputAddressed, * Impure derivation type
DerivationType_ContentAddressed, *
DerivationType_Impure * This is similar at buil-time to the content addressed, not standboxed, not fixed
> _DerivationTypeRaw; * type, but has some restrictions on its usage.
*/
struct Impure {
GENERATE_CMP(Impure);
};
struct DerivationType : _DerivationTypeRaw { typedef std::variant<
using Raw = _DerivationTypeRaw; InputAddressed,
using Raw::Raw; ContentAddressed,
using InputAddressed = DerivationType_InputAddressed; Impure
using ContentAddressed = DerivationType_ContentAddressed; > Raw;
using Impure = DerivationType_Impure;
Raw raw;
GENERATE_CMP(DerivationType, me->raw);
MAKE_WRAPPER_CONSTRUCTOR(DerivationType);
/**
* Force choosing a variant
*/
DerivationType() = delete;
/** /**
* Do the outputs of the derivation have paths calculated from their * Do the outputs of the derivation have paths calculated from their
@ -257,10 +266,6 @@ struct DerivationType : _DerivationTypeRaw {
* closure, or if fixed output. * closure, or if fixed output.
*/ */
bool hasKnownOutputPaths() const; bool hasKnownOutputPaths() const;
inline const Raw & raw() const {
return static_cast<const Raw &>(*this);
}
}; };
struct BasicDerivation struct BasicDerivation

10
src/libstore/downstream-placeholder.cc
View file

@ -39,16 +39,18 @@ DownstreamPlaceholder DownstreamPlaceholder::unknownDerivation(
} }
DownstreamPlaceholder DownstreamPlaceholder::fromSingleDerivedPathBuilt( DownstreamPlaceholder DownstreamPlaceholder::fromSingleDerivedPathBuilt(
const SingleDerivedPath::Built & b) const SingleDerivedPath::Built & b,
const ExperimentalFeatureSettings & xpSettings)
{ {
return std::visit(overloaded { return std::visit(overloaded {
[&](const SingleDerivedPath::Opaque & o) { [&](const SingleDerivedPath::Opaque & o) {
return DownstreamPlaceholder::unknownCaOutput(o.path, b.output); return DownstreamPlaceholder::unknownCaOutput(o.path, b.output, xpSettings);
}, },
[&](const SingleDerivedPath::Built & b2) { [&](const SingleDerivedPath::Built & b2) {
return DownstreamPlaceholder::unknownDerivation( return DownstreamPlaceholder::unknownDerivation(
DownstreamPlaceholder::fromSingleDerivedPathBuilt(b2), DownstreamPlaceholder::fromSingleDerivedPathBuilt(b2, xpSettings),
b.output); b.output,
xpSettings);
}, },
}, b.drvPath->raw()); }, b.drvPath->raw());
} }

3
src/libstore/downstream-placeholder.hh
View file

@ -84,7 +84,8 @@ public:
* `SingleDerivedPath::Built.drvPath` chain. * `SingleDerivedPath::Built.drvPath` chain.
*/ */
static DownstreamPlaceholder fromSingleDerivedPathBuilt( static DownstreamPlaceholder fromSingleDerivedPathBuilt(
const SingleDerivedPath::Built & built); const SingleDerivedPath::Built & built,
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
}; };
} }

2
src/libstore/filetransfer.cc
View file

@ -863,6 +863,8 @@ void FileTransfer::download(FileTransferRequest && request, Sink & sink)
} }
chunk = std::move(state->data); chunk = std::move(state->data);
/* Reset state->data after the move, since we check data.empty() */
state->data = "";
state->request.notify_one(); state->request.notify_one();
} }

8
src/libstore/misc.cc
View file

@ -88,7 +88,7 @@ const ContentAddress * getDerivationCA(const BasicDerivation & drv)
auto out = drv.outputs.find("out"); auto out = drv.outputs.find("out");
if (out == drv.outputs.end()) if (out == drv.outputs.end())
return nullptr; return nullptr;
if (auto dof = std::get_if<DerivationOutput::CAFixed>(&out->second)) { if (auto dof = std::get_if<DerivationOutput::CAFixed>(&out->second.raw)) {
return &dof->ca; return &dof->ca;
} }
return nullptr; return nullptr;
@ -370,7 +370,7 @@ OutputPathMap resolveDerivedPath(Store & store, const DerivedPath::Built & bfd,
} }
return outputsOpt; return outputsOpt;
}, },
}, bfd.outputs.raw()); }, bfd.outputs.raw);
OutputPathMap outputs; OutputPathMap outputs;
for (auto & [outputName, outputPathOpt] : outputsOpt) { for (auto & [outputName, outputPathOpt] : outputsOpt) {
@ -418,7 +418,7 @@ OutputPathMap resolveDerivedPath(Store & store, const DerivedPath::Built & bfd)
[&](const OutputsSpec::Names & names) { [&](const OutputsSpec::Names & names) {
return static_cast<StringSet>(names); return static_cast<StringSet>(names);
}, },
}, bfd.outputs.raw()); }, bfd.outputs.raw);
for (auto iter = outputMap.begin(); iter != outputMap.end();) { for (auto iter = outputMap.begin(); iter != outputMap.end();) {
auto & outputName = iter->first; auto & outputName = iter->first;
if (bfd.outputs.contains(outputName)) { if (bfd.outputs.contains(outputName)) {
@ -431,7 +431,7 @@ OutputPathMap resolveDerivedPath(Store & store, const DerivedPath::Built & bfd)
if (!outputsLeft.empty()) if (!outputsLeft.empty())
throw Error("derivation '%s' does not have an outputs %s", throw Error("derivation '%s' does not have an outputs %s",
store.printStorePath(drvPath), store.printStorePath(drvPath),
concatStringsSep(", ", quoteStrings(std::get<OutputsSpec::Names>(bfd.outputs)))); concatStringsSep(", ", quoteStrings(std::get<OutputsSpec::Names>(bfd.outputs.raw))));
return outputMap; return outputMap;
} }

20
src/libstore/outputs-spec.cc
View file

@ -17,7 +17,7 @@ bool OutputsSpec::contains(const std::string & outputName) const
[&](const OutputsSpec::Names & outputNames) { [&](const OutputsSpec::Names & outputNames) {
return outputNames.count(outputName) > 0; return outputNames.count(outputName) > 0;
}, },
}, raw()); }, raw);
} }
static std::string outputSpecRegexStr = static std::string outputSpecRegexStr =
@ -49,7 +49,7 @@ OutputsSpec OutputsSpec::parse(std::string_view s)
std::optional spec = parseOpt(s); std::optional spec = parseOpt(s);
if (!spec) if (!spec)
throw Error("invalid outputs specifier '%s'", s); throw Error("invalid outputs specifier '%s'", s);
return *spec; return std::move(*spec);
} }
@ -85,7 +85,7 @@ std::string OutputsSpec::to_string() const
[&](const OutputsSpec::Names & outputNames) -> std::string { [&](const OutputsSpec::Names & outputNames) -> std::string {
return concatStringsSep(",", outputNames); return concatStringsSep(",", outputNames);
}, },
}, raw()); }, raw);
} }
@ -98,7 +98,7 @@ std::string ExtendedOutputsSpec::to_string() const
[&](const ExtendedOutputsSpec::Explicit & outputSpec) -> std::string { [&](const ExtendedOutputsSpec::Explicit & outputSpec) -> std::string {
return "^" + outputSpec.to_string(); return "^" + outputSpec.to_string();
}, },
}, raw()); }, raw);
} }
@ -118,9 +118,9 @@ OutputsSpec OutputsSpec::union_(const OutputsSpec & that) const
ret.insert(thoseNames.begin(), thoseNames.end()); ret.insert(thoseNames.begin(), thoseNames.end());
return ret; return ret;
}, },
}, that.raw()); }, that.raw);
}, },
}, raw()); }, raw);
} }
@ -142,9 +142,9 @@ bool OutputsSpec::isSubsetOf(const OutputsSpec & that) const
ret = false; ret = false;
return ret; return ret;
}, },
}, raw()); }, raw);
}, },
}, that.raw()); }, that.raw);
} }
} }
@ -169,7 +169,7 @@ void adl_serializer<OutputsSpec>::to_json(json & json, OutputsSpec t) {
[&](const OutputsSpec::Names & names) { [&](const OutputsSpec::Names & names) {
json = names; json = names;
}, },
}, t.raw()); }, t.raw);
} }
@ -189,7 +189,7 @@ void adl_serializer<ExtendedOutputsSpec>::to_json(json & json, ExtendedOutputsSp
[&](const ExtendedOutputsSpec::Explicit & e) { [&](const ExtendedOutputsSpec::Explicit & e) {
adl_serializer<OutputsSpec>::to_json(json, e); adl_serializer<OutputsSpec>::to_json(json, e);
}, },
}, t.raw()); }, t.raw);
} }
} }

95
src/libstore/outputs-spec.hh
View file

@ -6,62 +6,57 @@
#include <set> #include <set>
#include <variant> #include <variant>
#include "comparator.hh"
#include "json-impls.hh" #include "json-impls.hh"
#include "comparator.hh"
#include "variant-wrapper.hh"
namespace nix { namespace nix {
/** struct OutputsSpec {
* A non-empty set of outputs, specified by name /**
*/ * A non-empty set of outputs, specified by name
struct OutputNames : std::set<std::string> { */
using std::set<std::string>::set; struct Names : std::set<std::string> {
using std::set<std::string>::set;
/* These need to be "inherited manually" */ /* These need to be "inherited manually" */
OutputNames(const std::set<std::string> & s) Names(const std::set<std::string> & s)
: std::set<std::string>(s) : std::set<std::string>(s)
{ assert(!empty()); } { assert(!empty()); }
/**
* Needs to be "inherited manually"
*/
Names(std::set<std::string> && s)
: std::set<std::string>(s)
{ assert(!empty()); }
/* This set should always be non-empty, so we delete this
constructor in order make creating empty ones by mistake harder.
*/
Names() = delete;
};
/** /**
* Needs to be "inherited manually" * The set of all outputs, without needing to name them explicitly
*/ */
OutputNames(std::set<std::string> && s) struct All : std::monostate { };
: std::set<std::string>(s)
{ assert(!empty()); }
/* This set should always be non-empty, so we delete this typedef std::variant<All, Names> Raw;
constructor in order make creating empty ones by mistake harder.
*/
OutputNames() = delete;
};
/** Raw raw;
* The set of all outputs, without needing to name them explicitly
*/
struct AllOutputs : std::monostate { };
typedef std::variant<AllOutputs, OutputNames> _OutputsSpecRaw; GENERATE_CMP(OutputsSpec, me->raw);
struct OutputsSpec : _OutputsSpecRaw { MAKE_WRAPPER_CONSTRUCTOR(OutputsSpec);
using Raw = _OutputsSpecRaw;
using Raw::Raw;
/** /**
* Force choosing a variant * Force choosing a variant
*/ */
OutputsSpec() = delete; OutputsSpec() = delete;
using Names = OutputNames;
using All = AllOutputs;
inline const Raw & raw() const {
return static_cast<const Raw &>(*this);
}
inline Raw & raw() {
return static_cast<Raw &>(*this);
}
bool contains(const std::string & output) const; bool contains(const std::string & output) const;
/** /**
@ -84,20 +79,22 @@ struct OutputsSpec : _OutputsSpecRaw {
std::string to_string() const; std::string to_string() const;
}; };
struct DefaultOutputs : std::monostate { }; struct ExtendedOutputsSpec {
struct Default : std::monostate { };
typedef std::variant<DefaultOutputs, OutputsSpec> _ExtendedOutputsSpecRaw;
struct ExtendedOutputsSpec : _ExtendedOutputsSpecRaw {
using Raw = _ExtendedOutputsSpecRaw;
using Raw::Raw;
using Default = DefaultOutputs;
using Explicit = OutputsSpec; using Explicit = OutputsSpec;
inline const Raw & raw() const { typedef std::variant<Default, Explicit> Raw;
return static_cast<const Raw &>(*this);
} Raw raw;
GENERATE_CMP(ExtendedOutputsSpec, me->raw);
MAKE_WRAPPER_CONSTRUCTOR(ExtendedOutputsSpec);
/**
* Force choosing a variant
*/
ExtendedOutputsSpec() = delete;
/** /**
* Parse a string of the form 'prefix^output1,...outputN' or * Parse a string of the form 'prefix^output1,...outputN' or

2
src/libstore/path-with-outputs.cc
View file

@ -63,7 +63,7 @@ StorePathWithOutputs::ParseResult StorePathWithOutputs::tryFromDerivedPath(const
[&](const OutputsSpec::Names & outputs) { [&](const OutputsSpec::Names & outputs) {
return static_cast<StringSet>(outputs); return static_cast<StringSet>(outputs);
}, },
}, bfd.outputs.raw()), }, bfd.outputs.raw),
}; };
}, },
[&](const SingleDerivedPath::Built &) -> StorePathWithOutputs::ParseResult { [&](const SingleDerivedPath::Built &) -> StorePathWithOutputs::ParseResult {

2
src/libutil/args.cc
View file

@ -410,8 +410,8 @@ nlohmann::json MultiCommand::toJSON()
auto cat = nlohmann::json::object(); auto cat = nlohmann::json::object();
cat["id"] = command->category(); cat["id"] = command->category();
cat["description"] = trim(categories[command->category()]); cat["description"] = trim(categories[command->category()]);
j["category"] = std::move(cat);
cat["experimental-feature"] = command->experimentalFeature(); cat["experimental-feature"] = command->experimentalFeature();
j["category"] = std::move(cat);
cmds[name] = std::move(j); cmds[name] = std::move(j);
} }

2
src/libutil/logging.cc
View file

@ -67,7 +67,7 @@ public:
case lvlWarn: c = '4'; break; case lvlWarn: c = '4'; break;
case lvlNotice: case lvlInfo: c = '5'; break; case lvlNotice: case lvlInfo: c = '5'; break;
case lvlTalkative: case lvlChatty: c = '6'; break; case lvlTalkative: case lvlChatty: c = '6'; break;
case lvlDebug: case lvlVomit: c = '7'; case lvlDebug: case lvlVomit: c = '7'; break;
default: c = '7'; break; // should not happen, and missing enum case is reported by -Werror=switch-enum default: c = '7'; break; // should not happen, and missing enum case is reported by -Werror=switch-enum
} }
prefix = std::string("<") + c + ">"; prefix = std::string("<") + c + ">";

2
src/libutil/url.cc
View file

@ -44,7 +44,7 @@ ParsedURL parseURL(const std::string & url)
.base = base, .base = base,
.scheme = scheme, .scheme = scheme,
.authority = authority, .authority = authority,
.path = path, .path = percentDecode(path),
.query = decodeQuery(query), .query = decodeQuery(query),
.fragment = percentDecode(std::string(fragment)) .fragment = percentDecode(std::string(fragment))
}; };

30
src/libutil/variant-wrapper.hh Normal file
View file

@ -0,0 +1,30 @@
#pragma once
///@file
// not used, but will be used by callers
#include <variant>
/**
* Force the default versions of all constructors (copy, move, copy
* assignment).
*/
#define FORCE_DEFAULT_CONSTRUCTORS(CLASS_NAME) \
CLASS_NAME(const CLASS_NAME &) = default; \
CLASS_NAME(CLASS_NAME &) = default; \
CLASS_NAME(CLASS_NAME &&) = default; \
\
CLASS_NAME & operator =(const CLASS_NAME &) = default; \
CLASS_NAME & operator =(CLASS_NAME &) = default;
/**
* Make a wrapper constructor. All args are forwarded to the
* construction of the "raw" field. (Which we assume is the only one.)
*
* The moral equivalent of `using Raw::Raw;`
*/
#define MAKE_WRAPPER_CONSTRUCTOR(CLASS_NAME) \
FORCE_DEFAULT_CONSTRUCTORS(CLASS_NAME) \
\
CLASS_NAME(auto &&... arg) \
: raw(std::forward<decltype(arg)>(arg)...) \
{ }

2
src/nix/app.cc
View file

@ -81,7 +81,7 @@ UnresolvedApp InstallableValue::toApp(EvalState & state)
.path = o.path, .path = o.path,
}; };
}, },
}, c.raw())); }, c.raw));
} }
return UnresolvedApp{App { return UnresolvedApp{App {

2
src/nix/bundle.cc
View file

@ -80,7 +80,7 @@ struct CmdBundle : InstallableValueCommand
auto [bundlerFlakeRef, bundlerName, extendedOutputsSpec] = parseFlakeRefWithFragmentAndExtendedOutputsSpec(bundler, absPath(".")); auto [bundlerFlakeRef, bundlerName, extendedOutputsSpec] = parseFlakeRefWithFragmentAndExtendedOutputsSpec(bundler, absPath("."));
const flake::LockFlags lockFlags{ .writeLockFile = false }; const flake::LockFlags lockFlags{ .writeLockFile = false };
InstallableFlake bundler{this, InstallableFlake bundler{this,
evalState, std::move(bundlerFlakeRef), bundlerName, extendedOutputsSpec, evalState, std::move(bundlerFlakeRef), bundlerName, std::move(extendedOutputsSpec),
{"bundlers." + settings.thisSystem.get() + ".default", {"bundlers." + settings.thisSystem.get() + ".default",
"defaultBundler." + settings.thisSystem.get() "defaultBundler." + settings.thisSystem.get()
}, },

2
src/nix/develop.cc
View file

@ -547,7 +547,7 @@ struct CmdDevelop : Common, MixEnvironment
state, state,
std::move(nixpkgs), std::move(nixpkgs),
"bashInteractive", "bashInteractive",
DefaultOutputs(), ExtendedOutputsSpec::Default(),
Strings{}, Strings{},
Strings{"legacyPackages." + settings.thisSystem.get() + "."}, Strings{"legacyPackages." + settings.thisSystem.get() + "."},
nixpkgsLockFlags); nixpkgsLockFlags);

2
src/nix/flake.cc
View file

@ -778,7 +778,7 @@ struct CmdFlakeInitCommon : virtual Args, EvalCommand
auto [templateFlakeRef, templateName] = parseFlakeRefWithFragment(templateUrl, absPath(".")); auto [templateFlakeRef, templateName] = parseFlakeRefWithFragment(templateUrl, absPath("."));
auto installable = InstallableFlake(nullptr, auto installable = InstallableFlake(nullptr,
evalState, std::move(templateFlakeRef), templateName, DefaultOutputs(), evalState, std::move(templateFlakeRef), templateName, ExtendedOutputsSpec::Default(),
defaultTemplateAttrPaths, defaultTemplateAttrPaths,
defaultTemplateAttrPathsPrefixes, defaultTemplateAttrPathsPrefixes,
lockFlags); lockFlags);

1
src/nix/main.cc
View file

@ -359,6 +359,7 @@ void mainWrapped(int argc, char * * argv)
experimentalFeatureSettings.experimentalFeatures = { experimentalFeatureSettings.experimentalFeatures = {
Xp::Flakes, Xp::Flakes,
Xp::FetchClosure, Xp::FetchClosure,
Xp::DynamicDerivations,
}; };
evalSettings.pureEval = false; evalSettings.pureEval = false;
EvalState state({}, openStore("dummy://")); EvalState state({}, openStore("dummy://"));

9
tests/dyn-drv/dep-built-drv.sh Normal file
View file

@ -0,0 +1,9 @@
#!/usr/bin/env bash
source common.sh
out1=$(nix-build ./text-hashed-output.nix -A hello --no-out-link)
clearStore
expectStderr 1 nix-build ./text-hashed-output.nix -A wrapper --no-out-link | grepQuiet "Dependencies on the outputs of dynamic derivations are not yet supported"

80
tests/dyn-drv/eval-outputOf.sh Normal file
View file

@ -0,0 +1,80 @@
#!/usr/bin/env bash
source ./common.sh
# Without the dynamic-derivations XP feature, we don't have the builtin.
nix --experimental-features 'nix-command' eval --impure --expr \
'assert ! (builtins ? outputOf); ""'
# Test that a string is required.
#
# We currently require a string to be passed, rather than a derivation
# object that could be coerced to a string. We might liberalise this in
# the future so it does work, but there are some design questions to
# resolve first. Adding a test so we don't liberalise it by accident.
expectStderr 1 nix --experimental-features 'nix-command dynamic-derivations' eval --impure --expr \
'builtins.outputOf (import ../dependencies.nix) "out"' \
| grepQuiet "value is a set while a string was expected"
# Test that "DrvDeep" string contexts are not supported at this time
#
# Like the above, this is a restriction we could relax later.
expectStderr 1 nix --experimental-features 'nix-command dynamic-derivations' eval --impure --expr \
'builtins.outputOf (import ../dependencies.nix).drvPath "out"' \
| grepQuiet "has a context which refers to a complete source and binary closure. This is not supported at this time"
# Test using `builtins.outputOf` with static derivations
testStaticHello () {
nix eval --impure --expr \
'with (import ./text-hashed-output.nix); let
a = hello.outPath;
b = builtins.outputOf (builtins.unsafeDiscardOutputDependency hello.drvPath) "out";
in builtins.trace a
(builtins.trace b
(assert a == b; null))'
}
# Test with a regular old input-addresed derivation
#
# `builtins.outputOf` works without ca-derivations and doesn't create a
# placeholder but just returns the output path.
testStaticHello
# Test with content addressed derivation.
NIX_TESTS_CA_BY_DEFAULT=1 testStaticHello
# Test with derivation-producing derivation
#
# This is hardly different from the preceding cases, except that we're
# only taking 1 outputOf out of 2 possible outputOfs. Note that
# `.outPath` could be defined as `outputOf drvPath`, which is what we're
# testing here. The other `outputOf` that we're not testing here is the
# use of _dynamic_ derivations.
nix eval --impure --expr \
'with (import ./text-hashed-output.nix); let
a = producingDrv.outPath;
b = builtins.outputOf (builtins.builtins.unsafeDiscardOutputDependency producingDrv.drvPath) "out";
in builtins.trace a
(builtins.trace b
(assert a == b; null))'
# Test with unbuilt output of derivation-producing derivation.
#
# This function similar to `testStaticHello` used above, but instead of
# checking the property on a constant derivation, we check it on a
# derivation that's from another derivation's output (outPath).
testDynamicHello () {
nix eval --impure --expr \
'with (import ./text-hashed-output.nix); let
a = builtins.outputOf producingDrv.outPath "out";
b = builtins.outputOf (builtins.outputOf (builtins.unsafeDiscardOutputDependency producingDrv.drvPath) "out") "out";
in builtins.trace a
(builtins.trace b
(assert a == b; null))'
}
# inner dynamic derivation is input-addressed
testDynamicHello
# inner dynamic derivation is content-addressed
NIX_TESTS_CA_BY_DEFAULT=1 testDynamicHello

4
tests/dyn-drv/local.mk
View file

@ -1,7 +1,9 @@
dyn-drv-tests := \ dyn-drv-tests := \
$(d)/text-hashed-output.sh \ $(d)/text-hashed-output.sh \
$(d)/recursive-mod-json.sh \ $(d)/recursive-mod-json.sh \
$(d)/build-built-drv.sh $(d)/build-built-drv.sh \
$(d)/eval-outputOf.sh \
$(d)/dep-built-drv.sh
install-tests-groups += dyn-drv install-tests-groups += dyn-drv

2
tests/dyn-drv/recursive-mod-json.sh
View file

@ -3,6 +3,8 @@ source common.sh
# FIXME # FIXME
if [[ $(uname) != Linux ]]; then skipTest "Not running Linux"; fi if [[ $(uname) != Linux ]]; then skipTest "Not running Linux"; fi
export NIX_TESTS_CA_BY_DEFAULT=1
enableFeatures 'recursive-nix' enableFeatures 'recursive-nix'
restartDaemon restartDaemon

10
tests/dyn-drv/text-hashed-output.nix
View file

@ -12,9 +12,6 @@ rec {
mkdir -p $out mkdir -p $out
echo "Hello World" > $out/hello echo "Hello World" > $out/hello
''; '';
__contentAddressed = true;
outputHashMode = "recursive";
outputHashAlgo = "sha256";
}; };
producingDrv = mkDerivation { producingDrv = mkDerivation {
name = "hello.drv"; name = "hello.drv";
@ -26,4 +23,11 @@ rec {
outputHashMode = "text"; outputHashMode = "text";
outputHashAlgo = "sha256"; outputHashAlgo = "sha256";
}; };
wrapper = mkDerivation {
name = "use-dynamic-drv-in-non-dynamic-drv";
buildCommand = ''
echo "Copying the output of the dynamic derivation"
cp -r ${builtins.outputOf producingDrv.outPath "out"} $out
'';
};
} }

58
tests/repl.sh
View file

@ -56,11 +56,17 @@ testRepl
# Same thing (kind-of), but with a remote store. # Same thing (kind-of), but with a remote store.
testRepl --store "$TEST_ROOT/store?real=$NIX_STORE_DIR" testRepl --store "$TEST_ROOT/store?real=$NIX_STORE_DIR"
testReplResponse () { # Remove ANSI escape sequences. They can prevent grep from finding a match.
stripColors () {
sed -E 's/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g'
}
testReplResponseGeneral () {
local grepMode="$1"; shift
local commands="$1"; shift local commands="$1"; shift
local expectedResponse="$1"; shift local expectedResponse="$1"; shift
local response="$(nix repl "$@" <<< "$commands")" local response="$(nix repl "$@" <<< "$commands" | stripColors)"
echo "$response" | grepQuiet -s "$expectedResponse" \ echo "$response" | grepQuiet "$grepMode" -s "$expectedResponse" \
|| fail "repl command set: || fail "repl command set:
$commands $commands
@ -71,7 +77,16 @@ $expectedResponse
but with: but with:
$response" $response
"
}
testReplResponse () {
testReplResponseGeneral --basic-regexp "$@"
}
testReplResponseNoRegex () {
testReplResponseGeneral --fixed-strings "$@"
} }
# :a uses the newest version of a symbol # :a uses the newest version of a symbol
@ -85,9 +100,9 @@ testReplResponse '
# note the escaped \, # note the escaped \,
# \\ # \\
# because the second argument is a regex # because the second argument is a regex
testReplResponse ' testReplResponseNoRegex '
"$" + "{hi}" "$" + "{hi}"
' '"\\${hi}"' ' '"\${hi}"'
testReplResponse ' testReplResponse '
drvPath drvPath
@ -133,3 +148,34 @@ echo "changingThing"
) | nix repl ./flake --experimental-features 'flakes repl-flake') ) | nix repl ./flake --experimental-features 'flakes repl-flake')
echo "$replResult" | grepQuiet -s beforeChange echo "$replResult" | grepQuiet -s beforeChange
echo "$replResult" | grepQuiet -s afterChange echo "$replResult" | grepQuiet -s afterChange
# Test recursive printing and formatting
# Normal output should print attributes in lexicographical order non-recursively
testReplResponseNoRegex '
{ a = { b = 2; }; l = [ 1 2 3 ]; s = "string"; n = 1234; x = rec { y = { z = { inherit y; }; }; }; }
' '{ a = { ... }; l = [ ... ]; n = 1234; s = "string"; x = { ... }; }'
# Same for lists, but order is preserved
testReplResponseNoRegex '
[ 42 1 "thingy" ({ a = 1; }) ([ 1 2 3 ]) ]
' '[ 42 1 "thingy" { ... } [ ... ] ]'
# Same for let expressions
testReplResponseNoRegex '
let x = { y = { a = 1; }; inherit x; }; in x
' '{ x = { ... }; y = { ... }; }'
# The :p command should recursively print sets, but prevent infinite recursion
testReplResponseNoRegex '
:p { a = { b = 2; }; s = "string"; n = 1234; x = rec { y = { z = { inherit y; }; }; }; }
' '{ a = { b = 2; }; n = 1234; s = "string"; x = { y = { z = { y = «repeated»; }; }; }; }'
# Same for lists
testReplResponseNoRegex '
:p [ 42 1 "thingy" (rec { a = 1; b = { inherit a; inherit b; }; }) ([ 1 2 3 ]) ]
' '[ 42 1 "thingy" { a = 1; b = { a = 1; b = «repeated»; }; } [ 1 2 3 ] ]'
# Same for let expressions
testReplResponseNoRegex '
:p let x = { y = { a = 1; }; inherit x; }; in x
' '{ x = { x = «repeated»; y = { a = 1; }; }; y = «repeated»; }'