From b5605217ae7c3c29dbf887ffdf4e72b176f535b3 Mon Sep 17 00:00:00 2001 From: John Ericson Date: Wed, 8 May 2024 17:14:00 -0400 Subject: [PATCH] Document string context (#8595) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Document string context Now what we have enough primops, we can document how string contexts work. Co-authored-by: Robert Hensing Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com> Co-authored-by: Valentin Gagarin Co-authored-by: Felix Uhl --- doc/manual/src/SUMMARY.md.in | 1 + doc/manual/src/glossary.md | 11 ++ doc/manual/src/language/string-context.md | 134 ++++++++++++++++++++++ src/libexpr/primops/context.cc | 19 ++- 4 files changed, 161 insertions(+), 4 deletions(-) create mode 100644 doc/manual/src/language/string-context.md diff --git a/doc/manual/src/SUMMARY.md.in b/doc/manual/src/SUMMARY.md.in index c64c1daad..fdfd0a927 100644 --- a/doc/manual/src/SUMMARY.md.in +++ b/doc/manual/src/SUMMARY.md.in @@ -27,6 +27,7 @@ - [Language Constructs](language/constructs.md) - [String interpolation](language/string-interpolation.md) - [Lookup path](language/constructs/lookup-path.md) + - [String context](language/string-context.md) - [Operators](language/operators.md) - [Derivations](language/derivations.md) - [Advanced Attributes](language/advanced-attributes.md) diff --git a/doc/manual/src/glossary.md b/doc/manual/src/glossary.md index 88916bb50..cbffda187 100644 --- a/doc/manual/src/glossary.md +++ b/doc/manual/src/glossary.md @@ -218,6 +218,17 @@ - [output closure]{#gloss-output-closure}\ The [closure] of an [output path]. It only contains what is [reachable] from the output. +- [deriving path]{#gloss-deriving-path} + + Deriving paths are a way to refer to [store objects][store object] that ar not yet [realised][realise]. + This is necessary because, in general and particularly for [content-addressed derivations][content-addressed derivation], the [output path] of an [output] is not known in advance. + There are two forms: + + - *constant*: just a [store path] + It can be made [valid][validity] by copying it into the store: from the evaluator, command line interface or another store. + + - *output*: a pair of a [store path] to a [derivation] and an [output] name. + - [deriver]{#gloss-deriver} The [store derivation] that produced an [output path]. diff --git a/doc/manual/src/language/string-context.md b/doc/manual/src/language/string-context.md new file mode 100644 index 000000000..88ae0d8b0 --- /dev/null +++ b/doc/manual/src/language/string-context.md @@ -0,0 +1,134 @@ +# String context + +> **Note** +> +> This is an advanced topic. +> The Nix language is designed to be used without the programmer consciously dealing with string contexts or even knowing what they are. + +A string in the Nix language is not just a sequence of characters like strings in other languages. +It is actually a pair of a sequence of characters and a *string context*. +The string context is an (unordered) set of *string context elements*. + +The purpose of string contexts is to collect non-string values attached to strings via +[string concatenation](./operators.md#string-concatenation), +[string interpolation](./string-interpolation.md), +and similar operations. +The idea is that a user can combine together values to create a build instructions for derivations without manually keeping track of where they come from. +Then the Nix language implicitly does that bookkeeping to efficiently obtain the closure of derivation inputs. + +> **Note** +> +> String contexts are *not* explicitly manipulated in idiomatic Nix language code. + +String context elements come in different forms: + +- [deriving path]{#string-context-element-derived-path} + + A string context element of this type is a [deriving path](@docroot@/glossary.md#gloss-deriving-path). + They can be either of type [constant](#string-context-constant) or [output](#string-context-output), which correspond to the types of deriving paths. + + - [Constant string context elements]{#string-context-constant} + + > **Example** + > + > [`builtins.storePath`] creates a string with a single constant string context element: + > + > ```nix + > builtins.getContext (builtins.storePath "/nix/store/wkhdf9jinag5750mqlax6z2zbwhqb76n-hello-2.10") + > ``` + > evaluates to + > ```nix + > { + > "/nix/store/wkhdf9jinag5750mqlax6z2zbwhqb76n-hello-2.10" = { + > path = true; + > }; + > } + > ``` + + [deriving path]: @docroot@/glossary.md#gloss-deriving-path + [store path]: @docroot@/glossary.md#gloss-store-path + [`builtins.storePath`]: ./builtins.md#builtins-storePath + + - [Output string context elements]{#string-context-output} + + > **Example** + > + > The behavior of string contexts are best demonstrated with a built-in function that is still experimental: [`builtins.outputOf`]. + > This example will *not* work with stable Nix! + > + > ```nix + > builtins.getContext + > (builtins.outputOf + > (builtins.storePath "/nix/store/fvchh9cvcr7kdla6n860hshchsba305w-hello-2.12.drv") + > "out") + > ``` + > evaluates to + > ```nix + > { + > "/nix/store/fvchh9cvcr7kdla6n860hshchsba305w-hello-2.12.drv" = { + > outputs = [ "out" ]; + > }; + > } + > ``` + + [`builtins.outputOf`]: ./builtins.md#builtins-outputOf + +- [*derivation deep*]{#string-context-element-derivation-deep} + + *derivation deep* is an advanced feature intended to be used with the + [`exportReferencesGraph` derivation attribute](./advanced-attributes.html#adv-attr-exportReferencesGraph). + A *derivation deep* string context element is a derivation path, and refers to both its outputs and the entire build closure of that derivation: + all its outputs, all the other derivations the given derivation depends on, and all the outputs of those. + + > **Example** + > + > The best way to illustrate *derivation deep* string contexts is with [`builtins.addDrvOutputDependencies`]. + > Take a regular constant string context element pointing to a derivation, and transform it into a "Derivation deep" string context element. + > + > ```nix + > builtins.getContext + > (builtins.addDrvOutputDependencies + > (builtins.storePath "/nix/store/fvchh9cvcr7kdla6n860hshchsba305w-hello-2.12.drv")) + > ``` + > evaluates to + > ```nix + > { + > "/nix/store/fvchh9cvcr7kdla6n860hshchsba305w-hello-2.12.drv" = { + > allOutputs = true; + > }; + > } + > ``` + + [`builtins.addDrvOutputDependencies`]: ./builtins.md#builtins-addDrvOutputDependencies + [`builtins.unsafeDiscardOutputDependency`]: ./builtins.md#builtins-unsafeDiscardOutputDependency + +## Inspecting string contexts + +Most basically, [`builtins.hasContext`] will tell whether a string has a non-empty context. + +When more granular information is needed, [`builtins.getContext`] can be used. +It creates an [attribute set] representing the string context, which can be inspected as usual. + +[`builtins.hasContext`]: ./builtins.md#builtins-hasContext +[`builtins.getContext`]: ./builtins.md#builtins-getContext +[attribute set]: ./values.md#attribute-set + +## Clearing string contexts + +[`buitins.unsafeDiscardStringContext`](./builtins.md#builtins-unsafeDiscardStringContext) will make a copy of a string, but with an empty string context. +The returned string can be used in more ways, e.g. by operators that require the string context to be empty. +The requirement to explicitly discard the string context in such use cases helps ensure that string context elements are not lost by mistake. +The "unsafe" marker is only there to remind that Nix normally guarantees that dependencies are tracked, whereas the returned string has lost them. + +## Constructing string contexts + +[`builtins.appendContext`] will create a copy of a string, but with additional string context elements. +The context is specified explicitly by an [attribute set] in the format that [`builtins.hasContext`] produces. +A string with arbitrary contexts can be made like this: + +1. Create a string with the desired string context elements. + (The contents of the string do not matter.) +2. Dump its context with [`builtins.getContext`]. +3. Combine it with a base string and repeated [`builtins.appendContext`] calls. + +[`builtins.appendContext`]: ./builtins.md#builtins-appendContext diff --git a/src/libexpr/primops/context.cc b/src/libexpr/primops/context.cc index 2d3013132..8c3f1b4e8 100644 --- a/src/libexpr/primops/context.cc +++ b/src/libexpr/primops/context.cc @@ -14,8 +14,11 @@ static void prim_unsafeDiscardStringContext(EvalState & state, const PosIdx pos, static RegisterPrimOp primop_unsafeDiscardStringContext({ .name = "__unsafeDiscardStringContext", - .arity = 1, - .fun = prim_unsafeDiscardStringContext + .args = {"s"}, + .doc = R"( + Discard the [string context](@docroot@/language/string-context.md) from a value that can be coerced to a string. + )", + .fun = prim_unsafeDiscardStringContext, }); @@ -75,7 +78,11 @@ static RegisterPrimOp primop_unsafeDiscardOutputDependency({ .name = "__unsafeDiscardOutputDependency", .args = {"s"}, .doc = R"( - Create a copy of the given string where every "derivation deep" string context element is turned into a constant string context element. + Create a copy of the given string where every + [derivation deep](@docroot@/language/string-context.md#string-context-element-derivation-deep) + string context element is turned into a + [constant](@docroot@/language/string-context.md#string-context-element-constant) + string context element. This is the opposite of [`builtins.addDrvOutputDependencies`](#builtins-addDrvOutputDependencies). @@ -137,7 +144,11 @@ static RegisterPrimOp primop_addDrvOutputDependencies({ .name = "__addDrvOutputDependencies", .args = {"s"}, .doc = R"( - Create a copy of the given string where a single constant string context element is turned into a "derivation deep" string context element. + Create a copy of the given string where a single + [constant](@docroot@/language/string-context.md#string-context-element-constant) + string context element is turned into a + [derivation deep](@docroot@/language/string-context.md#string-context-element-derivation-deep) + string context element. The store path that is the constant string context element should point to a valid derivation, and end in `.drv`.