From edf3ecc497d9931f84d8a28679b51773c761fdd8 Mon Sep 17 00:00:00 2001 From: John Ericson Date: Sun, 22 Oct 2023 20:01:01 -0400 Subject: [PATCH] Document JSON formats Good to document these formats separately from commands that happen to use them. Eventually I would like this and `builtins.derivation` to refer to a store section on derivations that is authoritative, but that doesn't yet exist, and will take some time to make. So I think we're just best off merging this now as is. Co-authored-by: Valentin Gagarin --- doc/manual/src/SUMMARY.md.in | 3 + doc/manual/src/glossary.md | 2 +- doc/manual/src/json/derivation.md | 71 +++++++++++++++++ doc/manual/src/json/store-object-info.md | 97 ++++++++++++++++++++++++ src/libstore/globals.hh | 2 +- src/nix/derivation-add.md | 7 +- src/nix/derivation-show.md | 60 +-------------- 7 files changed, 181 insertions(+), 61 deletions(-) create mode 100644 doc/manual/src/json/derivation.md create mode 100644 doc/manual/src/json/store-object-info.md diff --git a/doc/manual/src/SUMMARY.md.in b/doc/manual/src/SUMMARY.md.in index c67ddc6cb..10fe51fc9 100644 --- a/doc/manual/src/SUMMARY.md.in +++ b/doc/manual/src/SUMMARY.md.in @@ -104,6 +104,9 @@ - [Channels](command-ref/files/channels.md) - [Default Nix expression](command-ref/files/default-nix-expression.md) - [Architecture and Design](architecture/architecture.md) +- [JSON Formats](json/index.md) + - [Store Object Info](json/store-object-info.md) + - [Derivation](json/derivation.md) - [Protocols](protocols/index.md) - [Serving Tarball Flakes](protocols/tarball-fetcher.md) - [Derivation "ATerm" file format](protocols/derivation-aterm.md) diff --git a/doc/manual/src/glossary.md b/doc/manual/src/glossary.md index 3c0570a44..124dc8d2e 100644 --- a/doc/manual/src/glossary.md +++ b/doc/manual/src/glossary.md @@ -127,7 +127,7 @@ non-[fixed-output](#gloss-fixed-output-derivation) derivation. -- [output-addressed store object]{#gloss-output-addressed-store-object} +- [content-addressed store object]{#gloss-content-addressed-store-object} A [store object] whose [store path] is determined by its contents. This includes derivations, the outputs of [content-addressed derivations](#gloss-content-addressed-derivation), and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation). diff --git a/doc/manual/src/json/derivation.md b/doc/manual/src/json/derivation.md new file mode 100644 index 000000000..649d543cc --- /dev/null +++ b/doc/manual/src/json/derivation.md @@ -0,0 +1,71 @@ +# Derivation JSON Format + +> **Warning** +> +> This JSON format is currently +> [**experimental**](@docroot@/contributing/experimental-features.md#xp-feature-nix-command) +> and subject to change. + +The JSON serialization of a +[derivations](@docroot@/glossary.md#gloss-store-derivation) +is a JSON object with the following fields: + +* `name`: + The name of the derivation. + This is used when calculating the store paths of the derivation's outputs. + +* `outputs`: + Information about the output paths of the derivation. + This is a JSON object with one member per output, where the key is the output name and the value is a JSON object with these fields: + + * `path`: The output path. + + * `hashAlgo`: + For fixed-output derivations, the hashing algorithm (e.g. `sha256`), optionally prefixed by `r:` if `hash` denotes a NAR hash rather than a flat file hash. + + * `hash`: + For fixed-output derivations, the expected content hash in base-16. + + > **Example** + > + > ```json + > "outputs": { + > "out": { + > "path": "/nix/store/2543j7c6jn75blc3drf4g5vhb1rhdq29-source", + > "hashAlgo": "r:sha256", + > "hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62" + > } + > } + > ``` + +* `inputSrcs`: + A list of store paths on which this derivation depends. + +* `inputDrvs`: + A JSON object specifying the derivations on which this derivation depends, and what outputs of those derivations. + + > **Example** + > + > ```json + > "inputDrvs": { + > "/nix/store/6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"], + > "/nix/store/fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"] + > } + > ``` + + specifies that this derivation depends on the `dev` output of `curl`, and the `out` output of `unzip`. + +* `system`: + The system type on which this derivation is to be built + (e.g. `x86_64-linux`). + +* `builder`: + The absolute path of the program to be executed to run the build. + Typically this is the `bash` shell + (e.g. `/nix/store/r3j288vpmczbl500w6zz89gyfa4nr0b1-bash-4.4-p23/bin/bash`). + +* `args`: + The command-line arguments passed to the `builder`. + +* `env`: + The environment passed to the `builder`. diff --git a/doc/manual/src/json/store-object-info.md b/doc/manual/src/json/store-object-info.md new file mode 100644 index 000000000..db43c2fa1 --- /dev/null +++ b/doc/manual/src/json/store-object-info.md @@ -0,0 +1,97 @@ +# Store object info JSON format + +> **Warning** +> +> This JSON format is currently +> [**experimental**](@docroot@/contributing/experimental-features.md#xp-feature-nix-command) +> and subject to change. + +Info about a [store object]. + +* `path`: + + [Store path][store path] to the given store object. + +* `narHash`: + + Hash of the [file system object] part of the store object when serialized as a [Nix Archive](#gloss-nar). + +* `narSize`: + + Size of the [file system object] part of the store object when serialized as a [Nix Archive](#gloss-nar). + +* `references`: + + An array of [store paths][store path], possibly including this one. + +* `ca` (optional): + + Content address of this store object's file system object, used to compute its store path. + +[store path]: @docroot@/glossary.md#gloss-store-path +[file system object]: @docroot@/store/file-system-object.md + +## Impure fields + +These are not intrinsic properties of the store object. +In other words, the same store object residing in different store could have different values for these properties. + +* `deriver` (optional): + + The path to the [derivation] from which this store object is produced. + + [derivation]: @docroot@/glossary.md#gloss-store-derivation + +* `registrationTime` (optional): + + When this derivation was added to the store. + +* `ultimate` (optional): + + Whether this store object is trusted because we built it ourselves, rather than substituted a build product from elsewhere. + +* `signatures` (optional): + + Signatures claiming that this store object is what it claims to be. + Not relevant for [content-addressed] store objects, + but useful for [input-addressed] store objects. + + [content-addressed]: @docroot@/glossary.md#gloss-content-addressed-store-object + [input-addressed]: @docroot@/glossary.md#gloss-input-addressed-store-object + +### `.narinfo` extra fields + +This meta data is specific to the "binary cache" family of Nix store types. +This information is not intrinsic to the store object, but about how it is stored. + +* `url`: + + Where to download a compressed archive of the file system objects of this store object. + +* `compression`: + + The compression format that the archive is in. + +* `fileHash`: + + A digest for the compressed archive itself, as opposed to the data contained within. + +* `fileSize`: + + The size of the compressed archive itself. + +## Computed closure fields + +These fields are not stored at all, but computed by traverising the other other fields across all the store objects in a [closure]. + +* `closureSize`: + + The total size of the compressed archive itself for this object, and the compressed archive of every object in this object's [closure]. + +### `.narinfo` extra fields + +* `closureSize`: + + The total size of this store object and every other object in its [closure]. + +[closure]: @docroot@/glossary.md#gloss-closure diff --git a/src/libstore/globals.hh b/src/libstore/globals.hh index 49a4c1f2a..3107c8aed 100644 --- a/src/libstore/globals.hh +++ b/src/libstore/globals.hh @@ -635,7 +635,7 @@ public: - the store object has been signed using a key in the trusted keys list - the [`require-sigs`](#conf-require-sigs) option has been set to `false` - - the store object is [output-addressed](@docroot@/glossary.md#gloss-output-addressed-store-object) + - the store object is [content-addressed](@docroot@/glossary.md#gloss-content-addressed-store-object) )", {"binary-cache-public-keys"}}; diff --git a/src/nix/derivation-add.md b/src/nix/derivation-add.md index f116681ab..d9b8467df 100644 --- a/src/nix/derivation-add.md +++ b/src/nix/derivation-add.md @@ -9,10 +9,11 @@ Store derivations are used internally by Nix. They are store paths with extension `.drv` that represent the build-time dependency graph to which a Nix expression evaluates. -[store derivation]: ../../glossary.md#gloss-store-derivation -The JSON format is documented under the [`derivation show`] command. +[store derivation]: @docroot@/glossary.md#gloss-store-derivation -[`derivation show`]: ./nix3-derivation-show.md +`nix derivation add` takes a single derivation in the following format: + +{{#include ../../json/derivation.md}} )"" diff --git a/src/nix/derivation-show.md b/src/nix/derivation-show.md index 1296e2885..884f1adc6 100644 --- a/src/nix/derivation-show.md +++ b/src/nix/derivation-show.md @@ -5,8 +5,6 @@ R""( * Show the [store derivation] that results from evaluating the Hello package: - [store derivation]: ../../glossary.md#gloss-store-derivation - ```console # nix derivation show nixpkgs#hello { @@ -48,62 +46,12 @@ a Nix expression evaluates. By default, this command only shows top-level derivations, but with `--recursive`, it also shows their dependencies. -The JSON output is a JSON object whose keys are the store paths of the -derivations, and whose values are a JSON object with the following -fields: +[store derivation]: @docroot@/glossary.md#gloss-store-derivation -* `name`: The name of the derivation. This is used when calculating the - store paths of the derivation's outputs. +`nix derivation show` outputs a JSON map of [store path]s to derivations in the following format: -* `outputs`: Information about the output paths of the - derivation. This is a JSON object with one member per output, where - the key is the output name and the value is a JSON object with these - fields: +[store path]: @docroot@/glossary.md#gloss-store-path - * `path`: The output path. - * `hashAlgo`: For fixed-output derivations, the hashing algorithm - (e.g. `sha256`), optionally prefixed by `r:` if `hash` denotes a - NAR hash rather than a flat file hash. - * `hash`: For fixed-output derivations, the expected content hash in - base-16. - - Example: - - ```json - "outputs": { - "out": { - "path": "/nix/store/2543j7c6jn75blc3drf4g5vhb1rhdq29-source", - "hashAlgo": "r:sha256", - "hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62" - } - } - ``` - -* `inputSrcs`: A list of store paths on which this derivation depends. - -* `inputDrvs`: A JSON object specifying the derivations on which this - derivation depends, and what outputs of those derivations. For - example, - - ```json - "inputDrvs": { - "/nix/store/6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"], - "/nix/store/fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"] - } - ``` - - specifies that this derivation depends on the `dev` output of - `curl`, and the `out` output of `unzip`. - -* `system`: The system type on which this derivation is to be built - (e.g. `x86_64-linux`). - -* `builder`: The absolute path of the program to be executed to run - the build. Typically this is the `bash` shell - (e.g. `/nix/store/r3j288vpmczbl500w6zz89gyfa4nr0b1-bash-4.4-p23/bin/bash`). - -* `args`: The command-line arguments passed to the `builder`. - -* `env`: The environment passed to the `builder`. +{{#include ../../json/derivation.md}} )""