mirror of
https://github.com/privatevoid-net/nix-super.git
synced 2024-11-22 05:56:15 +02:00
Document store object content addressing & improve JSON format
The JSON format no longer uses the legacy ATerm `r:` prefixing nonsese, but separate fields. Progress on #9866 Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
This commit is contained in:
parent
ba2911b03b
commit
1c75af969a
21 changed files with 268 additions and 65 deletions
12
doc/manual/rl-next/derivation-json-change.md
Normal file
12
doc/manual/rl-next/derivation-json-change.md
Normal file
|
@ -0,0 +1,12 @@
|
||||||
|
---
|
||||||
|
synopsis: Modify `nix derivation {add,show}` JSON format
|
||||||
|
issues: 9866
|
||||||
|
prs: 10722
|
||||||
|
---
|
||||||
|
|
||||||
|
The JSON format for derivations has been slightly revised to better conform to our [JSON guidelines](@docroot@contributing/cli-guideline#returning-future-proof-json).
|
||||||
|
In particular, the hash algorithm and content addressing method of content-addresed derivation outputs is now separated into two fields `hashAlgo` and `method`,
|
||||||
|
rather than one field with an arcane `:`-separated format.
|
||||||
|
|
||||||
|
This JSON format is only used by the experimental `nix derivation` family of commands, at this time.
|
||||||
|
Future revisions are expected as the JSON format is still not entirely in compliance even after these changes.
|
|
@ -20,6 +20,7 @@
|
||||||
- [File System Object](store/file-system-object.md)
|
- [File System Object](store/file-system-object.md)
|
||||||
- [Content-Addressing File System Objects](store/file-system-object/content-address.md)
|
- [Content-Addressing File System Objects](store/file-system-object/content-address.md)
|
||||||
- [Store Object](store/store-object.md)
|
- [Store Object](store/store-object.md)
|
||||||
|
- [Content-Addressing Store Objects](store/store-object/content-address.md)
|
||||||
- [Store Path](store/store-path.md)
|
- [Store Path](store/store-path.md)
|
||||||
- [Store Types](store/types/index.md)
|
- [Store Types](store/types/index.md)
|
||||||
{{#include ./store/types/SUMMARY.md}}
|
{{#include ./store/types/SUMMARY.md}}
|
||||||
|
|
|
@ -197,37 +197,40 @@ Derivations can declare some infrequently used optional attributes.
|
||||||
`outputHashAlgo` can only be `null` when `outputHash` follows the SRI format.
|
`outputHashAlgo` can only be `null` when `outputHash` follows the SRI format.
|
||||||
|
|
||||||
The `outputHashMode` attribute determines how the hash is computed.
|
The `outputHashMode` attribute determines how the hash is computed.
|
||||||
It must be one of the following two values:
|
It must be one of the following values:
|
||||||
|
|
||||||
<!-- FIXME link to store object content-addressing not file system object content addressing once we have the page for that. -->
|
- [`"flat"`](@docroot@/store/store-object/content-address.md#method-flat)
|
||||||
|
|
||||||
- `"flat"`
|
|
||||||
|
|
||||||
The output must be a non-executable regular file; if it isn’t, the build fails.
|
|
||||||
The hash is
|
|
||||||
[simply computed over the contents of that file](@docroot@/store/file-system-object/content-address.md#serial-flat)
|
|
||||||
(so it’s equal to what Unix commands like `sha256sum` or `sha1sum` produce).
|
|
||||||
|
|
||||||
This is the default.
|
This is the default.
|
||||||
|
|
||||||
- `"recursive"` or `"nar"`
|
- [`"recursive"` or `"nar"`](@docroot@/store/store-object/content-address.md#method-nix-archive)
|
||||||
|
|
||||||
The hash is computed over the
|
> **Compatibility**
|
||||||
[Nix Archive (NAR)](@docroot@/store/file-system-object/content-address.md#serial-nix-archive)
|
>
|
||||||
dump of the output (i.e., the result of [`nix-store --dump`](@docroot@/command-ref/nix-store/dump.md)).
|
> `"recursive"` is the traditional way of indicating this,
|
||||||
In this case, the output is allowed to be any [file system object], including directories and more.
|
> and is supported since 2005 (virtually the entire history of Nix).
|
||||||
|
> `"nar"` is more clear, and consistent with other parts of Nix (such as the CLI),
|
||||||
|
> however support for it is only added in Nix version 2.21.
|
||||||
|
|
||||||
`"recursive"` is the traditional way of indicating this,
|
- [`"text"`](@docroot@/store/store-object/content-address.md#method-text)
|
||||||
and is supported since 2005 (virtually the entire history of Nix).
|
|
||||||
`"nar"` is more clear, and consistent with other parts of Nix (such as the CLI),
|
> **Warning**
|
||||||
however support for it is only added in Nix version 2.21.
|
>
|
||||||
|
> The use of this method for derivation outputs is part of the [`dynamic-derivations`][xp-feature-dynamic-derivations] experimental feature.
|
||||||
|
|
||||||
|
- [`"git"`](@docroot@/store/store-object/content-address.md#method-git)
|
||||||
|
|
||||||
|
> **Warning**
|
||||||
|
>
|
||||||
|
> This method is part of the [`git-hashing`][xp-feature-git-hashing] experimental feature.
|
||||||
|
|
||||||
- [`__contentAddressed`]{#adv-attr-__contentAddressed}
|
- [`__contentAddressed`]{#adv-attr-__contentAddressed}
|
||||||
|
|
||||||
> **Warning**
|
> **Warning**
|
||||||
> This attribute is part of an [experimental feature](@docroot@/contributing/experimental-features.md).
|
> This attribute is part of an [experimental feature](@docroot@/contributing/experimental-features.md).
|
||||||
>
|
>
|
||||||
> To use this attribute, you must enable the
|
> To use this attribute, you must enable the
|
||||||
> [`ca-derivations`](@docroot@/contributing/experimental-features.md#xp-feature-ca-derivations) experimental feature.
|
> [`ca-derivations`][xp-feature-ca-derivations] experimental feature.
|
||||||
> For example, in [nix.conf](../command-ref/conf-file.md) you could add:
|
> For example, in [nix.conf](../command-ref/conf-file.md) you could add:
|
||||||
>
|
>
|
||||||
> ```
|
> ```
|
||||||
|
@ -359,3 +362,7 @@ Derivations can declare some infrequently used optional attributes.
|
||||||
```
|
```
|
||||||
|
|
||||||
ensures that the derivation can only be built on a machine with the `kvm` feature.
|
ensures that the derivation can only be built on a machine with the `kvm` feature.
|
||||||
|
|
||||||
|
[xp-feature-ca-derivations]: @docroot@/contributing/experimental-features.md#xp-feature-ca-derivations
|
||||||
|
[xp-feature-dynamic-derivations]: @docroot@/contributing/experimental-features.md#xp-feature-dynamic-derivations
|
||||||
|
[xp-feature-git-hashing]: @docroot@/contributing/experimental-features.md#xp-feature-git-hashing
|
||||||
|
|
|
@ -18,10 +18,30 @@ is a JSON object with the following fields:
|
||||||
Information about the output paths of the derivation.
|
Information about the output paths of the derivation.
|
||||||
This is a JSON object with one member per output, where the key is the output name and the value is a JSON object with these fields:
|
This is a JSON object with one member per output, where the key is the output name and the value is a JSON object with these fields:
|
||||||
|
|
||||||
* `path`: The output path.
|
* `path`:
|
||||||
|
The output path, if it is known in advanced.
|
||||||
|
Otherwise, `null`.
|
||||||
|
|
||||||
|
|
||||||
|
* `method`:
|
||||||
|
For an output which will be [content addresed], a string representing the [method](@docroot@/store/store-object/content-address.md) of content addressing that is chosen.
|
||||||
|
Valid method strings are:
|
||||||
|
|
||||||
|
- [`flat`](@docroot@/store/store-object/content-address.md#method-flat)
|
||||||
|
- [`nar`](@docroot@/store/store-object/content-address.md#method-nix-archive)
|
||||||
|
- [`text`](@docroot@/store/store-object/content-address.md#method-text)
|
||||||
|
- [`git`](@docroot@/store/store-object/content-address.md#method-git)
|
||||||
|
|
||||||
|
Otherwise, `null`.
|
||||||
|
|
||||||
* `hashAlgo`:
|
* `hashAlgo`:
|
||||||
For fixed-output derivations, the hashing algorithm (e.g. `sha256`), optionally prefixed by `r:` if `hash` denotes a NAR hash rather than a flat file hash.
|
For an output which will be [content addresed], the name of the hash algorithm used.
|
||||||
|
Valid algorithm strings are:
|
||||||
|
|
||||||
|
- `md5`
|
||||||
|
- `sha1`
|
||||||
|
- `sha256`
|
||||||
|
- `sha512`
|
||||||
|
|
||||||
* `hash`:
|
* `hash`:
|
||||||
For fixed-output derivations, the expected content hash in base-16.
|
For fixed-output derivations, the expected content hash in base-16.
|
||||||
|
@ -32,7 +52,8 @@ is a JSON object with the following fields:
|
||||||
> "outputs": {
|
> "outputs": {
|
||||||
> "out": {
|
> "out": {
|
||||||
> "path": "/nix/store/2543j7c6jn75blc3drf4g5vhb1rhdq29-source",
|
> "path": "/nix/store/2543j7c6jn75blc3drf4g5vhb1rhdq29-source",
|
||||||
> "hashAlgo": "r:sha256",
|
> "method": "nar",
|
||||||
|
> "hashAlgo": "sha256",
|
||||||
> "hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62"
|
> "hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62"
|
||||||
> }
|
> }
|
||||||
> }
|
> }
|
||||||
|
|
|
@ -36,18 +36,23 @@ where
|
||||||
- `type` = one of:
|
- `type` = one of:
|
||||||
|
|
||||||
- ```ebnf
|
- ```ebnf
|
||||||
| "text" ( ":" store-path )*
|
| "text" { ":" store-path }
|
||||||
```
|
```
|
||||||
|
|
||||||
for encoded derivations written to the store.
|
This is for the
|
||||||
|
["Text"](@docroot@/store/store-object/content-address.md#method-text)
|
||||||
|
method of content addressing store objects.
|
||||||
The optional trailing store paths are the references of the store object.
|
The optional trailing store paths are the references of the store object.
|
||||||
|
|
||||||
- ```ebnf
|
- ```ebnf
|
||||||
| "source" ( ":" store-path )*
|
| "source" { ":" store-path } [ ":self" ]
|
||||||
```
|
```
|
||||||
|
|
||||||
For paths copied to the store and hashed via a [Nix Archive (NAR)] and [SHA-256][sha-256].
|
This is for the
|
||||||
Just like in the text case, we can have the store objects referenced by their paths.
|
["Nix Archive"](@docroot@/store/store-object/content-address.md#method-nix-archive)
|
||||||
|
method of content addressing store objects,
|
||||||
|
if the hash algorithm is [SHA-256].
|
||||||
|
Just like in the "Text" case, we can have the store objects referenced by their paths.
|
||||||
Additionally, we can have an optional `:self` label to denote self reference.
|
Additionally, we can have an optional `:self` label to denote self reference.
|
||||||
|
|
||||||
- ```ebnf
|
- ```ebnf
|
||||||
|
@ -55,8 +60,12 @@ where
|
||||||
```
|
```
|
||||||
|
|
||||||
For either the outputs built from derivations,
|
For either the outputs built from derivations,
|
||||||
paths copied to the store hashed that area single file hashed directly, or the via a hash algorithm other than [SHA-256][sha-256].
|
or content-addressed store objects that are not using one of the two above cases.
|
||||||
(in that case "source" is used; this is only necessary for compatibility).
|
To be explicit about the latter, that is currently these methods:
|
||||||
|
|
||||||
|
- ["Flat"](@docroot@/store/store-object/content-address.md#method-flat)
|
||||||
|
- ["Git"](@docroot@/store/store-object/content-address.md#method-git)
|
||||||
|
- ["Nix Archive"](@docroot@/store/store-object/content-address.md#method-nix-archive) if the hash algorithm is not [SHA-256].
|
||||||
|
|
||||||
`id` is the name of the output (usually, "out").
|
`id` is the name of the output (usually, "out").
|
||||||
For content-addressed store objects, `id`, is always "out".
|
For content-addressed store objects, `id`, is always "out".
|
||||||
|
@ -116,7 +125,7 @@ where
|
||||||
Also note that NAR + SHA-256 must not use this case, and instead must use the `type` = `"source:" ...` case.
|
Also note that NAR + SHA-256 must not use this case, and instead must use the `type` = `"source:" ...` case.
|
||||||
|
|
||||||
[Nix Archive (NAR)]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
|
[Nix Archive (NAR)]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
|
||||||
[sha-256]: https://en.m.wikipedia.org/wiki/SHA-256
|
[SHA-256]: https://en.m.wikipedia.org/wiki/SHA-256
|
||||||
|
|
||||||
### Historical Note
|
### Historical Note
|
||||||
|
|
||||||
|
|
|
@ -1,7 +1,9 @@
|
||||||
# Content-Addressing File System Objects
|
# Content-Addressing File System Objects
|
||||||
|
|
||||||
For many operations, Nix needs to calculate [a content addresses](@docroot@/glossary.md#gloss-content-address) of [a file system object][file system object].
|
For many operations, Nix needs to calculate [a content addresses](@docroot@/glossary.md#gloss-content-address) of [a file system object][file system object].
|
||||||
Usually this is needed as part of content addressing [store objects], since store objects always have a root file system object.
|
Usually this is needed as part of
|
||||||
|
[content addressing store objects](../store-object/content-address.md),
|
||||||
|
since store objects always have a root file system object.
|
||||||
But some command-line utilities also just work on "raw" file system objects, not part of any store object.
|
But some command-line utilities also just work on "raw" file system objects, not part of any store object.
|
||||||
|
|
||||||
Every content addressing scheme Nix uses ultimately involves feeding data into a [hash function](https://en.wikipedia.org/wiki/Hash_function), and getting back an opaque fixed-size digest which is deemed a content address.
|
Every content addressing scheme Nix uses ultimately involves feeding data into a [hash function](https://en.wikipedia.org/wiki/Hash_function), and getting back an opaque fixed-size digest which is deemed a content address.
|
||||||
|
@ -18,6 +20,9 @@ A single file object can just be hashed by its contents.
|
||||||
This is not enough information to encode the fact that the file system object is a file,
|
This is not enough information to encode the fact that the file system object is a file,
|
||||||
but if we *already* know that the FSO is a single non-executable file by other means, it is sufficient.
|
but if we *already* know that the FSO is a single non-executable file by other means, it is sufficient.
|
||||||
|
|
||||||
|
Because the hashed data is just the raw file, as is, this choice is good for compatibility with other systems.
|
||||||
|
For example, Unix commands like `sha256sum` or `sha1sum` will produce hashes for single files that match this.
|
||||||
|
|
||||||
### Nix Archive (NAR) { #serial-nix-archive }
|
### Nix Archive (NAR) { #serial-nix-archive }
|
||||||
|
|
||||||
For the other cases of [file system objects][file system object], especially directories with arbitrary descendents, we need a more complex serialisation format.
|
For the other cases of [file system objects][file system object], especially directories with arbitrary descendents, we need a more complex serialisation format.
|
||||||
|
@ -69,7 +74,7 @@ every non-directory object is owned by a parent directory, and the entry that re
|
||||||
However, if the root object is not a directory, then we have no way of knowing which one of an executable file, non-executable file, or symlink it is supposed to be.
|
However, if the root object is not a directory, then we have no way of knowing which one of an executable file, non-executable file, or symlink it is supposed to be.
|
||||||
|
|
||||||
In response to this, we have decided to treat a bare file as non-executable file.
|
In response to this, we have decided to treat a bare file as non-executable file.
|
||||||
This is similar to do what we do with [flat serialisation](#flat), which also lacks this information.
|
This is similar to do what we do with [flat serialisation](#serial-flat), which also lacks this information.
|
||||||
To avoid an address collision, attempts to hash a bare executable file or symlink will result in an error (just as would happen for flat serialisation also).
|
To avoid an address collision, attempts to hash a bare executable file or symlink will result in an error (just as would happen for flat serialisation also).
|
||||||
Thus, Git can encode some, but not all of Nix's "File System Objects", and this sort of content-addressing is likewise partial.
|
Thus, Git can encode some, but not all of Nix's "File System Objects", and this sort of content-addressing is likewise partial.
|
||||||
|
|
||||||
|
|
123
doc/manual/src/store/store-object/content-address.md
Normal file
123
doc/manual/src/store/store-object/content-address.md
Normal file
|
@ -0,0 +1,123 @@
|
||||||
|
# Content-Addressing Store Objects
|
||||||
|
|
||||||
|
Just [like][fso-ca] [File System Objects][File System Object],
|
||||||
|
[Store Objects][Store Object] can also be [content-addressed](@docroot@/glossary.md#gloss-content-addressed),
|
||||||
|
unless they are [input-addressed](@docroot@/glossary.md#gloss-input-addressed-store-object).
|
||||||
|
|
||||||
|
For store objects, the content address we produce will take the form of a [Store Path] rather than regular hash.
|
||||||
|
In particular, the content-addressing scheme will ensure that the digest of the store path is solely computed from the
|
||||||
|
|
||||||
|
- file system object graph (the root one and its children, if it has any)
|
||||||
|
- references
|
||||||
|
- [store directory](../store-path.md#store-directory)
|
||||||
|
- name
|
||||||
|
|
||||||
|
of the store object, and not any other information, which would not be an intrinsic property of that store object.
|
||||||
|
|
||||||
|
For the full specification of the algorithms involved, see the [specification of store path digests][sp-spec].
|
||||||
|
|
||||||
|
[File System Object]: ../file-system-object.md
|
||||||
|
[Store Object]: ../store-object.md
|
||||||
|
[Store Path]: ../store-path.md
|
||||||
|
|
||||||
|
## Content addressing each part of a store object
|
||||||
|
|
||||||
|
### File System Objects
|
||||||
|
|
||||||
|
With all currently supported store object content addressing methods, the file system object is always [content-addressed][fso-ca] first, and then that hash is incorporated into content address computation for the store object.
|
||||||
|
|
||||||
|
### References
|
||||||
|
|
||||||
|
With all currently supported store object content addressing methods,
|
||||||
|
other objects are referred to by their regular (string-encoded-) [store paths][Store Path].
|
||||||
|
|
||||||
|
Self-references however cannot be referred to by their path, because we are in the midst of describing how to compute that path!
|
||||||
|
|
||||||
|
> The alternative would require finding as hash function fixed point, i.e. the solution to an equation in the form
|
||||||
|
> ```
|
||||||
|
> digest = hash(..... || digest || ....)
|
||||||
|
> ```
|
||||||
|
> which is computationally infeasible.
|
||||||
|
> As far as we know, this is equivalent to finding a hash collision.
|
||||||
|
|
||||||
|
Instead we just have a "has self reference" boolean, which will end up affecting the digest.
|
||||||
|
|
||||||
|
### Name and Store Directory
|
||||||
|
|
||||||
|
These two items affect the digest in a way that is standard for store path digest computations and not specific to content-addressing.
|
||||||
|
Consult the [specification of store path digests][sp-spec] for further details.
|
||||||
|
|
||||||
|
## Content addressing Methods
|
||||||
|
|
||||||
|
For historical reasons, we don't support all features in all combinations.
|
||||||
|
Each currently supported method of content addressing chooses a single method of file system object hashing, and may offer some restrictions on references.
|
||||||
|
The names and store directories are unrestricted however.
|
||||||
|
|
||||||
|
### Flat { #method-flat }
|
||||||
|
|
||||||
|
This uses the corresponding [Flat](../file-system-object/content-address.md#serial-flat) method of file system object content addressing.
|
||||||
|
|
||||||
|
References are not supported: store objects with flat hashing *and* references can not be created.
|
||||||
|
|
||||||
|
### Text { #method-text }
|
||||||
|
|
||||||
|
This also uses the corresponding [Flat](../file-system-object/content-address.md#serial-flat) method of file system object content addressing.
|
||||||
|
|
||||||
|
References to other store objects are supported, but self references are not.
|
||||||
|
|
||||||
|
This is the only store-object content-addressing method that is not named identically with a corresponding file system object method.
|
||||||
|
It is somewhat obscure, mainly used for "drv files"
|
||||||
|
(derivations serialized as store objects in their ["ATerm" file format](@docroot@/protocols/derivation-aterm.md)).
|
||||||
|
Prefer another method if possible.
|
||||||
|
|
||||||
|
### Nix Archive { #method-nix-archive }
|
||||||
|
|
||||||
|
This uses the corresponding [Nix Archive](../file-system-object/content-address.md#serial-nix-archive) method of file system object content addressing.
|
||||||
|
|
||||||
|
References (to other store objects and self references alike) are supported so long as the hash algorithm is SHA-256, but not (neither kind) otherwise.
|
||||||
|
|
||||||
|
### Git { #method-git }
|
||||||
|
|
||||||
|
> **Warning**
|
||||||
|
>
|
||||||
|
> This method is part of the [`git-hashing`][xp-feature-git-hashing] experimental feature.
|
||||||
|
|
||||||
|
This uses the corresponding [Git](../file-system-object/content-address.md#serial-git) method of file system object content addressing.
|
||||||
|
|
||||||
|
References are not supported.
|
||||||
|
|
||||||
|
Only SHA-1 is supported at this time.
|
||||||
|
If [SHA-256-based Git](https://git-scm.com/docs/hash-function-transition)
|
||||||
|
becomes more widespread, this restriction will be revisited.
|
||||||
|
|
||||||
|
### Reproducibility
|
||||||
|
|
||||||
|
The above system is more complex than it needs to be to support all types of file system objects and references, owing to accretion of features over time.
|
||||||
|
However, there's a lot of value in supporting old expressions and reproducing the same hashes with any version of Nix.
|
||||||
|
Still, the fundamental property remains that if one knows how a store object is supposed to be hashed
|
||||||
|
--- all the non-Hash, non-references information above
|
||||||
|
--- one can recompute a store object's store path just from that metadata and its content proper (its references and file system objects).
|
||||||
|
Collectively, we can call this information the "content address method".
|
||||||
|
|
||||||
|
By storing the "Content address method" extra information as part of store object
|
||||||
|
--- making it data not metadata
|
||||||
|
--- we achieve the key property of making content-addressed store objects *trustless*.
|
||||||
|
|
||||||
|
What this is means is that they are just plain old data, not containing any "claim" that could be false.
|
||||||
|
All this information is free to vary, and if any of it varies one gets (ignoring the possibility of hash collisions, as usual) a different store path.
|
||||||
|
Store paths referring to content-addressed store objects uniquely identify a store object, and given that object, one can recompute the store path.
|
||||||
|
Any content-addressed store object purporting to be the referee of a store object can be readily verified to see whether it in fact does without any extra information.
|
||||||
|
No other party claiming a store object corresponds to a store path need be trusted because this verification can be done instead.
|
||||||
|
|
||||||
|
Content addressing currently is used when adding data like source code to the store.
|
||||||
|
Such data are "basal inputs", not produced from any other derivation (to our knowledge).
|
||||||
|
Content addressing is thus the only way to address them of our two options.
|
||||||
|
([Input addressing](@docroot@/glossary.md#gloss-input-addressed-store-object), is only valid for store paths produced from derivations.)
|
||||||
|
|
||||||
|
Additionally, content addressing is also used for the outputs of certain sorts of derivations.
|
||||||
|
It is very nice to be able to uniformly content-address all data rather than rely on a mix of content addressing and input addressing.
|
||||||
|
This however, is in some cases still experimental, so in practice input addressing is still (as of 2022) widely used.
|
||||||
|
|
||||||
|
[fso-ca]: ../file-system-object/content-address.md
|
||||||
|
[sp-spec]: @docroot@/protocols/store-path.md
|
||||||
|
[xp-feature-git-hashing]: @docroot@/contributing/experimental-features.md#xp-feature-git-hashing
|
|
@ -103,27 +103,27 @@ Args::Flag contentAddressMethod(ContentAddressMethod * method)
|
||||||
return Args::Flag {
|
return Args::Flag {
|
||||||
.longName = "mode",
|
.longName = "mode",
|
||||||
// FIXME indentation carefully made for context, this is messed up.
|
// FIXME indentation carefully made for context, this is messed up.
|
||||||
/* FIXME link to store object content-addressing not file system
|
|
||||||
object content addressing once we have that page. */
|
|
||||||
.description = R"(
|
.description = R"(
|
||||||
How to compute the content-address of the store object.
|
How to compute the content-address of the store object.
|
||||||
One of:
|
One of:
|
||||||
|
|
||||||
- `nar` (the default):
|
- [`nar`](@docroot@/store/store-object/content-address.md#method-nix-archive)
|
||||||
|
(the default):
|
||||||
Serialises the input as a
|
Serialises the input as a
|
||||||
[Nix Archive](@docroot@/store/file-system-object/content-address.md#serial-nix-archive)
|
[Nix Archive](@docroot@/store/file-system-object/content-address.md#serial-nix-archive)
|
||||||
and passes that to the hash function.
|
and passes that to the hash function.
|
||||||
|
|
||||||
- `flat`:
|
- [`flat`](@docroot@/store/store-object/content-address.md#method-flat):
|
||||||
Assumes that the input is a single file and
|
Assumes that the input is a single file and
|
||||||
[directly passes](@docroot@/store/file-system-object/content-address.md#serial-flat)
|
[directly passes](@docroot@/store/file-system-object/content-address.md#serial-flat)
|
||||||
it to the hash function.
|
it to the hash function.
|
||||||
|
|
||||||
- `text`: Like `flat`, but used for
|
- [`text`](@docroot@/store/store-object/content-address.md#method-text):
|
||||||
|
Like `flat`, but used for
|
||||||
[derivations](@docroot@/glossary.md#store-derivation) serialized in store object and
|
[derivations](@docroot@/glossary.md#store-derivation) serialized in store object and
|
||||||
[`builtins.toFile`](@docroot@/language/builtins.html#builtins-toFile).
|
[`builtins.toFile`](@docroot@/language/builtins.html#builtins-toFile).
|
||||||
For advanced use-cases only;
|
For advanced use-cases only;
|
||||||
for regular usage prefer `nar` and `flat.
|
for regular usage prefer `nar` and `flat`.
|
||||||
)",
|
)",
|
||||||
.labels = {"content-address-method"},
|
.labels = {"content-address-method"},
|
||||||
.handler = {[method](std::string s) {
|
.handler = {[method](std::string s) {
|
||||||
|
|
|
@ -1216,16 +1216,19 @@ nlohmann::json DerivationOutput::toJSON(
|
||||||
},
|
},
|
||||||
[&](const DerivationOutput::CAFixed & dof) {
|
[&](const DerivationOutput::CAFixed & dof) {
|
||||||
res["path"] = store.printStorePath(dof.path(store, drvName, outputName));
|
res["path"] = store.printStorePath(dof.path(store, drvName, outputName));
|
||||||
res["hashAlgo"] = dof.ca.printMethodAlgo();
|
res["method"] = std::string { dof.ca.method.render() };
|
||||||
|
res["hashAlgo"] = printHashAlgo(dof.ca.hash.algo);
|
||||||
res["hash"] = dof.ca.hash.to_string(HashFormat::Base16, false);
|
res["hash"] = dof.ca.hash.to_string(HashFormat::Base16, false);
|
||||||
// FIXME print refs?
|
// FIXME print refs?
|
||||||
},
|
},
|
||||||
[&](const DerivationOutput::CAFloating & dof) {
|
[&](const DerivationOutput::CAFloating & dof) {
|
||||||
res["hashAlgo"] = std::string { dof.method.renderPrefix() } + printHashAlgo(dof.hashAlgo);
|
res["method"] = std::string { dof.method.render() };
|
||||||
|
res["hashAlgo"] = printHashAlgo(dof.hashAlgo);
|
||||||
},
|
},
|
||||||
[&](const DerivationOutput::Deferred &) {},
|
[&](const DerivationOutput::Deferred &) {},
|
||||||
[&](const DerivationOutput::Impure & doi) {
|
[&](const DerivationOutput::Impure & doi) {
|
||||||
res["hashAlgo"] = std::string { doi.method.renderPrefix() } + printHashAlgo(doi.hashAlgo);
|
res["method"] = std::string { doi.method.render() };
|
||||||
|
res["hashAlgo"] = printHashAlgo(doi.hashAlgo);
|
||||||
res["impure"] = true;
|
res["impure"] = true;
|
||||||
},
|
},
|
||||||
}, raw);
|
}, raw);
|
||||||
|
@ -1245,12 +1248,13 @@ DerivationOutput DerivationOutput::fromJSON(
|
||||||
keys.insert(key);
|
keys.insert(key);
|
||||||
|
|
||||||
auto methodAlgo = [&]() -> std::pair<ContentAddressMethod, HashAlgorithm> {
|
auto methodAlgo = [&]() -> std::pair<ContentAddressMethod, HashAlgorithm> {
|
||||||
auto & str = getString(valueAt(json, "hashAlgo"));
|
auto & method_ = getString(valueAt(json, "method"));
|
||||||
std::string_view s = str;
|
ContentAddressMethod method = ContentAddressMethod::parse(method_);
|
||||||
ContentAddressMethod method = ContentAddressMethod::parsePrefix(s);
|
|
||||||
if (method == TextIngestionMethod {})
|
if (method == TextIngestionMethod {})
|
||||||
xpSettings.require(Xp::DynamicDerivations);
|
xpSettings.require(Xp::DynamicDerivations);
|
||||||
auto hashAlgo = parseHashAlgo(s);
|
|
||||||
|
auto & hashAlgo_ = getString(valueAt(json, "hashAlgo"));
|
||||||
|
auto hashAlgo = parseHashAlgo(hashAlgo_);
|
||||||
return { std::move(method), std::move(hashAlgo) };
|
return { std::move(method), std::move(hashAlgo) };
|
||||||
};
|
};
|
||||||
|
|
||||||
|
@ -1260,7 +1264,7 @@ DerivationOutput DerivationOutput::fromJSON(
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
else if (keys == (std::set<std::string_view> { "path", "hashAlgo", "hash" })) {
|
else if (keys == (std::set<std::string_view> { "path", "method", "hashAlgo", "hash" })) {
|
||||||
auto [method, hashAlgo] = methodAlgo();
|
auto [method, hashAlgo] = methodAlgo();
|
||||||
auto dof = DerivationOutput::CAFixed {
|
auto dof = DerivationOutput::CAFixed {
|
||||||
.ca = ContentAddress {
|
.ca = ContentAddress {
|
||||||
|
@ -1273,7 +1277,7 @@ DerivationOutput DerivationOutput::fromJSON(
|
||||||
return dof;
|
return dof;
|
||||||
}
|
}
|
||||||
|
|
||||||
else if (keys == (std::set<std::string_view> { "hashAlgo" })) {
|
else if (keys == (std::set<std::string_view> { "method", "hashAlgo" })) {
|
||||||
xpSettings.require(Xp::CaDerivations);
|
xpSettings.require(Xp::CaDerivations);
|
||||||
auto [method, hashAlgo] = methodAlgo();
|
auto [method, hashAlgo] = methodAlgo();
|
||||||
return DerivationOutput::CAFloating {
|
return DerivationOutput::CAFloating {
|
||||||
|
@ -1286,7 +1290,7 @@ DerivationOutput DerivationOutput::fromJSON(
|
||||||
return DerivationOutput::Deferred {};
|
return DerivationOutput::Deferred {};
|
||||||
}
|
}
|
||||||
|
|
||||||
else if (keys == (std::set<std::string_view> { "hashAlgo", "impure" })) {
|
else if (keys == (std::set<std::string_view> { "method", "hashAlgo", "impure" })) {
|
||||||
xpSettings.require(Xp::ImpureDerivations);
|
xpSettings.require(Xp::ImpureDerivations);
|
||||||
auto [method, hashAlgo] = methodAlgo();
|
auto [method, hashAlgo] = methodAlgo();
|
||||||
return DerivationOutput::Impure {
|
return DerivationOutput::Impure {
|
||||||
|
|
|
@ -134,7 +134,9 @@ The following generic flake reference attributes are supported:
|
||||||
repository or tarball. The default is the root directory of the
|
repository or tarball. The default is the root directory of the
|
||||||
flake.
|
flake.
|
||||||
|
|
||||||
* `narHash`: The hash of the NAR serialisation (in SRI format) of the
|
* `narHash`: The hash of the
|
||||||
|
[Nix Archive (NAR) serialisation][Nix Archive]
|
||||||
|
(in SRI format) of the
|
||||||
contents of the flake. This is useful for flake types such as
|
contents of the flake. This is useful for flake types such as
|
||||||
tarballs that lack a unique content identifier such as a Git commit
|
tarballs that lack a unique content identifier such as a Git commit
|
||||||
hash.
|
hash.
|
||||||
|
@ -423,8 +425,9 @@ The following attributes are supported in `flake.nix`:
|
||||||
* `lastModified`: The commit time of the revision `rev` as an integer
|
* `lastModified`: The commit time of the revision `rev` as an integer
|
||||||
denoting the number of seconds since 1970.
|
denoting the number of seconds since 1970.
|
||||||
|
|
||||||
* `narHash`: The SHA-256 (in SRI format) of the NAR serialization of
|
* `narHash`: The SHA-256 (in SRI format) of the
|
||||||
the flake's source tree.
|
[Nix Archive (NAR) serialisation][Nix Archive]
|
||||||
|
NAR serialization of the flake's source tree.
|
||||||
|
|
||||||
The value returned by the `outputs` function must be an attribute
|
The value returned by the `outputs` function must be an attribute
|
||||||
set. The attributes can have arbitrary values; however, various
|
set. The attributes can have arbitrary values; however, various
|
||||||
|
@ -703,4 +706,5 @@ will not look at the lock files of dependencies. However, lock file
|
||||||
generation itself *does* use the lock files of dependencies by
|
generation itself *does* use the lock files of dependencies by
|
||||||
default.
|
default.
|
||||||
|
|
||||||
|
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
|
||||||
)""
|
)""
|
||||||
|
|
|
@ -2,7 +2,7 @@ R""(
|
||||||
|
|
||||||
# Examples
|
# Examples
|
||||||
|
|
||||||
* List a file in a NAR and pipe it through `gunzip`:
|
* List a file in a [Nix Archive (NAR)][Nix Archive] and pipe it through `gunzip`:
|
||||||
|
|
||||||
```console
|
```console
|
||||||
# nix nar cat ./hello.nar /share/man/man1/hello.1.gz | gunzip
|
# nix nar cat ./hello.nar /share/man/man1/hello.1.gz | gunzip
|
||||||
|
@ -16,4 +16,5 @@ R""(
|
||||||
This command prints on standard output the contents of the regular
|
This command prints on standard output the contents of the regular
|
||||||
file *path* inside the NAR file *nar*.
|
file *path* inside the NAR file *nar*.
|
||||||
|
|
||||||
|
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
|
||||||
)""
|
)""
|
||||||
|
|
|
@ -2,7 +2,7 @@ R""(
|
||||||
|
|
||||||
# Examples
|
# Examples
|
||||||
|
|
||||||
* To serialise directory `foo` as a NAR:
|
* To serialise directory `foo` as a [Nix Archive (NAR)][Nix Archive]:
|
||||||
|
|
||||||
```console
|
```console
|
||||||
# nix nar pack ./foo > foo.nar
|
# nix nar pack ./foo > foo.nar
|
||||||
|
@ -10,8 +10,10 @@ R""(
|
||||||
|
|
||||||
# Description
|
# Description
|
||||||
|
|
||||||
This command generates a NAR file containing the serialisation of
|
This command generates a [Nix Archive (NAR)][Nix Archive] file containing the serialisation of
|
||||||
*path*, which must contain only regular files, directories and
|
*path*, which must contain only regular files, directories and
|
||||||
symbolic links. The NAR is written to standard output.
|
symbolic links. The NAR is written to standard output.
|
||||||
|
|
||||||
|
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
|
||||||
|
|
||||||
)""
|
)""
|
||||||
|
|
|
@ -2,7 +2,7 @@ R""(
|
||||||
|
|
||||||
# Examples
|
# Examples
|
||||||
|
|
||||||
* To list a specific file in a NAR:
|
* To list a specific file in a [NAR][Nix Archive]:
|
||||||
|
|
||||||
```console
|
```console
|
||||||
# nix nar ls --long ./hello.nar /bin/hello
|
# nix nar ls --long ./hello.nar /bin/hello
|
||||||
|
@ -19,6 +19,8 @@ R""(
|
||||||
|
|
||||||
# Description
|
# Description
|
||||||
|
|
||||||
This command shows information about a *path* inside NAR file *nar*.
|
This command shows information about a *path* inside [Nix Archive (NAR)][Nix Archive] file *nar*.
|
||||||
|
|
||||||
|
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
|
||||||
|
|
||||||
)""
|
)""
|
||||||
|
|
|
@ -3,11 +3,14 @@ R""(
|
||||||
# Description
|
# Description
|
||||||
|
|
||||||
`nix nar` provides several subcommands for creating and inspecting
|
`nix nar` provides several subcommands for creating and inspecting
|
||||||
*Nix Archives* (NARs).
|
[*Nix Archives* (NARs)][Nix Archive].
|
||||||
|
|
||||||
# File format
|
# File format
|
||||||
|
|
||||||
For the definition of the NAR file format, see Figure 5.2 in
|
For the definition of the Nix Archive file format, see
|
||||||
https://edolstra.github.io/pubs/phd-thesis.pdf.
|
[within the protocols chapter](@docroot@/protocols/nix-archive.md)
|
||||||
|
of the manual.
|
||||||
|
|
||||||
|
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
|
||||||
|
|
||||||
)""
|
)""
|
||||||
|
|
|
@ -17,7 +17,9 @@ R""(
|
||||||
|
|
||||||
# Description
|
# Description
|
||||||
|
|
||||||
This command generates a NAR file containing the serialisation of the
|
This command generates a [Nix Archive (NAR)][Nix Archive] file containing the serialisation of the
|
||||||
store path [*installable*](./nix.md#installables). The NAR is written to standard output.
|
store path [*installable*](./nix.md#installables). The NAR is written to standard output.
|
||||||
|
|
||||||
|
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
|
||||||
|
|
||||||
)""
|
)""
|
||||||
|
|
|
@ -46,4 +46,6 @@ The exit status of this command is the sum of the following values:
|
||||||
* **4** if any path couldn't be verified for any other reason (such as
|
* **4** if any path couldn't be verified for any other reason (such as
|
||||||
an I/O error).
|
an I/O error).
|
||||||
|
|
||||||
|
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
|
||||||
|
|
||||||
)""
|
)""
|
||||||
|
|
|
@ -1,5 +1,6 @@
|
||||||
{
|
{
|
||||||
"hash": "894517c9163c896ec31a2adbd33c0681fd5f45b2c0ef08a64c92a03fb97f390f",
|
"hash": "894517c9163c896ec31a2adbd33c0681fd5f45b2c0ef08a64c92a03fb97f390f",
|
||||||
"hashAlgo": "sha256",
|
"hashAlgo": "sha256",
|
||||||
|
"method": "flat",
|
||||||
"path": "/nix/store/rhcg9h16sqvlbpsa6dqm57sbr2al6nzg-drv-name-output-name"
|
"path": "/nix/store/rhcg9h16sqvlbpsa6dqm57sbr2al6nzg-drv-name-output-name"
|
||||||
}
|
}
|
||||||
|
|
|
@ -1,5 +1,6 @@
|
||||||
{
|
{
|
||||||
"hash": "894517c9163c896ec31a2adbd33c0681fd5f45b2c0ef08a64c92a03fb97f390f",
|
"hash": "894517c9163c896ec31a2adbd33c0681fd5f45b2c0ef08a64c92a03fb97f390f",
|
||||||
"hashAlgo": "r:sha256",
|
"hashAlgo": "sha256",
|
||||||
|
"method": "nar",
|
||||||
"path": "/nix/store/c015dhfh5l0lp6wxyvdn7bmwhbbr6hr9-drv-name-output-name"
|
"path": "/nix/store/c015dhfh5l0lp6wxyvdn7bmwhbbr6hr9-drv-name-output-name"
|
||||||
}
|
}
|
||||||
|
|
|
@ -1,5 +1,6 @@
|
||||||
{
|
{
|
||||||
"hash": "894517c9163c896ec31a2adbd33c0681fd5f45b2c0ef08a64c92a03fb97f390f",
|
"hash": "894517c9163c896ec31a2adbd33c0681fd5f45b2c0ef08a64c92a03fb97f390f",
|
||||||
"hashAlgo": "text:sha256",
|
"hashAlgo": "sha256",
|
||||||
|
"method": "text",
|
||||||
"path": "/nix/store/6s1zwabh956jvhv4w9xcdb5jiyanyxg1-drv-name-output-name"
|
"path": "/nix/store/6s1zwabh956jvhv4w9xcdb5jiyanyxg1-drv-name-output-name"
|
||||||
}
|
}
|
||||||
|
|
|
@ -1,3 +1,4 @@
|
||||||
{
|
{
|
||||||
"hashAlgo": "r:sha256"
|
"hashAlgo": "sha256",
|
||||||
|
"method": "nar"
|
||||||
}
|
}
|
||||||
|
|
|
@ -1,4 +1,5 @@
|
||||||
{
|
{
|
||||||
"hashAlgo": "r:sha256",
|
"hashAlgo": "sha256",
|
||||||
"impure": true
|
"impure": true,
|
||||||
|
"method": "nar"
|
||||||
}
|
}
|
||||||
|
|
Loading…
Reference in a new issue