Commit graph

2245 commits

Author SHA1 Message Date
Eelco Dolstra
c8eb3b82e9
Merge pull request #9129 from fricklerhandwerk/ae-be
AE -> BE; fix redirects
2023-10-09 16:56:07 +02:00
John Ericson
217d863f7a Merge branch 'master' into lookup-path 2023-10-09 10:54:54 -04:00
John Ericson
67eddc05ef
Merge pull request #9116 from fricklerhandwerk/doc-import
reword and reformat description of `builtins.import`
2023-10-09 10:26:41 -04:00
John Ericson
81d3a8542a
doc: Slight reword of "interpolated expression" in paragraph
I was sleepy and confused that "interpolated expression" was a new type of thing at first. This nudges the reader to understand that its just a regular expression, and these conditions are imposed by the interpolation operation.
2023-10-09 10:21:18 -04:00
Valentin Gagarin
0246de1896 remove unnecessary indentation from markdown list
this makes it a bit easier to work with, as some tooling doesn't work
well with too much indentation.
2023-10-09 10:14:29 +02:00
Valentin Gagarin
896a905202 AE -> BE; fix redirects 2023-10-09 10:07:46 +02:00
Valentin Gagarin
f00a5eb11b introduce lookup paths as a distinct language construct
so far they did not really have a name, and were at best referred to as
"angle bracket syntax".
2023-10-07 04:44:09 +02:00
Valentin Gagarin
a7ba8c3f4a complete example on attribute name interpolation 2023-10-07 02:49:55 +02:00
Valentin Gagarin
a67cee965a expand on interpolated expressions 2023-10-07 02:49:55 +02:00
Valentin Gagarin
517c547dec remove duplicate redirects entry 2023-10-06 23:34:08 +02:00
John Ericson
68c81c7375 Put functional tests in tests/functional
I think it is bad for these reasons when `tests/` contains a mix of
functional and integration tests

 - Concepts is harder to understand, the documentation makes a good
   unit vs functional vs integration distinction, but when the
   integration tests are just two subdirs within `tests/` this is not
   clear.

 - Source filtering in the `flake.nix` is more complex. We need to
   filter out some of the dirs from `tests/`, rather than simply pick
   the dirs we want and take all of them. This is a good sign the
   structure of what we are trying to do is not matching the structure
   of the files.

With this change we have a clean:
```shell-session
$ git show 'HEAD:tests'
tree HEAD:tests

functional/
installer/
nixos/
```
2023-10-06 09:05:56 -04:00
Valentin Gagarin
369b076986 add links and anchors 2023-10-06 11:46:41 +02:00
Valentin Gagarin
d12fb4b1f1
Merge pull request #9017 from fricklerhandwerk/contributing-docs
add contributing guide for documentation
2023-10-05 09:23:27 +02:00
Valentin Gagarin
e0e47c0a68 accommodate inconsistent output from lowdown
the `term` output mode leaves inline HTML around verbatim, while `nroff`
mode (used for `man` pages) does not.

the correct solution would be to pre-render all output with a more
benign tool so we have less liabilities in our own code, but this has to
do for now.
2023-10-05 01:20:26 +02:00
Valentin Gagarin
64b73476ce always show anchors on setting listings
refactor the templates for readability
2023-10-05 00:05:08 +02:00
Maximilian Bosch
42e3c6d658 doc: reference NIX_ATTRS_*_FILE vars at the env var reference for drvs 2023-10-01 13:25:29 +01:00
Maximilian Bosch
bfdd908f7d structured attrs: improve support / usage of NIX_ATTRS_{SH,JSON}_FILE
In #4770 I implemented proper `nix-shell(1)` support for derivations
using `__structuredAttrs = true;`. Back then we decided to introduce two
new environment variables, `NIX_ATTRS_SH_FILE` for `.attrs.sh` and
`NIX_ATTRS_JSON_FILE` for `.attrs.json`. This was to avoid having to
copy these files to `$NIX_BUILD_TOP` in a `nix-shell(1)` session which
effectively meant copying these files to the project dir without
cleaning up afterwords[1].

On last NixCon I resumed hacking on `__structuredAttrs = true;` by
default for `nixpkgs` with a few other folks and getting back to it,
I identified a few problems with the how it's used in `nixpkgs`:

* A lot of builders in `nixpkgs` don't care about the env vars and
  assume that `.attrs.sh` and `.attrs.json` are in `$NIX_BUILD_TOP`.
  The sole reason why this works is that `nix-shell(1)` sources
  the contents of `.attrs.sh` and then sources `$stdenv/setup` if it
  exists. This may not be pretty, but it mostly works. One notable
  difference when using nixpkgs' stdenv as of now is however that
  `$__structuredAttrs` is set to `1` on regular builds, but set to
  an empty string in a shell session.

  Also, `.attrs.json` cannot be used in shell sessions because
  it can only be accessed by `$NIX_ATTRS_JSON_FILE` and not by
  `$NIX_BUILD_TOP/.attrs.json`.

  I considered changing Nix to be compatible with what nixpkgs
  effectively does, but then we'd have to either move $NIX_BUILD_TOP for
  shell sessions to a temporary location (and thus breaking a lot of
  assumptions) or we'd reintroduce all the problems we solved back then
  by using these two env vars.

  This is partly because I didn't document these variables back
  then (mea culpa), so I decided to drop all mentions of
  `.attrs.{json,sh}` in the  manual and only refer to `$NIX_ATTRS_SH_FILE`
  and `$NIX_ATTRS_JSON_FILE`. The same applies to all our integration tests.
  Theoretically we could deprecated using `"$NIX_BUILD_TOP"/.attrs.sh` in
  the future now.

* `nix develop` and `nix print-dev-env` don't support this environment
  variable at all even though they're supposed to be part of the replacement
  for `nix-shell` - for the drv debugging part to be precise.

  This isn't a big deal for the vast majority of derivations, i.e.
  derivations relying on nixpkgs' `stdenv` wiring things together
  properly. This is because `nix develop` effectively "clones" the
  derivation and replaces the builder with a script that dumps all of
  the environment, shell variables, functions etc, so the state of
  structured attrs being "sourced" is transmitted into the dev shell and
  most of the time you don't need to worry about `.attrs.sh` not
  existing because the shell is correctly configured and the

      if [ -e .attrs.sh ]; then source .attrs.sh; fi

  is simply omitted.

  However, this will break when having a derivation that reads e.g. from
  `.attrs.json` like

      with import <nixpkgs> {};
      runCommand "foo" { __structuredAttrs = true; foo.bar = 23; } ''
        cat $NIX_ATTRS_JSON_FILE # doesn't work because it points to /build/.attrs.json
      ''

  To work around this I employed a similar approach as it exists for
  `nix-shell`: the `NIX_ATTRS_{JSON,SH}_FILE` vars are replaced with
  temporary locations.

  The contents of `.attrs.sh` and `.attrs.json` are now written into the
  JSON by `get-env.sh`, the builder that `nix develop` injects into the
  derivation it's debugging. So finally the exact file contents are
  present and exported by `nix develop`.

  I also made `.attrs.json` a JSON string in the JSON printed by
  `get-env.sh` on purpose because then it's not necessary to serialize
  the object structure again. `nix develop` only needs the JSON
  as string because it's only written into the temporary file.

  I'm not entirely sure if it makes sense to also use a temporary
  location for `nix print-dev-env` (rather than just skipping the
  rewrite in there), but this would probably break certain cases where
  it's relied upon `$NIX_ATTRS_SH_FILE` to exist (prime example are the
  `nix print-dev-env` test-cases I wrote in this patch using
  `tests/shell.nix`, these would fail because the env var exists, but it
  cannot read from it).

[1] https://github.com/NixOS/nix/pull/4770#issuecomment-836799719
2023-10-01 13:22:48 +01:00
Théophane Hufschmitt
784c7df5bf
Merge pull request #9004 from fricklerhandwerk/release-support
add information on release cycle and backports
2023-09-29 15:04:00 +02:00
Théophane Hufschmitt
a66bd8f1f2
Merge pull request #9065 from fricklerhandwerk/testing-hints
add hint for troubleshooting tests
2023-09-29 13:52:54 +02:00
Valentin Gagarin
1dd03c62ad add hint for troubleshooting tests 2023-09-29 10:46:42 +02:00
tomberek
976f596579
Merge branch 'master' into tomberek.absolute.attrpath.notation 2023-09-28 10:01:57 -04:00
Ilan Joselevich
13ed5d7106 flakes: adopt repl-flake behavior as default 2023-09-27 20:47:10 -04:00
Robert Hensing
ea2fa8b6f3
Merge pull request #8241 from Luabee/patch-1
Mention `$DRV_PATH` in post-build-hook docs
2023-09-28 00:11:00 +01:00
Robert Hensing
dafa38213b
Update doc/manual/src/advanced-topics/post-build-hook.md
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-09-27 23:10:39 +01:00
Théophane Hufschmitt
c6faef61a6
Merge pull request #8923 from obsidiansystems/test-proto
Unit test some worker protocol serializers
2023-09-26 17:12:24 +02:00
Eelco Dolstra
706b91ef62
Merge pull request #9044 from fricklerhandwerk/doc-realise
realisation: reformat for readability
2023-09-26 15:28:20 +02:00
Valentin Gagarin
a757749fcf reword for readability 2023-09-26 06:28:17 +02:00
Valentin Gagarin
9428d7dcd1
Merge pull request #9045 from fricklerhandwerk/fix-redirects
fix broken redirects script
2023-09-26 04:27:33 +02:00
Valentin Gagarin
503b02d3f8
Merge pull request #9046 from fricklerhandwerk/fixup-derivation
fix broken reference link
2023-09-26 04:27:23 +02:00
Valentin Gagarin
60a155d01c
Merge pull request #8706 from fricklerhandwerk/doc-system-features
document system features
2023-09-26 04:21:31 +02:00
Valentin Gagarin
b17f200b11
Document "Import From Derivation" (#7332)
* document "Import From Derivation"

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-09-26 01:49:03 +00:00
Valentin Gagarin
c6f8247032 fix broken reference link 2023-09-26 03:10:12 +02:00
Théophane Hufschmitt
9a78d87bc0
Merge pull request #6614 from RasmusRendal/spaces
Implement support for percent encoded filepaths for flakerefs
2023-09-26 02:27:09 +02:00
Valentin Gagarin
45de35bcf1 fix broken redirects script 2023-09-26 01:24:07 +02:00
Valentin Gagarin
887cbcd395 add contributing guide for documentation 2023-09-26 01:06:47 +02:00
Valentin Gagarin
5b0336b3b1 reword example for clarity 2023-09-26 00:58:50 +02:00
Valentin Gagarin
7de66f19f8 example: headers -> dev
make the example more realistic, since `headers` is not an output name
used in Nixpkgs

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-26 00:46:27 +02:00
Valentin Gagarin
75a231147f be more precise about system semantics 2023-09-26 00:45:00 +02:00
Valentin Gagarin
5e4734a08b reword to avoid saying "build"
derivations are about data transformation, so the term "build" does not
add any information. there was also some feedback that "build task" is
not more helpful than "derivation" if you have no prior experience with
Nix or build systems, while existing associations may be misleading.
2023-09-26 00:45:00 +02:00
Valentin Gagarin
17884f54d1 clarification on extra attributes
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-26 00:45:00 +02:00
Valentin Gagarin
026c24e378 add example for store path using the given name 2023-09-26 00:45:00 +02:00
Valentin Gagarin
d621dd17f2 more precise wording
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-26 00:45:00 +02:00
Valentin Gagarin
e2f118efed restructure and reword input attributes section on derivation 2023-09-26 00:43:58 +02:00
Valentin Gagarin
541890463d make separate section for builder execution 2023-09-26 00:43:58 +02:00
Valentin Gagarin
cba53b3a18 reformat for readability 2023-09-26 00:42:23 +02:00
John Ericson
728767db03
Merge pull request #9028 from Ericson2314/nix3-config-options
Misc options rendering adjustments
2023-09-25 09:04:56 -04:00
John Ericson
4606a07bb6
generate-manpage.nix: Add comment explaining one bit 2023-09-25 08:20:39 -04:00
Maximilian Bosch
89e5e68799
doc/hacking: fix make target to build the docs (#9033)
Was confused why `make html` didn't work while working on #9032, but
then I realized that after this section was written, the target was
renamed to `manual-html` in 6910f5dcb6.
2023-09-24 20:28:34 +02:00
John Ericson
9f93972c4d manual / manpages: Make option category names a proper subheader
Before they were an "ad-hoc" header with bold and a colon; now they are
a proper subheader.

For the man pages, this doesn't make much of a difference, but it will
help more on for the HTML manual, where things can be restyled. Again,
good separation of content vs presentation.
2023-09-23 00:37:44 -04:00
John Ericson
1d9fd3a6f8 manual / manpages: Adjust option filter filtering, move from C++ to Nix
Behavior change:

Before we only showed uption if the command-specific options were
non-empty. But that is somewhat odd since we also show common options.
Now, we do everything based on the union of both sorts of options (with
hidden-categories filtered, as before).

Implementation change:

The JSON dumping once again includes all options; the filtering of
hidden categories is done in the Nix instead. This is better separation
of "content" vs "presentation", and prepare the way for the HTML manual
vs manpages / `--help` doing different things.
2023-09-23 00:34:51 -04:00
John Ericson
9c640c1229 doc: showOptions: Simplify code with builtins.groupBy
This makes grouping options by category much nicer. No behavior should
be changed.
2023-09-23 00:28:16 -04:00
John Ericson
694810ba34 doc: showOptions: Move union to caller
`showOptions` itself doesn't care, so it shouldn't take two separate
arguments.
2023-09-22 23:54:04 -04:00
Théophane Hufschmitt
f89b84919c
Merge pull request #8931 from fricklerhandwerk/nix3-config-options
do not show configuration override flags for each command
2023-09-22 14:13:51 +02:00
Rasmus Rendal
3411507696 Document the percent-encoding mechanism 2023-09-22 10:07:14 +02:00
Valentin Gagarin
1b560ea502
more detail on backports
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-09-21 23:23:14 +02:00
John Ericson
6b6cfec8a5
Merge pull request #8990 from fricklerhandwerk/example-nix-env-install-outputs
add clarifying example to `nix-env` output selection
2023-09-21 12:27:35 -04:00
John Ericson
0273ad73fa
Merge pull request #9012 from fricklerhandwerk/fix-multiline-doc
fix rendering error for consecutive spaces
2023-09-21 11:57:39 -04:00
John Ericson
7bafcdba79
Merge pull request #9013 from fricklerhandwerk/test-coverage-docs
Move test coverage section to testing page
2023-09-21 11:50:04 -04:00
Valentin Gagarin
cf6ba7256f delete unused files 2023-09-21 02:57:00 +02:00
Valentin Gagarin
02649d247b move test coverage section to testing page 2023-09-21 02:52:28 +02:00
Valentin Gagarin
954890a42f add information on release cycle and backports 2023-09-21 02:52:05 +02:00
Valentin Gagarin
747b2baf21 fix rendering error for consecutive spaces 2023-09-21 02:51:54 +02:00
Eelco Dolstra
10ad052f7d Release notes 2023-09-20 11:42:49 +02:00
Valentin Gagarin
408055a9dd add clarifying example to nix-env output selection
there is a very confusing warning in the Nixpkgs manual that
mischaracterises `nix-env` behavior, and this example shows what's
really happening.

note that it doesn't use `pkgs.runCommand` or other `pkgs.stdenv`
facilities, as deep down those set `meta.outputsToInstall` to very
particular defaults that do not generally apply to Nix.
2023-09-18 20:11:42 +02:00
Emil Nikolov
2cdc9c32e7
docs: fixed the default priority of nix-env --install (#8945) 2023-09-09 06:54:39 +00:00
John Ericson
7ad66cb3ef Allow dynamic derivation deps in inputDrvs
We use the same nested map representation we used for goals, again in
order to save space. We might someday want to combine with `inputDrvs`,
by doing `V = bool` instead of `V = std::set<OutputName>`, but we are
not doing that yet for sake of a smaller diff.

The ATerm format for Derivations also needs to be extended, in addition
to the in-memory format. To accomodate this, we added a new basic
versioning scheme, so old versions of Nix will get nice errors. (And
going forward, if the ATerm format changes again the errors will be even
better.)

`parsedStrings`, an internal function used as part of parsing
derivations in A-Term format, used to consume the final `]` but expect
the initial `[` to already be consumed. This made for what looked like
unbalanced brackets at callsites, which was confusing. Now it consumes
both which is hopefully less confusing.

As part of testing, we also created a unit test for the A-Term format for
regular non-experimental derivations too.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

Apply suggestions from code review

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-07 10:39:37 -04:00
Valentin Gagarin
aa46f536e8 add note on overriding settings for stable commands 2023-09-07 02:02:27 +02:00
Valentin Gagarin
09eb7f1ef6 do not show configuration override flags for each command
this removes a lot of noise from the web search, which precludes finding
the actual documentation.

some configuration settings have enough documentation to warrant
individual pages, so the alternative of including full setting
documentation in each command page doesn't make much sense here.

this change technically means that the command line flags to override
settings are "invisible", and not exported as JSON. this may or may not
be desirable. a more explicit approach would be adding a `hidden` field
to the flag's JSON output, but would also require adjusting
post-processing of that JSON for manual rendering.
2023-09-07 02:02:27 +02:00
Valentin Gagarin
b0fe7f560d add missing link 2023-09-07 01:38:53 +02:00
Valentin Gagarin
589fd897fb add todo on store docs 2023-09-07 01:38:53 +02:00
John Ericson
3a62651bd6
Merge pull request #8933 from fricklerhandwerk/option-anchors
Add anchors to option listings
2023-09-06 09:39:33 -04:00
John Ericson
216c66ddcf
Merge pull request #8932 from fricklerhandwerk/formatting
dedent common options listing; one sentence per line
2023-09-06 09:30:25 -04:00
John Ericson
754528241a
Merge pull request #8927 from obsidiansystems/test-derivation-aterm
Test and begin documentation of the ATerm format for derivations
2023-09-06 09:24:52 -04:00
John Ericson
d568877eab
Retitle section as Robert suggests
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-06 08:43:16 -04:00
Valentin Gagarin
391f18063c add anchors to option listings 2023-09-06 09:52:56 +02:00
Valentin Gagarin
02c2679f0e dedent common options listing; one sentence per line
this is a pure reformatting, contents were not changed

one sentence per line makes reviewing diffs and making suggestions much
more convenient. the indentation was an artifat of the DocBook
migration.
2023-09-06 09:49:00 +02:00
Valentin Gagarin
5c23d3a90c disambiguate output from output path 2023-09-06 09:09:02 +02:00
John Ericson
880d9cabed Test and begin documentation of the ATerm format for derivations
Wanted to do this before the last dynamic derivations PR when I
introduce a variation, to make sure I wasn't changing the old version by
mistake.
2023-09-05 11:16:39 -04:00
John Ericson
7ff43435f9 Unit test some worker protocol serializers
Continue with the characterization testing idioms begun in
c70484454f, but this time for unit tests.

Co-authored-by: Andreas Rammhold <andreas@rammhold.de>
2023-09-05 10:48:02 -04:00
Valentin Gagarin
f34484d18a
Merge pull request #7592 from fricklerhandwerk/nix-store-realise
reword description of how realisation works
2023-09-05 11:55:01 +02:00
Eelco Dolstra
919781cacc
Merge branch 'master' into valid_deriver_2 2023-09-01 13:35:05 +02:00
Valentin Gagarin
1ac181759d revert some random change 2023-08-31 21:27:13 +02:00
Valentin Gagarin
d38a539437 make description open-ended, add TODO
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-31 21:27:11 +02:00
Valentin Gagarin
894cbe43bc don't invent terms yet 2023-08-31 21:26:46 +02:00
Valentin Gagarin
b951e862d0 more meaningful tagline 2023-08-31 21:26:46 +02:00
Valentin Gagarin
cf4e14d58d accommodate "do nothing" branch 2023-08-31 21:26:46 +02:00
Valentin Gagarin
d460dbdd30 be more precise about substituting store derivations
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-31 21:26:46 +02:00
Valentin Gagarin
6b3320ab05 mention remote builders
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-31 21:26:44 +02:00
Valentin Gagarin
0cd8f36644 add anchor to builder 2023-08-31 21:26:15 +02:00
Valentin Gagarin
d50f116421 add reference link 2023-08-31 21:25:39 +02:00
Valentin Gagarin
b7e9e29605 remove abstract description 2023-08-31 21:25:39 +02:00
Valentin Gagarin
a57e0e8c5c reword introductory sentence 2023-08-31 21:25:39 +02:00
Valentin Gagarin
315a11bcc9 remove superfluous word 2023-08-31 21:25:39 +02:00
Valentin Gagarin
1bc9257d7c reword description of how realisation works 2023-08-31 21:25:39 +02:00
Cole Helbling
46478b44ff
docs/testing: point out the existence of GTEST_FILTER (#8883) 2023-08-29 22:13:35 +00:00
Eelco Dolstra
5b5f56a9d4
Merge pull request #8859 from edolstra/tarball-last-modified
Tarball trees: Propagate lastModified
2023-08-29 17:02:06 +02:00
Eelco Dolstra
56763ff918 Document that redirected tarball flakerefs can specify lastModified 2023-08-29 16:08:29 +02:00
Silvan Mosberger
151120a1ae
Document nix-prefetch-url defaults (#8878) 2023-08-28 22:14:01 +02:00
Valentin Gagarin
880fef9cdf do not change existing release notes 2023-08-28 20:52:30 +02:00
tomberek
b563ef38cc
Merge pull request #8819 from VertexA115/fix/deep-follow-paths
Fix follow path checking at depths greater than 2
2023-08-25 10:33:05 -04:00
Guillaume Girol
925a444b92 add nix-store --query --valid-derivers command
notably useful when nix-store --query --deriver returns a non-existing
path.

Co-authored-by: Felix Uhl <iFreilicht@users.noreply.github.com>
2023-08-24 11:37:24 +02:00
Valentin Gagarin
d5b130ef13 glossary: dedent list and do not use forced line breaks
this makes it slightly easier to work with and consistent with all the
other markdown lists in use
2023-08-24 10:00:17 +02:00
Uri Zafrir
4a435ad228
Add introductory sentence to advanced topics (#8861) 2023-08-23 15:18:25 +00:00
Tom Bereknyei
c609be4072 doc: explain the . attrPath prefix notation 2023-08-19 17:19:52 -04:00
Alex Zero
37a509ca2d Add release notes for the previous commit 2023-08-14 18:56:02 +01:00
John Ericson
44c8d83831 Create outputOf primop.
In the Nix language, given a drv path, we should be able to construct
another string referencing to one of its output. We can do this today
with `(import drvPath).output`, but this only works for derivations we
already have.

With dynamic derivations, however, that doesn't work well because the
`drvPath` isn't yet built: importing it like would need to trigger IFD,
when the whole point of this feature is to do "dynamic build graph"
without IFD!

Instead, what we want to do is create a placeholder value with the right
string context to refer to the output of the as-yet unbuilt derivation.
A new primop in the language, analogous to `builtins.placeholder` can be
used to create one. This will achieve all the right properties. The
placeholder machinery also will match out the `outPath` attribute for CA
derivations works.

In 60b7121d2c we added that type of
placeholder, and the derived path and string holder changes necessary to
support it. Then in the previous commit we cleaned up the code
(inspiration finally hit me!) to deduplicate the code and expose exactly
what we need. Now, we can wire up the primop trivally!

Part of RFC 92: dynamic derivations (tracking issue #6316)

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-14 09:37:37 -04:00
John Ericson
60b7121d2c Make the Derived Path family of types inductive for dynamic derivations
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).

To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.

`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!

Important note: some JSON formats have changed.

We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-08-10 00:08:32 -04:00
Théophane Hufschmitt
ad410abbe0 Stabilize discard-references
It has been there for a few releases now (landed in 2.14.0), doesn't
seem to cause any major issue and is wanted in a few places
(https://github.com/NixOS/nix/pull/7087#issuecomment-1544471346).
2023-08-07 16:53:37 +02:00
John Ericson
9113b4252b
Merge pull request #8760 from iFreilicht/fix-json-load-assertion-errors
Fix derivation load assertion errors
2023-08-06 17:07:43 -07:00
Felix Uhl
3fefc2b284 Fix derivation load assertion errors
When loading a derivation from a JSON, malformed input would trigger
cryptic "assertion failed" errors. Simply replacing calls to `operator []`
with calls to `.at()` was not enough, as this would cause json.execptions
to be printed verbatim.

Display nice error messages instead and give some indication where the
error happened.

*Before:*

```
$ echo 4 | nix derivation add
error: [json.exception.type_error.305] cannot use operator[] with a string argument with number

$ nix derivation show nixpkgs#hello | nix derivation add
Assertion failed: (it != m_value.object->end()), function operator[], file /nix/store/8h9pxgq1776ns6qi5arx08ifgnhmgl22-nlohmann_json-3.11.2/include/nlohmann/json.hpp, line 2135.

$ nix derivation show nixpkgs#hello | jq '.[] | .name = 5' | nix derivation add
error: [json.exception.type_error.302] type must be string, but is object

$ nix derivation show nixpkgs#hello | jq '.[] | .outputs = { out: "/nix/store/8j3f8j-hello" }' | nix derivation add
error: [json.exception.type_error.302] type must be object, but is string

```

*After:*

```
$ echo 4 | nix derivation add
error: Expected JSON of derivation to be of type 'object', but it is of type 'number'

$ nix derivation show nixpkgs#hello | nix derivation add
error: Expected JSON object to contain key 'name' but it doesn't

$ nix derivation show nixpkgs#hello | jq '.[] | .name = 5' | nix derivation add
error: Expected JSON value to be of type 'string' but it is of type 'number'

$ nix derivation show nixpkgs#hello | jq '.[] | .outputs = { out: "/nix/store/8j3f8j-hello" }' | nix derivation add
error:
       … while reading key 'outputs'

       error: Expected JSON value to be of type 'object' but it is of type 'string'
```
2023-08-05 01:34:30 +02:00
John Ericson
3b592c880a Add infra for experimental store implemenations
This is analogous to that for experimental settings and flags that we
have also added as of late.
2023-08-02 15:46:38 -04:00
Robert Hensing
33d58a90c2 toJSON: Add attribute path to trace 2023-07-31 13:02:54 +02:00
Alex Ameen
2d1d81114d
Add parseFlakeRef and flakeRefToString builtins (#8670)
Over the last year or so I've run into several use cases where I need to
parse and/or serialize URLs for use by `builtins.fetchTree` or
`builtins.getFlake`, largely in order to produce _lockfile-like_ files
for lang2nix frameworks or tools which use `nix` internally to drive
builds.

I've gone through the painstaking process of emulating
`nix::FlakeRef::fromAttrs` and `nix::parseFlakeRef` several times with
mixed success; but these are difficult to create and even harder to
maintain if I hope to stay aligned with changes to the real
parser/serializer.

I understand why adding new `builtins` isn't something we want to do
flagrantly. I'm recommending this addition simply because I keep
encountering use cases where I need to parse/serialize these URIs in
`nix` expressions, and I want a reliable solution.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-07-25 17:43:33 +00:00
Naïm Favier
1b756e300f
doc: clarify release notes about nested attribute merges 2023-07-25 16:09:56 +02:00
Eelco Dolstra
c51be0345e Release notes 2023-07-24 17:19:31 +02:00
Naïm Favier
570a1a3ad7
parser: merge nested dynamic attributes
Fixes https://github.com/NixOS/nix/issues/7115
2023-07-21 17:14:03 +02:00
Valentin Gagarin
85d0eb6316
fix broken links (#8722) 2023-07-20 17:58:14 +02:00
Valentin Gagarin
0e4f6dfcf7 revert anchor prefix for builtin constants
the original change broke many pre-existing anchor links.

also change formatting of the constants listing slightly:
- the type should not be part of the anchor
- add highlight to the "impure only" note
2023-07-20 10:27:38 +02:00
John Ericson
453c4be93c
Merge pull request #8680 from NixLayeredStore/test-groups
Introduce notion of a test group, use for CA tests
2023-07-19 11:17:57 -04:00
Valentin Gagarin
b0173716f6
clarify wording on args@ default handling (#8596)
* clarify wording on args@ default handling

Most importantly use shorter sentences and emphasize the key point that defaults aren't taken into account

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-07-19 13:07:07 +00:00
Robert Hensing
32494cbb29
Merge pull request #7973 from fricklerhandwerk/remove-channels
remove the Channels section
2023-07-19 14:02:26 +02:00
Valentin Gagarin
2fa90e5824 add more details on CA derivations 2023-07-19 13:59:18 +02:00
Valentin Gagarin
5f37ebcf83 document all special system features and their behavior 2023-07-19 13:34:03 +02:00
Valentin Gagarin
6c3cd429a6 fix broken links 2023-07-19 11:01:48 +02:00
Valentin Gagarin
1a220bed93 do not mention output attributes
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-07-19 10:38:12 +02:00
Valentin Gagarin
c8f04e2024 note that naming convention is from Autotools
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-07-19 10:37:40 +02:00
Valentin Gagarin
4944e37ec0 expand on the system type in hacking guide 2023-07-19 10:37:40 +02:00
Valentin Gagarin
e14c8a359e list moving parts of channels 2023-07-19 10:26:25 +02:00
Valentin Gagarin
4bab5a6208 revert channel files overview 2023-07-19 09:42:53 +02:00
Valentin Gagarin
cd0e39bd89 remove redundant information from channel profile description 2023-07-19 09:39:04 +02:00
Valentin Gagarin
ee72ede389 remove the Channels section
this is a how-to guide which should not be in the reference manual.
it also refers to `nix-env`, which should not be the first thing readers
of the reference manual encounter, as it behaves very differently in
spirit from the rest of Nix.

slightly reword the documentation to be more concise and informative.
2023-07-19 09:39:04 +02:00
John Ericson
259e328de8 Introduce notion of a test group, use for CA tests
Grouping our tests should make it easier to understand the intent than
one long poorly-arranged list. It also is convenient for running just
the tests for a specific component when working on that component.

We need at least one test group so this isn't dead code; I decided to
collect the tests for the `ca-derivations` and `dynamic-derivations`
experimental features in groups. Do
```bash
make ca.test-group -jN
```
and
```bash
make dyn-drv.test-group -jN
```
to try running just them.

I originally did this as part of #8397 for being able to just the local
overlay store alone. I am PRing it separately now so we can separate
general infra from new features.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-07-18 09:31:13 -04:00
Sinan Mohd
a5c88f8609
Nix Reference Manual: keep nix expressions uptodate with nixpkgs (#8703) 2023-07-16 09:25:11 +00:00
Mathnerd314
c70484454f Expanded test suite
* Lang now verifies errors and parse output

* Some new miscellaneous tests

* Easy way to update the tests

* Document workflow in manual

* Use `!` not `~` as separater char for sed

  It is confusing to use `~` when we are talking about paths and home
  directories!

* Test test suite itself (`test/lang-test/infra.sh`)

Additionally, run shellcheck on `tests/lang.sh` to help ensure it is
correct, now that is is more complex.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-07-11 21:43:09 -04:00
Robert Hensing
40052c7613 fetchClosure: Docs and error message improvements
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-30 18:23:42 +02:00
Robert Hensing
50de11d662 doc: Improve fetchClosure documentation 2023-06-30 18:23:24 +02:00
Robert Hensing
32c69e2b17 doc: Typo 2023-06-30 18:22:47 +02:00
Eelco Dolstra
a0c617348b
Merge pull request #8589 from jfroche/sign-paths-as-allowed-user
Allow to sign path as unprivileged user
2023-06-30 13:13:42 +02:00
John Ericson
ca49e13414 Split testing into its own page in the contribution guide
`hacking.md` has gotten really big!
2023-06-27 18:27:49 -04:00
John Ericson
2ccc02515f Trailing commas in redirects
This avoids diff noise when more are added. Unlike with JSON, this is
allowed in JS.
2023-06-27 18:23:06 -04:00
Jean-François Roche
80c9259756 Allow to sign path as unprivileged user
User can now sign path as unprivileged/allowed user

refs #1708
2023-06-27 18:31:31 +02:00
John Ericson
22b278e011 Automatically document builtin constants
This is done in roughly the same way builtin functions are documented.

Also auto-link experimental features for primops, subsuming PR #8371.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-27 09:37:54 -04:00
John Ericson
d40f0e534d Don't say this when we still pollute the global scope
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-06-27 09:37:31 -04:00
John Ericson
4da7c86618 Switch example to a primop this is less ill-advised
Any primop will do for this, so might as well use one that isn't impure.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-06-27 09:34:36 -04:00
John Ericson
97df060588 Better document build failure exit codes
- Improved API docs from comment

- Exit codes are for `nix-build`, not just `nix-store --release`

- Make note in tests so the magic numbers are not surprising

Picking up where #8387 left off.
2023-06-22 14:29:45 -04:00
John Ericson
78e1e3c753
Merge pull request #8565 from obsidiansystems/profile-delete-docs
Clarify docs on deleting generations, including fixing a mistake
2023-06-22 10:27:40 -04:00
Valentin Gagarin
0ab962d83f
Merge pull request #8556 from fricklerhandwerk/hacking-headings
hacking guide: use more self-descriptive section headings
2023-06-22 13:45:31 +02:00
John Ericson
5cc22e3370 Clarify docs on deleting generations, including fixing a mistake
Deleting store info corrected (there is a foot-gun in Nix with
`--delete-generations old`!)

Also a few things are cleaned up based on feedback.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2023-06-21 20:05:41 -04:00
Valentin Gagarin
5f9a921bc1
do not use "target", as it's a loaded term in the domain of compilers
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-06-21 14:31:09 +02:00
Valentin Gagarin
085104944b add redirects to changed anchors 2023-06-21 09:46:23 +02:00
Valentin Gagarin
a78f929065 fix anchor link 2023-06-21 09:43:22 +02:00
Valentin Gagarin
71317162c5 use more self-descriptive section headings 2023-06-20 13:44:43 +02:00
Valentin Gagarin
3c618c43c6
Merge pull request #8532 from fricklerhandwerk/nix.conf-sections
split nix.conf man page into sections
2023-06-20 13:16:14 +02:00
Valentin Gagarin
68c6219343 clarify setting options on the command line 2023-06-20 12:13:32 +02:00
Valentin Gagarin
bc7324e912 clarify read order for configuration settings 2023-06-20 12:13:26 +02:00
fricklerhandwerk
38bd1cc9bc split configuration file page into sections
this makes it easier to scan for specific information, such as the
format of command line flags
2023-06-20 12:11:01 +02:00
fricklerhandwerk
f2b54e3b71 add links to environment variables documentation 2023-06-20 12:11:01 +02:00
Travis A. Everett
33d3889831 redirect old platform uninstall instruction links
Uninstall instructions were moved to their own page in #8267. The
overall section link was redirected in #8286, but platform-specific
links (which I give out frequently when I triage installer trouble)
weren't included.
2023-06-19 15:17:50 -05:00
Silvan Mosberger
3910430b9d
Add more links in nix-build documentation (#8545)
* Add more links in nix-build documentation


Co-authored-by: John Ericson <git@JohnEricson.me>
2023-06-19 21:00:49 +02:00
John Ericson
469d06f9bc Split out worker protocol template definitions from declarations
This is generally a fine practice: Putting implementations in headers
makes them harder to read and slows compilation. Unfortunately it is
necessary for templates, but we can ameliorate that by putting them in a
separate header. Only files which need to instantiate those templates
will need to include the header with the implementation; the rest can
just include the declaration.

This is now documenting in the contributing guide.

Also, it just happens that these polymorphic serializers are the
protocol agnostic ones. (Worker and serve protocol have the same logic
for these container types.) This means by doing this general template
cleanup, we are also getting a head start on better indicating which
code is protocol-specific and which code is shared between protocols.
2023-06-19 11:45:59 -04:00
Valentin Gagarin
7bf17f8825
Add description for file system objects (#8500)
While this is not actually a notion in the implementation, it is
explicitly described in the thesis and quite important for understanding
how the store works.

Co-authored-by: John Ericson <git@JohnEricson.me>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-06-18 23:45:08 -04:00
John Ericson
60d81b5163
Merge pull request #8472 from NixOS/nix-language-purpose
Refine wording on the purpose of the Nix language
2023-06-18 23:38:05 -04:00
Christina Sørensen
741f7837f8
Fix wikipedia links (#8533) 2023-06-17 09:06:17 +00:00
Eelco Dolstra
e503eadafc
Merge pull request #8477 from edolstra/tarball-flake-redirects
Tarball flake improvements
2023-06-16 18:03:50 +02:00
Eelco Dolstra
b1ed9b4b0c
Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-06-16 16:48:37 +02:00
Eelco Dolstra
cab03fb779 Add docs 2023-06-16 15:58:42 +02:00
Valentin Gagarin
c3c4076342 make domain-specificity more specific
also slightly reword the purpose statement to introduce (and explain)
derivations right away.
2023-06-15 03:08:07 +02:00
John Ericson
946cd9e3f9
Merge pull request #8351 from obsidiansystems/delete-profiles-tests-docs
Expanding tests and docs relating to deleting profiles
2023-06-15 01:47:21 +02:00
John Ericson
5b7e285727 Improve nix-collect-garbage docs
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-14 19:01:08 -04:00
John Ericson
b55f26c65f Improve nix-env --delete-generations docs
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-14 19:01:08 -04:00
Valentin Gagarin
a1cf16563f
Fixup description of substituters (#8291)
Introduce what substituters "are" in the configuration option entry.
Remove arbitrary line breaks for easier editing in the future.
Link glossary some more.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-06-14 20:49:58 +00:00
John Ericson
37bffbc28f
Merge pull request #8488 from Mic92/update-requirements
nix actually needs c++20 now
2023-06-14 20:37:52 +02:00
John Ericson
ff905cb796
Merge pull request #4803 from ShamrockLee/nix-channel-list-generations
Add `nix-channel --list-generations`
2023-06-14 18:30:35 +02:00
Jörg Thalheim
c51f3f1eb2 nix actually needs c++20 now 2023-06-10 13:56:05 +02:00
Valentin Gagarin
db680e0e57
refine wording on the purpose of the Nix language
packages and configurations are not really a concept in Nix or the Nix language. the idea of transforming files into other files clearly captures what it's all about, and the new phrasing should make the term "derivation" more obvious both in terms of meaning and origin.
2023-06-08 02:00:05 +02:00
Théophane Hufschmitt
527eb4a99a
Merge pull request #8317 from fricklerhandwerk/doc-identifier
document identifier syntax for attribute sets
2023-06-02 13:10:27 +02:00
Théophane Hufschmitt
331f0967c4 Add a release note for nix-channel --list-generations 2023-06-02 10:14:53 +02:00
Shamrock Lee
b292177eec Add nix-channel --list-generations
Add support to --list-generations
as another way to say
nix-env --profile /nix/var/nix/profiles/per-user/$USER/channels --list-generations
the way we did for nix-channel --rollback [generation id]
2023-06-02 10:12:35 +02:00
Eelco Dolstra
008f89fa50 Typo 2023-05-31 14:05:00 +02:00
Eelco Dolstra
afc24e6a66 Release notes 2023-05-31 12:38:05 +02:00
Eelco Dolstra
9a5b9c588f
Merge pull request #8421 from fricklerhandwerk/doc-replaceStrings
update documentation according to release notes
2023-05-31 12:10:32 +02:00
Valentin Gagarin
738c0d5064
Merge pull request #8318 from fricklerhandwerk/doc-currentTime
document `builtins.currentTime`
2023-05-31 03:15:54 +02:00
Valentin Gagarin
52004696c0
Merge pull request #8413 from doronbehar/doc/distributed-builds/clearer-warning
distributed-builds.md: Clarify warning ssh access requirements
2023-05-31 02:05:05 +02:00
Valentin Gagarin
e2f33e0df1 rewrap lines to use one line per sentence
this makes future reviews easier as it reduces diff noise
2023-05-31 01:32:03 +02:00
Doron Behar
76287a9207 distributed-builds.md: Clarify warning ssh access requirements 2023-05-31 01:28:15 +02:00
Valentin Gagarin
7a5731aa07 link to mentioned builtin 2023-05-30 22:58:00 +02:00
polykernel
a382919d7d
primops: lazy evaluation of replaceStrings replacements
The primop `builtins.replaceStrings` currently always strictly evaluates the
replacement strings, however time and space are wasted for their computation
if the corresponding pattern do not occur in the input string. This commit
makes the evaluation of the replacement strings lazy by deferring their
evaluation to when the corresponding pattern are matched and memoize the result
for efficient retrieval on subsequent matches.

The testcases for replaceStrings was updated to check for lazy evaluation
of the replacements. A note was also added in the release notes to
document the behavior change.
2023-05-25 18:35:23 -04:00
Bernardo Meurer
bf693319f6
feat: add always-allow-substitutes
This adds a new configuration option to Nix, `always-allow-substitutes`,
whose effect is simple: it causes the `allowSubstitutes` attribute in
derivations to be ignored, and for substituters to always be used.

This is extremely valuable for users of Nix in CI, where usually
`nix-build-uncached` is used. There, derivations which disallow
substitutes cause headaches as the inputs for building already-cached
derivations need to be fetched to spuriously rebuild some simple text
file.

This option should be a good middle-ground, since it doesn't imply
rebuilding the world, such as the approach I took in
https://github.com/NixOS/nixpkgs/pull/221048
2023-05-22 19:42:29 +01:00
John Ericson
684e9be8b9
Merge pull request #8337 from fricklerhandwerk/channel-files
list files used by `nix-channel` on its own man page
2023-05-17 12:21:20 -04:00
John Ericson
16a60520f2
Merge pull request #8358 from fricklerhandwerk/output-addressed-references
fix wording on output-addressed store objects
2023-05-17 09:38:18 -04:00
Valentin Gagarin
76f40c02bf fix wording on output-addressed store objects
hashing is an implementation detail.
add references to the other terms.
2023-05-17 15:04:08 +02:00
Valentin Gagarin
6a5a8f51bb add cross-references to pure evaluation mode
use consistent wording everywhere.
add some details on the configuration option documentation.
2023-05-17 15:01:54 +02:00
Valentin Gagarin
549f8dc97b document builtins.currentTime 2023-05-17 13:49:35 +02:00
Valentin Gagarin
d8bfeda164 document identifier syntax for attribute sets
it's more likely for readers to find it right there.

this also slightly rewords examples to make them stand out better.
in the long run there probably needs to be a dedicated section on formal syntax, and better highlighting of examples.
2023-05-17 13:41:34 +02:00
Alexander Schmolck
8d4b6766e2 Convert short nix options to long ones
e.g. nix-env -e subversion => nix-env --uninstall subversion

The aim is to make the documentation less cryptic for newcomers and the
long options are more self-documenting.

The change was made with the following script:

<https://github.com/aschmolck/convert-short-nix-opts-to-long-ones>

and sanity checked visually.
2023-05-17 08:10:30 +01:00
Valentin Gagarin
8976769a1c
Reword info on XDG base dirs (#8338)
This gives some more context and should clarify why it works that way.
Also link it from the section on `NIX_USER_CONF_FILES`.

Co-authored-by: John Ericson <git@JohnEricson.me>
2023-05-15 23:30:20 -04:00
Valentin Gagarin
275468a1f0 list files used by nix-channel 2023-05-15 15:26:14 +02:00
John Ericson
914672dc4f
Merge pull request #8141 from tweag/user-files-doc
Document user files of nix
2023-05-15 07:11:47 -04:00
Victor Engmark
6e1bfb93dc refactor: Join commands to remove files 2023-05-15 11:05:26 +12:00
Victor Engmark
ed016a5bb0 docs: Mention more files referenced by the installer
`/etc/bash.bashrc` is backed up as `/etc/bash.bashrc.backup-before-nix`,
but since other changes might have been introduced in the meantime we can't
just tell the user to revert.
2023-05-15 11:03:27 +12:00
Victor Engmark
e26aad22c6 docs: Remove references to non-existing files
At least on Ubuntu 22.04, these files are not created as part of a multi-
user installation.
2023-05-15 10:26:47 +12:00
Victor Engmark
17a1c1ee4e docs: Sort files by name 2023-05-15 10:16:12 +12:00
Victor Engmark
3cc9b8630b docs: Remove Nix profile content from correct directory
At least on Ubuntu 22.04, the Nix installer creates
`/etc/profile.d/nix.sh`, not `/etc/profile/nix.sh`.
2023-05-15 10:13:46 +12:00
Victor Engmark
ccf512f4b8 docs: Remove redundant uninstall command
`sudo systemctl disable nix-daemon.socket nix-daemon.service` removes these
files already.
2023-05-15 10:06:27 +12:00
Valentin Gagarin
454e273451
Merge pull request #8328 from urbas/max-substitution-jobs 2023-05-14 11:59:40 +02:00
Matej Urbas
c66a7af0c6 max-substitution-jobs release note entry 2023-05-14 09:51:11 +01:00
Ana Hobden
17fb346ec6 Describe the or operation on attribute sets slightly more 2023-05-12 11:03:09 -07:00
Alexander Bantyev
992be330ab
Update doc/manual/src/SUMMARY.md.in
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-05-12 16:58:01 +04:00
Valentin Gagarin
70eb9c7ddb reword introduction to built-in functions
add anchor to `builtins.derivation` and list some built-in functions that are
exposed in the global scope.

I decided not to list everything, because we probably don't want to
encourage people using them that way.
2023-05-11 20:07:42 +02:00
Valentin Gagarin
69a0e6f86c reword documentation on builtins
- add anchor to `builtins`
- add type information
- reword description of `builtins` to offer more information concisely
2023-05-11 17:46:37 +02:00
Valentin Gagarin
2ef7f14e28 do not indent markdown list 2023-05-11 15:29:03 +02:00
Valentin Gagarin
dc8191ae14 add redirect to track moved uninstall section 2023-05-03 11:39:29 +02:00
Valentin Gagarin
5d78dc4176
doc rendering: add functions to scope explicitly (#7378)
* doc rendering: add functions to scope explicitly

this especially helps beginners with code readability, since the origin
of names is always immediately visible.
2023-05-03 07:16:29 +00:00
Valentin Gagarin
f8620758aa display documentation on manifest files separately
it's probably better not to show the manifest file documentation in the
command-specific pages, because these are implementation details that are not really practically useful.

this means no additional hassle for building the manual, but clutters
the table of contents a bit.
2023-04-28 12:10:36 +02:00
Valentin Gagarin
9b2a4a4729 move uninstall instructions to a separate page
placed in a subsection of the binary install, the instructions are hard
to find. putting them in a separate page that is shown in the table of
contents should make it easier for users to find what they need when
they need it.
2023-04-28 11:53:38 +02:00
Valentin Gagarin
da0dbf36cb move manifest information to the bottom of the page 2023-04-26 15:39:35 +02:00
Valentin Gagarin
be7c236565 move compatibility info to nix profile docs 2023-04-26 15:39:35 +02:00
Valentin Gagarin
219aaf5a0b increase heading level
otherwise the headings won't match at the point where they are included.
this is a bit hacky and brittle, but works for now.
2023-04-26 15:39:35 +02:00
Valentin Gagarin
bb8e3b5d86 update description on how profiles work
adapt to the example listing
2023-04-26 15:39:35 +02:00
Valentin Gagarin
7770d82240 fix typos and wording 2023-04-26 15:39:24 +02:00
Valentin Gagarin
9d386fe2ee add colons to connect listings with descriptions 2023-04-26 15:39:16 +02:00
Valentin Gagarin
a1c996dc7e list information regular users first
this is to make it consistent everywhere
2023-04-26 15:39:02 +02:00
Alexander Bantyev
8a93b5a551 Document user files of nix 2023-04-26 15:38:19 +02:00
Rynn Blackmon
c4df53f154
Mention $DRV_PATH in post-build-hook docs 2023-04-19 16:32:47 -07:00
Michael Utz
40fcb22313
Update installing-binary.md 2023-04-18 13:18:30 +03:00
Noah Snelson
324ed0c367 Documentation: fix typo for Nix database link in manual
Fixes broken link for `Nix database` anchor in the Glossary page of the
Nix manual.
2023-04-17 20:15:08 -07:00
Valentin Gagarin
39e0f609cc
Merge pull request #7514 from fricklerhandwerk/opt-I
remove incorrect reference to `NIX_PATH` documentation
2023-04-17 16:27:23 +02:00
Valentin Gagarin
f4119a67cc
use @docroot@ link 2023-04-17 14:54:30 +02:00
Valentin Gagarin
d0cf615cbb
add link to nix-conf setting
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-04-17 14:27:26 +02:00
John Ericson
9800c1e807 Mark experimental configuration settings programmatically
Fix #8162

The test is changed to compare `nlohmann::json` values, not strings of dumped
JSON, which allows us to format things more nicely.
2023-04-16 10:58:04 -04:00
John Ericson
ba9ae691b6 Add optionalString to manual Nix lang utilities
Use it everywhere it could be also.
2023-04-16 10:57:46 -04:00
John Ericson
b41f739068
Merge pull request #7864 from obsidiansystems/quickstart-long-options
Use long options instead of short ones in the "quick start"
2023-04-14 09:13:16 -04:00
John Ericson
d93e76fbb8 Start cross-referencing experimental features
- Create a glossary entry for experimental features.

- Have the man page experimental feature notice link `nix-commmand`.

  (Eventually this should be programmed, based on whether the command is
  experimental, and if so what experimental feature does it depend on.)

- Document which installables depend on which experimental features.

  I tried to use the same style (bold warning and block quote) that the
  top of the man page uses.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-04-14 07:45:08 -04:00
Eelco Dolstra
e570a91661 Release notes 2023-04-11 12:40:56 +02:00
Eelco Dolstra
60a1bf08b6
Merge pull request #7798 from peeley/list-experimental-features
Documentation: list experimental features in manual
2023-04-11 11:55:35 +02:00
John Ericson
73eb6a2a57 Single page for experimental feature descriptions
As requested by @fricklerhandwerk.
2023-04-09 11:01:23 -04:00
John Ericson
9d1105824f Add release notes for nix derivation {add,show} 2023-04-07 08:35:59 -04:00
John Ericson
2b98af2e62 nix show-derivation -> nix derivation show 2023-04-07 08:34:58 -04:00
matthewcroughan
9207f94582 Add Store::isTrustedClient()
This function returns true or false depending on whether the Nix client
is trusted or not. Mostly relevant when speaking to a remote store with
a daemon.

We include this information in `nix ping store` and `nix doctor`

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2023-04-06 19:59:57 -04:00
John Ericson
bc192a95ef Describe active experimental features in the contributing guide
They are put in the manual separate pages under the new overarching
description of experimental features.

The settings page just lists the valid experimental feature names (so
people know what a valid setting entry looks like), with links to those
pages. It doesn't attempt to describe each experimental feature as that
is too much information for the configuration settings section.
2023-04-06 18:07:59 -04:00
John Ericson
b7cd87a853 Merge remote-tracking branch 'upstream/master' into list-experimental-features 2023-04-06 18:07:46 -04:00
Théophane Hufschmitt
9185639631
Document the concept of “experimental feature” (#5930)
Add a page explaining what “experimental features” are, when and how they should be used

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2023-04-06 11:25:22 +00:00
Valentin Gagarin
b11ae93581 remove incorrect reference
the semantics are not explained in the referenced section any more, they
have been moved to the documentation for common options in the new CLI [0].

[0]: 703d863a48
2023-04-05 14:03:11 +02:00
Valentin Gagarin
bbdb5a58c7
Merge pull request #7849 from milahu/fix-man-nix-shell
docs: fix nix-shell commands
2023-04-05 13:06:59 +02:00
milahu
1ac10808ba docs: fix nix-shell commands 2023-04-05 12:38:36 +02:00
Mike (stew) O'Connor
cab5266092 add a definition of the nix database to the glossary
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-04-05 10:50:34 +02:00
Valentin Gagarin
ed5d0c1362 add anchor to 'local store' 2023-04-05 10:48:59 +02:00
John Ericson
53d0836347 Assemble experimental feature docs outside of Nix itself
Instead of constructing a markdown list in C++ (which involved all sorts
of nasty string literals), export some JSON and assemble it with the
manual build system.

Besides following the precedent set with other dumped data, this is a
better separate of content and presentation; if we decide for example we
want to display this information in a different way, or in a different
section of the manual, it will become much easier to do so.
2023-04-04 22:57:11 -04:00
Valentin Gagarin
cde78f9417 process includes first
otherwise the order of found `.md` files will influence if `@docroot@`
is replaced before them being included, which may mess up relative
links.

the weirdest thing about it is that the mess-up happens
deterministically on macOS, but deterministically doesn't happen on
Linux!
2023-03-30 13:55:44 +02:00
Alexander Bantyev
36b059748d Split nix-env and nix-store documentation per-subcommand
Documentation on "classic" commands with many sub-commands are
notoriously hard to discover due to lack of overview and anchor links.
Additionally the information on common options and environment variables
is not accessible offline in man pages, and therefore often overlooked
by readers.

With this change, each sub-command of nix-store and nix-env gets its
own page in the manual (listed in the table of contents), and each own
man page.

Also, man pages for each subcommand now (again) list common options
and environment variables. While this makes each page quite long and
some common parameters don't apply, this should still make it easier
to navigate as that additional information was not accessible on the
command line at all.

It is now possible to run 'nix-store --<subcommand> --help` to display
help pages for the given subcommand.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-03-30 09:46:28 +02:00
Alexander Bantyev
84c2c09ec2 Manual: fail when #include-d file does not exist 2023-03-30 09:41:46 +02:00
John Ericson
8df6f7bb0f Fix more profile dirs in docs
Picking up where #8078 left off.
2023-03-27 12:10:32 -04:00
Eelco Dolstra
237587bc0a
Merge pull request #8084 from edolstra/store-docs
Auto-generate store documentation
2023-03-27 15:46:18 +02:00
Eelco Dolstra
5a0f5b5c34
Merge pull request #8062 from edolstra/ssl-cert-file
Add a setting for configuring the SSL certificates file
2023-03-27 15:37:15 +02:00
Eelco Dolstra
7a61a9aba3 Update release notes 2023-03-23 15:27:06 +01:00
Eelco Dolstra
05d9918a9c
Update doc/manual/src/release-notes/rl-next.md
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-03-23 10:13:51 +01:00
Eelco Dolstra
4d31618c15 showSetting: Drop ... 2023-03-23 10:09:25 +01:00
Eelco Dolstra
2851e239a7 Make useAnchors a named argument 2023-03-23 10:08:49 +01:00
Eelco Dolstra
fbc296e00f
Merge pull request #8095 from fricklerhandwerk/operators-formatting
doc: force line breaks where needed
2023-03-23 09:24:54 +01:00
Valentin Gagarin
0ced5132fd
Merge pull request #6865 from zuzuleinen/manual-nix-path-empty
document what happens when NIX_PATH is empty
2023-03-23 00:10:17 +01:00
Valentin Gagarin
96129246ad
add links 2023-03-22 23:37:27 +01:00
Valentin Gagarin
40eb88e427 doc: force line breaks where needed 2023-03-22 22:46:23 +01:00
Eelco Dolstra
5691bac202 Improve store setting descriptions / Markdown formatting 2023-03-22 14:23:36 +01:00
Eelco Dolstra
da8903ecc3 nix help-stores: Don't include anchors
Since setting names are not unique we don't have an easy way to
produce unique anchors.
2023-03-22 13:11:35 +01:00
Eelco Dolstra
006291e5be
Merge pull request #8082 from edolstra/add-links
Add links to API docs / coverage analysis to the hacking section
2023-03-22 11:27:18 +01:00
Eelco Dolstra
3fc4659d22 Formatting 2023-03-21 14:12:15 +01:00
Eelco Dolstra
9eb53bbf17 Support per-store Markdown documentation 2023-03-21 14:03:40 +01:00
Eelco Dolstra
7704118d28 nix describe-stores: Remove
This command was intended for docs generation, but it was never used
for that and we don't need it.
2023-03-21 14:03:40 +01:00
Eelco Dolstra
8d6d59cb1b nix store --help: Include store type documentation 2023-03-21 14:03:40 +01:00
Eelco Dolstra
6f62cb3611 generate-manpage.nix: Make more readable 2023-03-21 12:11:32 +01:00
Eelco Dolstra
4b9add9b35 Add links to API docs / coverage analysis to the hacking section 2023-03-21 11:47:21 +01:00
Eelco Dolstra
e53e5c38d4 Add a setting for configuring the SSL certificates file
This provides a platform-independent way to configure the SSL
certificates file in the Nix daemon. Previously we provided
instructions for overriding the environment variable in launchd, but
that obviously doesn't work with systemd. Now we can just tell users
to add

  ssl-cert-file = /etc/ssl/my-certificate-bundle.crt

to their nix.conf.
2023-03-17 18:32:18 +01:00
Yueh-Shun Li
c27d358abb nix-hash: support base-64 and SRI format
Add the --base64 and --sri flags for the Base64 and SRI format output.

Add the --base16 flag to explicitly specify the hexadecimal format.

Add the --to-base64 and --to-sri flag to convert a hash to the above
mentioned format.
2023-03-16 03:08:42 +08:00
Valentin Gagarin
d37b8a29ce
Merge pull request #8036 from Freed-Wu/patch-1 2023-03-13 14:35:09 +01:00
Eelco Dolstra
6f22e8b7d8
Merge pull request #7486 from fricklerhandwerk/doc-reference
reword definition of "reference"
2023-03-13 14:04:14 +01:00
wzy
5bf4ade0fa
Fix a typo of values.md 2023-03-13 15:34:01 +08:00
John Ericson
6910f5dcb6 Generate API docs with Doxygen
The motivation is as stated in issue #7814: even though the the C++ API
is internal and unstable, people still want it to be well documented for
sake of learning, code review, and other purposes that aren't predicated
on it being stable.

Fixes #7814

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-03-10 12:51:06 -05:00
Valentin Gagarin
66f49864f3
Merge pull request #7928 from serokell/doc-includes
Documentation: process `#include` directives
2023-03-10 17:06:42 +01:00
Valentin Gagarin
2af9fd20c6 clarify definition of "installable"
the term was hard to discover, as its definition and explanation were in
a very long document lacking an overview section.
search did not help because it occurs so often.

- clarify wording in the definition
- add an overview of installable types
- add "installable" to glossary
- link to definition from occurrences of the term
- be more precise about where store derivation outputs are processed
- installable Nix expressions must evaluate to a derivation

Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com>
2023-03-05 01:46:17 +01:00
Valentin Gagarin
e065131c1b
cosmetic indentation
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-03-03 12:56:23 +01:00
Théophane Hufschmitt
1f394d2107
Merge branch 'master' into paths-from-stdin 2023-03-02 19:20:51 +01:00
Eelco Dolstra
96111ad902
Merge pull request #7901 from hercules-ci/disable-tests
Add ./configure --disable-tests option
2023-03-02 10:00:15 +01:00
Alexander Bantyev
dd0aab2f94
Documentation: process #include directives 2023-03-01 14:01:45 +04:00
Alexander Bantyev
21fb1a5ec2
doc/manual/local.mk: Use tabs instead of spaces everywhere 2023-03-01 14:01:23 +04:00
Alexander Bantyev
2fc3a15861
Make nix-env less prominent in manual TOC 2023-03-01 13:54:45 +04:00
John Ericson
ea0adfc582 Get rid of .drv special-casing for store path installables
The release notes document the change in behavior, I don't include it
here so there is no risk to it getting out of sync.

> Motivation

>> Plumbing CLI should be simple

Store derivation installations are intended as "plumbing": very simple
utilities for advanced users and scripts, and not what regular users
interact with. (Similarly, regular Git users will use branch and tag
names not explicit hashes for most things.)

The plumbing CLI should prize simplicity over convenience; that is its
raison d'etre. If the user provides a path, we should treat it the same
way not caring what sort of path it is.

>> Scripting

This is especially important for the scripting use-case. when arbitrary
paths are sent to e.g. `nix copy` and the script author wants consistent
behavior regardless of what those store paths are. Otherwise the script
author needs to be careful to filter out `.drv` ones, and then run `nix
copy` again with those paths and `--derivation`. That is not good!

>> Surprisingly low impact

Only two lines in the tests need changing, showing that the impact of
this is pretty light.

Many command, like `nix log` will continue to work with just the
derivation passed as before. This because we used to:

- Special case the drv path and replace it with it's outputs (what this
  gets rid of).

- Turn those output path *back* into the original drv path.

Now we just skip that entire round trip!

> Context

Issue #7261 lays out a broader vision for getting rid of `--derivation`,
and has this as one of its dependencies. But we can do this with or
without that.

`Installable::toDerivations` is changed to handle the case of a
`DerivedPath::Opaque` ending in `.drv`, which is new: it simply doesn't
need to do any extra work in that case. On this basis, commands like
`nix {show-derivation,log} /nix/store/...-foo.drv` still work as before,
as described above.

When testing older daemons, the post-build-hook will be run against the
old CLI, so we need the old version of the post-build-hook to support
that use-case.

Co-authored-by: Travis A. Everett <travis.a.everett@gmail.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-02-28 17:07:05 -05:00
Timothy DeHerrera
639659dec2
doc/manual: add release note for --stdin flag 2023-02-28 12:31:05 -07:00
Timothy DeHerrera
df643051e2
nix-store: read paths from standard input
Resolves #7437 for new `nix-store` by adding a `--stdin` flag.
2023-02-28 12:29:16 -07:00
Robert Hensing
d0d0b9a748 doc/cli-guideline: Improve examples
Turns out that the settings themselves have a bad data model anyway, so we cut that. They do still occur in the first example, but not in focus.
2023-02-28 16:35:47 +01:00
Robert Hensing
17f70b10bf doc/cli-guideline: Apply suggestions from code review
Thanks Valentin!
2023-02-28 16:35:42 +01:00
Robert Hensing
f09ccd8ea9 doc/cli-guideline: Add JSON guideline 2023-02-28 16:35:36 +01:00
Eelco Dolstra
1e07102937 Release notes 2023-02-28 13:44:14 +01:00
Valentin Gagarin
1b49e6fea9
use the term reference correctly 2023-02-27 13:04:54 +01:00
Valentin Gagarin
b0c23999d2
add link to definition 2023-02-27 12:56:22 +01:00
Valentin Gagarin
ec317949f9
remove "references" as distinct term 2023-02-27 12:55:14 +01:00
Robert Hensing
e76619a402 rl-next: Describe fixed flake outPath semantics
The reference documentation already implies the correct semantics.
2023-02-26 14:41:23 +01:00
Robert Hensing
c6051cac6f doc: Add test dependencies to prerequisites 2023-02-24 09:59:25 +01:00
Robert Hensing
df6829e0d7
Merge pull request #7867 from NixOS/hacking-cross
doc/hacking.md: Corrections and additions for cross
2023-02-20 22:25:55 +01:00
Robert Hensing
807ef8f734
doc/hacking.md: Corrections and additions for cross 2023-02-20 12:20:08 +01:00
John Ericson
d7a4f08d42
Nix's own flake: Dedup and memoize more
- `nixpkgsFor` does all of native, static, cross, and the different stdenvs.

- The main Nix derivation is no longer duplicated for static.

- DRY nixpkgs.lib and lib.genAttrs calls.
2023-02-20 11:35:51 +01:00