Commit graph

9634 commits

Author SHA1 Message Date
Eelco Dolstra
d77a39a314 Fix indent 2023-12-21 16:22:34 +01:00
Eelco Dolstra
9d9d9ff0de Merge remote-tracking branch 'origin/master' into profile-names-instead-of-index 2023-12-21 16:21:26 +01:00
Silvan Mosberger
ea454d8687 Undeprecate isNull
There's no good reason to deprecate it:
- For consistency reasons it should continue to exist, such that all
primitive types have a corresponding `builtins.is*` primop.
- There's no implementation cost to continuing to have this function
- It costs users time to try to migrate away from it, e.g.
  https://github.com/NixOS/nixpkgs/pull/219747 and https://github.com/NixOS/nixpkgs/pull/275548
- Using it can give easier-to-read code like `all isNull list`

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-20 17:55:19 +01:00
Eelco Dolstra
7cfd6c0efe
Merge pull request #9325 from NixOS/accessor-add-to-store
Content addressing and adding to store cleanup
2023-12-19 15:10:31 +01:00
Andrew Marshall
7526b7ded6 Allow access to /dev/stderr in Darwin sandbox
We allow /dev/stdout, so why not this? Since it is process-local,
anyway, should not be possible to escape sandbox using it.
2023-12-18 19:33:20 -05:00
John Ericson
ed26b186fb Remove now-redundant text-hashing store methods
`addTextToStore` and `computeStorePathFromDump` are now redundant.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:44:10 -05:00
John Ericson
dfc876531f Organize content addressing, use SourceAccessor with Store::addToStore
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:41:54 -05:00
Théophane Hufschmitt
ebfbc5a6a5
Merge pull request #9628 from fricklerhandwerk/add-links
add cross-reference
2023-12-18 13:07:39 +01:00
Valentin Gagarin
4f95800854 add cross-reference 2023-12-18 11:41:52 +01:00
Rebecca Turner
5cb98095ba
Remove some blank lines from stack traces
This keeps hint messages, source location information, and source code
snippets grouped together, while making stack traces shorter (so that
more stack frames can be viewed on the same terminal).

Before:

    error:
           … while evaluating the attribute 'body'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:

                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:

                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed

           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|

After:

    error:
           … while evaluating the attribute 'body'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:
                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:
                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed
           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|
2023-12-15 23:57:26 -08:00
Matthew Bauer
bcbdb09ccf Add eval-system option
`eval-system` option overrides just the value of `builtins.currentSystem`.
This is more useful than overriding `system` since you can build these
derivations on remote builders which can work on the given system.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-14 19:04:00 -05:00
John Ericson
e13fc0bbdb Fix sys/xattr.h check
I wrote the `configure.ac` wrong, and so we just got no builds
supporting ACLs.

Also, it needs to be more precise because Darwin puts other stuff in
that same header, evidently.
2023-12-14 10:03:48 -05:00
Ramses
1e3d811840
worker protocol: serialise cgroup stats in BuildResult (#9598)
By doing so, they get reported when building through the daemon via either `unix://` or `ssh-ng://`.
2023-12-13 16:37:17 -05:00
Eelco Dolstra
1b7968ed86
Merge pull request #9547 from hercules-ci/allowed-scheme-without-slash
`allowed-uris`: match whole schemes without slashes
2023-12-13 20:23:33 +01:00
Théophane Hufschmitt
b1c633c6bb
Merge pull request #9600 from SharzyL/fix_nix_copy
fix: nix copy ssh-ng:// not respecting --substitute-on-destination
2023-12-13 18:08:38 +01:00
Eelco Dolstra
103ca0bde5 Improve SourcePath display 2023-12-13 13:27:29 +01:00
Eelco Dolstra
cc3913e458 Remove unused variable 2023-12-13 13:27:23 +01:00
SharzyL
04f454f2a0
fix: nix copy ssh-ng:// not respecting --substitute-on-destination 2023-12-13 10:30:28 +08:00
tomberek
7026abfdde
Merge pull request #9523 from fricklerhandwerk/conf-reword-always-allow-substitutes
reword documentation on settings and attributes related to substitution
2023-12-12 20:09:48 -05:00
tomberek
09041071bf
Merge pull request #9525 from fricklerhandwerk/conf-reword-builders-use-substitutes
reword description of the `builders-use-substitutes` setting
2023-12-12 20:08:00 -05:00
Robert Hensing
0b87ba50c0 Revert "Add nix::isASCII*, locale-independent"
This reverts commit 79eb2920bb.

Not used at this time.
2023-12-12 17:46:34 +01:00
Robert Hensing
4eaeda6604 isValidSchemeName: Use regex
As requested by Eelco Dolstra. I think it used to be simpler.
2023-12-12 17:46:34 +01:00
Robert Hensing
2e451a663e schemeRegex -> schemeNameRegex
Scheme could be understood to include the typical `:` separator.
2023-12-12 17:25:20 +01:00
Théophane Hufschmitt
0dfa66d120
Merge pull request #9593 from B4dM4n/fix-path-like-flake-query
Fix query parsing for path-like flakes
2023-12-12 17:15:30 +01:00
John Ericson
9f39dda66c Fix building CA derivations with and eval store
I don't love the way this code looks. There are two larger problems:

- eval, build/scratch, destination stores (#5025) should have different
  types to reflect the fact that they are used for different purposes
  and those purposes correspond to different operations. It should be
  impossible to "use the wrong store" in my cases.

- Since drvs can end up in both the eval and build/scratch store, we
  should have some sort of union/layered store (not on the file sytem
  level, just conceptual level) that allows accessing both. This would
  get rid of the ugly "check both" boilerplate in this PR.

Still, it might be better to land this now / soon after minimal cleanup,
so we have a concrete idea of what problem better abstractions are
supposed to solve.
2023-12-11 12:17:36 -05:00
John Ericson
8cddda4f89
Merge pull request #9588 from obsidiansystems/queryDerivationOutputMap-evalStore
Give `Store::queryDerivationOutputMap` and `evalStore` argument
2023-12-11 11:16:18 -05:00
John Ericson
5f30c8acc7 Give Store::queryDerivationOutputMap and evalStore argument
Picking up where https://github.com/NixOS/nix/pull/9563 left off.
2023-12-11 10:39:08 -05:00
Fabian Möller
f45d2ee2b7
Fix query parsing for path-like flakes 2023-12-11 16:02:09 +01:00
Robert Hensing
da58c00ee0
Merge pull request #9586 from obsidiansystems/legacy-ssh-store-header
Create header for `LegacySSHStore`
2023-12-11 12:21:56 +01:00
Robert Hensing
a05bc9eb92 allowed-uris: Match whole schemes also when scheme is not followed by slashes 2023-12-11 12:18:04 +01:00
Robert Hensing
d3a85b6834 isValidSchemeName: Add function 2023-12-11 12:12:43 +01:00
Robert Hensing
79eb2920bb Add nix::isASCII*, locale-independent 2023-12-11 12:12:43 +01:00
Robert Hensing
1fa958dda1 isAllowedURI: Format 2023-12-11 12:12:43 +01:00
Robert Hensing
6cbba914a7 isAllowedURI: Remove incorrect note 2023-12-11 12:12:43 +01:00
Robert Hensing
91ba7b2307 isAllowedURI: Extract function and test 2023-12-11 12:12:42 +01:00
Adam Joseph
e43bb655fe libstore/daemon.cc: note trust model difference in readDerivation()s
Below the comment added by this commit is a much longer comment
followed by a trust check, both of which have confused me on at
least two occasions.  I figured it out once, forgot it, then had to
ask @Ericson2314 to explain it, at which point I understood it
again.  I think this might confuse other people too, or maybe I will
just forget it a third time.  So let's add a comment.

Farther down in the function is the following check:

```
if (!(drvType.isCA() || trusted))
  throw Error("you are not privileged to build input-addressed derivations");
```

This seems really strange at first.  A key property of Nix is that
you can compute the outpath of a derivation using the derivation
(and its references-closure) without trusting anybody!

The missing insight is that at this point in the code the builder
doesn't necessarily have the references-closure of the derivation
being built, and therefore needs to trust that the derivation's
outPath is honest.  It's incredibly easy to overlook this, because
the only difference between these two cases is which of these
identically-named functions we used:

- `readDerivation(Source,Store)`
- `Store::readDerivation()`

These functions have different trust models (except in the special
case where the first function is used on the local store).  We
should call the reader's attention to this fact.

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2023-12-10 17:47:07 -08:00
John Ericson
deadb3bfe9 Create header for LegacySSHStore
In https://github.com/NixOS/nix/pull/6134#issuecomment-1079199888,
@thuffschmitt proposed exposing `LegacySSHStore` in Nix for
deduplication with Hydra, at least temporarily. I think that is a good
idea.

Note that the diff will look bad unless one ignores whitespace! Also try
this locally:

```shell-session
git diff --ignore-all-space HEAD^:src/libstore/legacy-ssh-store.cc HEAD:src/libstore/legacy-ssh-store.cc
git diff --ignore-all-space HEAD^:src/libstore/legacy-ssh-store.cc HEAD:src/libstore/legacy-ssh-store.hh
```
2023-12-10 14:29:09 -05:00
John Ericson
b7e016ab24
Merge pull request #9572 from obsidiansystems/serve-proto-build-options
Create `ServeProto::BuildOptions` and a serializer for it
2023-12-10 12:16:02 -05:00
Valentin Gagarin
3c200da242
document fetchTree (#9258)
* document `fetchTree`

* display experimental feature note at the top

we have to enable the new `fetchTree` experimental feature to render it
at all. this was a bug introduced when adding that new feature flag.

Co-authored-by: tomberek <tomberek@users.noreply.github.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Silvan Mosberger <github@infinisil.com>
2023-12-10 05:16:32 +00:00
John Ericson
5417990e31 Create ServeProto::BuildOptions and a serializer for it
More tests, and more serializers for Hydra reuse.
2023-12-09 11:35:13 -05:00
Robert Hensing
c8458bd731
Merge pull request #9555 from 9999years/positions-in-errors
Pass positions when evaluating
2023-12-09 03:55:58 +01:00
Robert Hensing
7cdc8786d9
Merge pull request #9568 from hercules-ci/revert-9553
Revert 9553
2023-12-09 03:49:53 +01:00
Robert Hensing
9b7b7a7561 Revert "Print the value in error: cannot coerce messages (#9553)"
This reverts commit f0ac2a35d5.

The request from the sibling PR, which also applies here, was not addressed.
https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735
2023-12-09 02:13:32 +01:00
John Ericson
ce4ca574d2 Clarify SourceAccessor methods should never implicitly follow symlinks
The code has already been fixed (yay!) so what is left of this commit is
just updating the API docs.

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2023-12-08 20:00:47 -05:00
Robert Hensing
d4f6b1d38b
Merge pull request #9497 from edolstra/move-access-control
Move restricted/pure-eval access control out of the evaluator and into the accessor
2023-12-08 22:21:50 +01:00
John Ericson
762af72728
Merge pull request #9564 from NixOS/fix-clang-build
Avoid `std::strstream`, fix the clang build
2023-12-08 15:07:36 -05:00
Théophane Hufschmitt
36ca6adc60
Merge pull request #9563 from obsidiansystems/tryResolve-evalStore
Give `Derivation::tryResolve` an `evalStore` argument
2023-12-08 19:21:35 +01:00
John Ericson
f9ee1bedcf Avoid std::strstream, fix the clang build
According https://en.cppreference.com/w/cpp/io/strstream, it has been
deprecated since C++98! The Clang + Linux build systems to not have it
at all, or at least be hiding it.

We can just use `std::stringstream` instead, I think.
2023-12-08 13:18:52 -05:00
Rebecca Turner
f0ac2a35d5
Print the value in error: cannot coerce messages (#9553)
* Print the value in `error: cannot coerce` messages

This extends the `error: cannot coerce a TYPE to a string` message
to print the value that could not be coerced. This helps with debugging
by making it easier to track down where the value is being produced
from, especially in errors with deep or unhelpful stack traces.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-08 16:36:57 +00:00
John Ericson
96dd757b0c Give Derivation::tryResolve an evalStore argument
This is needed for building CA deriations with a src store / dest store
split. In particular it is needed for Hydra.

https://github.com/NixOS/hydra/issues/838 currently puts realizations,
and thus build outputs, in the local store, but it should not.
2023-12-08 10:01:05 -05:00