Commit graph

2659 commits

Author SHA1 Message Date
John Ericson
1620ad4587 Split out GlobalConfig into its own header
This makes it easier to understand the reach of global variables /
global state in the config system.
2024-06-24 11:36:21 -04:00
John Ericson
b51e161af5 Cleanup ContentAddressMethod to match docs
The old `std::variant` is bad because we aren't adding a new case to
`FileIngestionMethod` so much as we are defining a separate concept ---
store object content addressing rather than file system object content
addressing. As such, it is more correct to just create a fresh
enumeration.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-24 10:24:06 -04:00
John Ericson
64e599ebe1 Rename Recursive -> NixArchive
For enums:

- `FileIngestionMethod`

- `FileSerialisationMethod`
2024-06-24 10:24:06 -04:00
Robert Hensing
6f64154eea
Merge pull request #10884 from tomberek/tomberek.warn_structuredAttrs_advanced
fix: warn and document when advanced attributes will have no impact d…
2024-06-24 07:56:26 +02:00
HaeNoe
7fb14201af Unit test for derivation "advanced attrs"
This tests the parser and JSON format using the DRV files from the tests
added in the previous commit.

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2024-06-23 21:42:56 -04:00
HaeNoe
9f9984e4d0 Functional test for derivation "advanced attrs"
This tests the Nix language side of things.

We are purposely skipping most of `common.sh` because it is overkill for
this test: we don't want to have an "overfit" test environment.

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2024-06-23 21:42:56 -04:00
John Ericson
490ca93cf8 Factor out a bit more language testings infra
Will be used in a second test after `lang.sh`.
2024-06-23 15:33:45 -04:00
Eelco Dolstra
d54590fdf3 Fix --no-sandbox
When sandboxing is disabled, we cannot put $TMPDIR underneath an
inaccessible directory.
2024-06-21 17:06:19 +02:00
Théophane Hufschmitt
1d3696f0fb Run the builds in a daemon-controled directory
Instead of running the builds under
`$TMPDIR/{unique-build-directory-owned-by-the-build-user}`, run them
under `$TMPDIR/{unique-build-directory-owned-by-the-daemon}/{subdir-owned-by-the-build-user}`
where the build directory is only readable and traversable by the daemon user.

This achieves two things:

1. It prevents builders from making their build directory world-readable
   (or even writeable), which would allow the outside world to interact
   with them.
2. It prevents external processes running as the build user (either
   because that somehow leaked, maybe as a consequence of 1., or because
   `build-users` isn't in use) from gaining access to the build
   directory.
2024-06-21 17:06:19 +02:00
Théophane Hufschmitt
717f3eea39 Add a test for the user sandboxing 2024-06-21 17:06:18 +02:00
Robert Hensing
d9684664c8 Revert "tests/functional/common/init.sh: Use parentheses around negation"
ShellCheck doesn't want us to add extra parentheses for show.

This reverts commit 7c9f3eeef8.
2024-06-20 22:31:32 +02:00
Robert Hensing
dcee46a0ef Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-06-20 14:54:11 +02:00
Robert Hensing
7c9f3eeef8 tests/functional/common/init.sh: Use parentheses around negation
roberth: Not strictly necessary, but probably a good habit

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-06-20 14:54:11 +02:00
Robert Hensing
648302b833 tests/functional: Enable more tests in NixOS VM 2024-06-20 14:54:11 +02:00
Robert Hensing
d208e9dd9f tests: Add quickBuild to all VM tests 2024-06-20 14:54:11 +02:00
Robert Hensing
8557d79650 tests/functional: Skip tests that don't work in NixOS environment yet 2024-06-20 14:54:11 +02:00
Robert Hensing
211aec473e tests/functional/timeout.sh: Find missing test case
This reproduces an instance of
https://github.com/NixOS/nix/issues/4813
2024-06-20 14:54:11 +02:00
Robert Hensing
439022c5ac tests: Add hydraJobs.tests.functional_* 2024-06-20 14:54:11 +02:00
PoweredByPie
8b81d083a7 Remove lookupPathForProgram and implement initial runProgram test
Apparently, CreateProcessW already searches path, so manual path search
isn't really necessary.
2024-06-18 01:01:52 -07:00
PoweredByPie
4f6e3b9402 Implement tests for lookupPathForProgram and fix bugs caught by tests 2024-06-17 18:46:08 -07:00
PoweredByPie
d7537f6955 Implement initial spawn tests (just testing windowsEscape for now) 2024-06-17 14:58:17 -07:00
Robert Hensing
316b58dd5f tests/shell.sh: Check that env is mostly unmodified 2024-06-17 17:03:58 +02:00
Robert Hensing
68b8a28bc4 tests/run.sh: Check that env is mostly unmodified 2024-06-17 17:03:58 +02:00
Robert Hensing
2894c1b38e WIP add testresults output 2024-06-16 16:34:54 +02:00
Robert Hensing
573e385a68
Merge pull request #10907 from hercules-ci/issue-10561
C API: Use opaque struct instead of void for `nix_value`
2024-06-15 10:12:13 +02:00
Robert Hensing
b94e1d6218 C API: Value -> nix_value
See issue https://github.com/NixOS/nix/issues/10434
2024-06-13 18:51:58 +02:00
Robert Hensing
0b56c98b1c C API: Value -> nix_value 2024-06-13 18:18:36 +02:00
Eelco Dolstra
1dc7c8e599 eval-fail-infinite-recursion-lambda: Reduce recursion depth
This prevents the test from failing in environments with a smaller
configured stack size.
2024-06-13 13:55:42 +02:00
John Ericson
33241887d1 More quote coalescing 2024-06-12 17:47:54 -04:00
John Ericson
d8ae28617d Try to fix quotes that don't go to end with sed 2024-06-12 17:41:16 -04:00
Cameron Dart
2d467b4731 housekeeping: shellcheck for tests/functional/import-derivation.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
1c93360989 housekeeping: shellcheck for tests/functional/hash-path.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
d1c476865a housekeeping: shellcheck for tests/functional/gc-runtime.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
cd46ec17f9 housekeeping: shellcheck for tests/functional/function-trace.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
b764dd9aa4 housekeeping: shellcheck for tests/functional/flakes/unlocked-override.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
e1ce349d05 housekeeping: shellcheck for tests/functional/flakes/search-root.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
b9768b4872 housekeeping: shellcheck for tests/functional/flakes/mercurial.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
c7b3468968 housekeeping: shellcheck for tests/functional/flakes/inputs.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
d95adb531e housekeeping: shellcheck for tests/functional/flakes/init.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
71d5baca47 housekeeping: shellcheck for tests/functional/flakes/flake-in-submodule.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
3b853e795b housekeeping: shellcheck for tests/functional/flakes/circular.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
ece86b7191 housekeeping: shellcheck for tests/functional/flakes/bundle.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
ee57c610ae housekeeping: shellcheck for tests/functional/flakes/build-paths.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
d81fd4a1c3 housekeeping: shellcheck for tests/functional/flakes/absolute-attr-paths.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
ae6a842c55 housekeeping: shellcheck for tests/functional/filter-source.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
224f5515b9 housekeeping: shellcheck for tests/functional/fetchTree-file.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:16 -04:00
Cameron Dart
f0492a6197 housekeeping: shellcheck for tests/functional/fetchPath.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
823d53c643 housekeeping: shellcheck for tests/functional/experimental-features.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
1c1abefdd2 housekeeping: shellcheck for tests/functional/dyn-drv/text-hashed-output.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
847842c4bb housekeeping: shellcheck for tests/functional/derivation-json.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
58a653dcc6 housekeeping: shellcheck for tests/functional/compression-levels.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
48520cb71e housekeeping: shellcheck for tests/functional/chroot-store.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
63272235e2 housekeeping: shellcheck for tests/functional/case-hacks.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:16 -04:00
Cameron Dart
4f04006bc1 housekeeping: shellcheck for tests/functional/check-reqs.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
d7bb5bde48 housekeeping: shellcheck for tests/functional/check-refs.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
259b502773 housekeeping: shellcheck for tests/functional/ca/substitute.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
04876c39e4 housekeeping: shellcheck for tests/functional/ca/signatures.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
8f85537629 housekeeping: shellcheck for tests/functional/ca/nix-run.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
deacc421eb housekeeping: shellcheck for tests/functional/ca/nix-copy.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
195c0da849 housekeeping: shellcheck for tests/functional/ca/duplicate-realisation-in-closure.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
2dfbba3e5e housekeeping: shellcheck for tests/functional/ca/derivation-json.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
627176fd54 housekeeping: shellcheck for tests/functional/ca/build.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
80c44138cb housekeeping: shellcheck for tests/functional/ca/build-cache.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:16 -04:00
Cameron Dart
c127625095 housekeeping: shellcheck for tests/functional/build-dry.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
f615489e0e housekeeping: shellcheck for tests/functional/build-delete.sh 2024-06-12 17:41:16 -04:00
Cameron Dart
7186c68f75 housekeeping: shellcheck for tests/functional/brotli.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:15 -04:00
Cameron Dart
aeed835a2e housekeeping: shellcheck for tests/functional/binary-cache.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:08 -04:00
Cameron Dart
4a28ba7877 housekeeping: shellcheck for tests/functional/binary-cache-build-remote.sh 2024-06-12 16:58:25 -04:00
Cameron Dart
7738b295e5 housekeeping: shellcheck for tests/functional/bash-profile.sh 2024-06-12 16:58:25 -04:00
John Ericson
7c2981fc55 Fix FreeBSD build
This restores some CPP'd code that was added in
c18911602e and accidentally lost in
2477e4e3b8.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-06-12 15:59:54 -04:00
Eelco Dolstra
bd37a70d8f
Update tests/functional/tarball.sh
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-11 19:39:42 +02:00
Eelco Dolstra
35bdb9cee7 Support hard links in tarballs
Fixes #10395.
2024-06-11 16:05:57 +02:00
Robert Hensing
de3fd52a95 Add tests/f/lang/eval-okay-derivation-legacy 2024-06-10 16:31:46 +02:00
John Ericson
b0283240a1
Merge pull request #10864 from Mic92/tarball-flakes-test
tests/nixos: make the tarball-flakes test better reflect real use cases
2024-06-05 18:40:16 -04:00
John Ericson
1a32490aca
Merge pull request #10835 from hercules-ci/update-nixpkgs-24.05
Update to Nixpkgs 24.05
2024-06-05 17:56:16 -04:00
Pierre Bourdon
162d94d975 tests/nixos: make the tarball-flakes test better reflect real use cases
In most real world cases, the Link header is set on the redirect, not on
the final file. This regressed in Lix earlier and while new unit tests
were added to cover it, this integration test should probably have also
caught it.

Source: a3256a9375
2024-06-05 20:14:24 +02:00
Robert Hensing
5d460d563e TMP: Disable tests.setuid.i686-linux
Temporarily(?) blocked on https://github.com/NixOS/nixpkgs/pull/297475#issuecomment-2145589501
2024-06-03 18:47:53 +02:00
Robert Hensing
27f880c098 Format after clang-format update 2024-06-03 18:38:56 +02:00
Robert Hensing
8a510f4ede Add tests.remoteBuilds_*_2_18 2024-06-03 17:34:55 +02:00
Robert Hensing
efc2508e8b Refactor hydraJobs.tests.remoteBuilds_*_2_18 2024-06-03 17:34:55 +02:00
Robert Hensing
6558025e77 Fix eval remoteBuilds_*_2_13 2024-06-03 17:34:55 +02:00
Robert Hensing
449e4b9232 Change checkOverrideNixVersion for NixOS 24.05 2024-06-03 17:34:55 +02:00
Eelco Dolstra
da92ad7dd2
Merge pull request #10592 from hercules-ci/builtins-warn
Add `builtins.warn`
2024-06-03 17:16:32 +02:00
Robert Hensing
da82d67022 builtins.warn: Require string argument
... so that we may perhaps later extend the interface.
Note that Nixpkgs' lib.warn already requires a string coercible
argument, so this is reasonable. Also note that string coercible
values aren't all strings, but in practice, for warn, they are.
2024-06-03 16:24:21 +02:00
Robert Hensing
2d4c9d8f4a Add builtins.warn 2024-06-03 16:24:21 +02:00
Eelco Dolstra
54a9fbe5d6 Merge remote-tracking branch 'origin/master' into large-path-warning 2024-06-03 16:17:52 +02:00
Eelco Dolstra
d07cdbd9c2
Merge pull request #10834 from obsidiansystems/fix-shellcheck
Fix shellcheck issue
2024-06-03 16:14:41 +02:00
Eelco Dolstra
ac3e5d22e3
Merge pull request #10028 from DavHau/fetchTree-shallow-default
fetchTree: shallow git fetching by default
2024-06-03 16:02:34 +02:00
John Ericson
1e99f324d9 Fix shellcheck issue
8b86f415c1 was merged from a CI run that
predated the new linting.
2024-06-03 09:36:48 -04:00
Eelco Dolstra
7f5b57d18f Merge remote-tracking branch 'origin/master' into large-path-warning 2024-06-03 15:32:27 +02:00
Eelco Dolstra
ecfad6a828
Merge pull request #10564 from edolstra/remove-forceErrors
AttrCursor: Remove forceErrors
2024-06-03 15:30:01 +02:00
Eelco Dolstra
d16fcaee21
Merge pull request #10782 from obsidiansystems/both-connections
Factor our connection code for worker proto like serve proto
2024-06-03 15:10:38 +02:00
John Ericson
c6add8873e
Merge pull request #9995 from NixOS/json-empty-sigs
`ValidPathInfo` JSON format should use `null` not omit field
2024-06-03 08:58:49 -04:00
Robert Hensing
de5050f73b
Merge pull request #9590 from wh0/patch-1
nix-profile: fix both profile links detection
2024-06-03 14:37:08 +02:00
John Ericson
84c65135a5 ValidPathInfo JSON format should use null not omit field
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-03 08:21:22 -04:00
Robert Hensing
f8bd4ba561
Merge pull request #10827 from SkamDart/skamdart/functional-add-sc
housekeeping: shellcheck test/functional/add.sh
2024-06-03 12:31:39 +02:00
Cameron Dart
6a507f5d3b housekeeping: shellcheck test/functional/add.sh 2024-06-02 13:41:51 -07:00
Ivan Trubach
68090d7ff1 Fix empty outputsToInstall for InstallableAttrPath
Fixes assertion failure if outputsToInstall is empty by defaulting to the "out"
output. That is, behavior between the following commands should be consistent:

	$ nix build --no-link --json .#nothing-to-install-no-out
	error: derivation '/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-nothing-to-install-no-out.drv' does not have wanted outputs 'out'

	$ nix build --no-link --file default.nix --json nothing-to-install-no-out
	error: derivation '/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-nothing-to-install-no-out.drv' does not have wanted outputs 'out'

Real-world example of this issue:

	$ nix build --json .#.legacyPackages.aarch64-linux.texlive.pkgs.iwona
	error: derivation '/nix/store/dj0h6b0pnlnan5nidnhqa0bmzq4rv6sx-iwona-0.995b.drv' does not have wanted outputs 'out'

	$ git rev-parse HEAD
	eee33247cf6941daea8398c976bd2dda7962b125
	$ nix build --json --file . texlive.pkgs.iwona
	nix: src/libstore/outputs-spec.hh:46: nix::OutputsSpec::Names::Names(std::set<std::__cxx11::basic_string<char> >&&): Assertion `!empty()' failed.
	Aborted (core dumped)
2024-06-02 14:26:18 +03:00
Robert Hensing
802b4e403b
Merge pull request #10814 from Mic92/fix-nix-edit
Fix nix edit
2024-05-31 13:30:24 +02:00
Jörg Thalheim
69c159811e add regression test for nix edit 2024-05-31 12:58:47 +02:00
Robert Hensing
c692f6af13 nix env shell: Move from nix shell, add shorthand alias 2024-05-30 19:41:58 +02:00
John Ericson
10f864c5ae Ensure all functional scripts are (a) executable (b) have shebang
This is good for shebang, and also good for future build system
simplifications
2024-05-28 12:46:24 -04:00
John Ericson
2e12b58126 Shellcheck some test scripts
Progress on #10795
2024-05-28 12:32:22 -04:00
John Ericson
1e2b26734b
Merge pull request #10799 from hercules-ci/safer-tab-completion
Add repl completion test
2024-05-28 11:30:56 -04:00
John Ericson
567265ae67 Start getting all shell scripts passing shellcheck
Like with the formatter, we are blacklisting most files by default.

Do a few files to get us started, and get a sense of what this looks
like.
2024-05-27 22:39:56 -04:00
Robert Hensing
ebfada36a1 Add repl completion test 2024-05-27 09:58:49 +02:00
John Ericson
f71b4da0b3 Factor our connection code for worker proto like serve proto
This increases test coverage, and gets the worker protocol ready to be
used by Hydra.

Why don't we just try to use the store interface in Hydra? Well, the
problem is that the store interface works on connection pools, with each
opreation getting potentially a different connection, but the way temp
roots work requires that we keep one logical "transaction" (temp root
session) using the same connection.

The longer-term solution probably is making connections themselves
implement the store interface, but that is something that builds on
this, so I feel OK that this is not churn in the wrong direction.

Fixes #9584
2024-05-27 00:43:46 -04:00
John Ericson
eeb89c28b0 Worker proto use proper serialiser for BuildMode
Do this instead of an unchecked cast

I redid this to use the serialisation framework (including a unit test),
but I am keeping the reference to credit Jade for spotting the issue.

Change-Id: Icf6af7935e8f139bef36b40ad475e973aa48855c
(adapted from commit 2a7a824d83dc5fb33326b8b89625685f283a743b)

Co-Authored-By: Jade Lovelace <lix@jade.fyi>
2024-05-27 00:22:55 -04:00
John Ericson
5cfa75ea16
Merge pull request #10737 from poweredbypie/mingw-stackSize
Implement `setStackSize` on Windows
2024-05-25 09:56:02 -04:00
John Ericson
e0c94b91ee
Merge pull request #10757 from obsidiansystems/fix-4977
Require `drvPath` attribute to end with `.drv`
2024-05-24 12:14:59 -04:00
Eelco Dolstra
8b86f415c1 Add test for the evaluation cache 2024-05-24 16:34:49 +02:00
PoweredByPie
0b7da099d1 Commit more stack size in some windows binaries
This way we can commit the same amount of stack size (64 MB) without a conditional.
Includes nix, libnixexpr-tests, libnixfetchers-tests, libnixstore-tests, libnixutil-tests.
2024-05-23 17:42:55 -07:00
Robert Hensing
97c3463291 C API: Refactor: use NIX_VALUE_CALL 2024-05-23 21:25:23 +02:00
Robert Hensing
2497d10351 C API: Add nix_value_call_multi, NIX_VALUE_CALL
_multi can be implemented more efficiently.
NIX_VALUE_CALL is a convenient way to invoke it.
2024-05-23 21:24:06 +02:00
Robert Hensing
ab106c5ca3 C API: Test arity 2 primop 2024-05-23 21:23:15 +02:00
Robert Hensing
8ef6efc184 C API: Require non-thunk value from primop definition 2024-05-23 21:22:59 +02:00
Robert Hensing
8884227045 C API: Require initialized value from primop definition 2024-05-23 21:22:21 +02:00
Robert Hensing
a942a34469 C API: Fix nix_c_primop_wrapper for strict initializers
https://github.com/NixOS/nix/pull/10555 added a check requiring
that output parameters always have an uninitialized Value as argument.

Unfortunately the output parameter of the primop callback received
a thunk instead.

See the comment for implementation considerations.
2024-05-23 18:32:49 +02:00
John Ericson
f2bcebc450 Restore exposing machine file parsing
This was accidentally removed in
e989c83b44. I restored it and also did a
few other cleanups:

- Make a static method for namespacing purposes

- Put the test files in the data dir with the other test data

- Avoid mutating globals in the machine config tests

This will be used by Hydra.
2024-05-23 00:03:52 -04:00
John Ericson
f923ed6b6a Require drvPath attribute to end with .drv
Fixes #4977
2024-05-22 12:50:24 -04:00
John Ericson
b3ebcc5aad Use the new StoreReference in Machine
This makes the remote builder abstract syntax more robust.
2024-05-22 09:20:15 -04:00
John Ericson
b59a7a14c4 Add StoreReference::render
This will be needed for the next step.

Also allows us to write round trip tests.
2024-05-22 09:20:15 -04:00
John Ericson
1d6c2316a9 Slightly change formatting style
For long expressions, one argument or parameter per line is just easier.
2024-05-22 09:20:15 -04:00
John Ericson
470c0501eb Ensure all store types support "real" URIs
In particular `local://<path>` and `unix://` (without any path) now
work, and mean the same things as `local` and `daemon`, respectively. We
thus now have the opportunity to desguar `local` and `daemon` early.

This will allow me to make a change to
https://github.com/NixOS/nix/pull/9839 requested during review to
desugar those earlier.

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-05-21 11:56:40 -04:00
John Ericson
8b369f90fd Query path infos (plural) and handshake version minimum for hydra
1. Hydra currently queries for multiple path infos at once, so let us
   make a connection item for that.

2. The minimum of the two versions should always be used, see #9584.
   (The issue remains open because the daemon protocol needs to be
   likewise updated.)
2024-05-20 17:43:18 -04:00
Robert Hensing
67db9e0c64
Merge pull request #10722 from obsidiansystems/ca-obj-docs
Content addressing store objects
2024-05-20 15:58:29 +02:00
John Ericson
1c75af969a Document store object content addressing & improve JSON format
The JSON format no longer uses the legacy ATerm `r:` prefixing nonsese,
but separate fields.

Progress on #9866

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-20 09:34:35 -04:00
Robert Hensing
e4be8abe42
Merge pull request #10701 from cole-h/nix-command-warn-unknown-settings
Warn on unknown settings when the first positional is an argument
2024-05-20 15:15:13 +02:00
Maximilian Bosch
19720d733f nix3-build: show all FOD errors with --keep-going
Basically I'd expect the same behavior as with `nix-build`, i.e.
with `--keep-going` the hash-mismatch error of each failing
fixed-output derivation is shown.

The approach is derived from `Store::buildPaths` (`entry-point.cc`):
instead of throwing the first build-result, check if there are any build
errors and if so, display all of them and throw after that.

Unfortunately, the BuildResult struct doesn't have an `ErrorInfo`
(there's a FIXME for that at least), so I have to construct my own here.
This is a rather cheap bugfix and I decided against touching too many
parts of libstore for that (also I don't know if that's in line with the
ongoing refactoring work).

Closes https://git.lix.systems/lix-project/lix/issues/302

Change-Id: I378ab984fa271e6808c6897c45e0f070eb4c6fac
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2024-05-17 18:48:52 +02:00
Eelco Dolstra
3026613893
Merge pull request #10675 from edolstra/zip-symlinks
Handle zip files containing symlinks
2024-05-15 22:07:22 +02:00
Cole Helbling
06e13465c5 tests/functional: test both clis warn on unknown settings 2024-05-15 12:25:03 -07:00
Eelco Dolstra
1da18e85ba tests/functional/common/init.sh: Make $TEST_ROOT writable before removing it
$TEST_ROOT typically contains read-only files/directories (e.g. the
Nix store). So we have to make it writable first.
2024-05-14 16:23:08 +02:00
Eelco Dolstra
9a58d90c73 tests/nixos/containers/containers.nix: Remove superfluous -v 2024-05-14 14:27:09 +02:00
Valentin Gagarin
7822ecbadf tests: always clean the test directory
previously the test directory could have been left untouched before executing
a test when `init.sh` was not run - and sometimes it isn't
supposed to be run - which made the test suite highly stateful and thus
behaving surprisingly on multiple runs.
2024-05-13 15:19:49 +02:00
Valentin Gagarin
33ca905cdb tests: simplify initialisation and wiring
pararameterisation is not actually needed the way things are currently
set up, and it confused me when trying to understand what the code does.

all but one test sources vars-and-functions.sh, which nominally only
defines variables, but in practice is always coupled with the actual
initialisation. while the cleaner way of making this more legible would
be to source variables and initialisation separately, this would produce
a huge diff.

the change requires a few small fixes to keep the tests working:

- only create test home directory during initialisation

  that vars-and-functions.sh wrote to the file system seems not write

- fix creation of the test directory

  due to statefulness, the test home directory was implicitly creating
  the test root, too. decoupling that made it apparent that this was
  probably not intentional, and certainly confusing.

- only source vars-and-functions.sh if init.sh is not needed

  there is one test case that only needs a helper function but no
  initialisation side effects

- remove some unnecessary cleanups and split parts of re-used test code

  there were confusing bits in how initialisation code was repurposed,
  which break if trying to refactor the outer layers naively...
2024-05-13 15:19:49 +02:00
Eelco Dolstra
f0b5628eb2 renderSize(): Add some unit tests 2024-05-13 12:08:51 +02:00
Robert Hensing
c940d11fb0
Merge pull request #10666 from tie/derivation-outputs-drv-path
Forbid drvPath in strictDerivation outputs attribute
2024-05-12 21:14:32 +02:00
Eelco Dolstra
9951e14ae0 Handle zip files containing symlinks
In streaming mode, libarchive doesn't handle symlinks in zip files
correctly. So write the entire file to disk so libarchive can access
it in random-access mode.

Fixes #10649. This was broken in cabee98152.
2024-05-09 19:33:09 +02:00
Eelco Dolstra
79c7d6205c Support unit prefixes in configuration settings
E.g. you can now say `--min-free 1G`.
2024-05-08 21:11:09 +02:00
Ivan Trubach
081faeda8c Forbid drvPath in strictDerivation outputs attribute
builtins.strictDerivation returns an attribute set with drvPath and
output paths. For some reason, current implementation forbids drv
instead of drvPath.
2024-05-08 18:20:37 +03:00
ramboman
b4950404ba
Honor the same set of proxy environment variables (#10611)
Different parts of the project honor different sets of proxy environment
variables. With this commit all parts of the project will honor the same
set of proxy environment variables.

---------

Co-authored-by: Your Name <you@example.com>
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-05-06 15:39:22 -04:00
Eelco Dolstra
da3381d51f
Merge pull request #10465 from edolstra/remove-locked
Fetcher cache cleanups
2024-05-06 21:32:26 +02:00
John Ericson
1ad7b5451d
Merge pull request #10655 from edolstra/use-source-path
Use `SourcePath` in more places
2024-05-06 14:57:01 -04:00
John Ericson
5e189025ca Fix build failure with clang
A slight issue with feb1d10f60.
2024-05-06 13:39:21 -04:00
Eelco Dolstra
eab2919119 Use SourcePath in more places
Now that SourcePath uses a SourceAccessor instead of an InputAccessor,
we can use it in function signatures instead of passing a
SourceAccessor and CanonPath separately.
2024-05-06 19:05:42 +02:00
Eelco Dolstra
2926ef0e90
Merge pull request #10652 from tweag/check-additionalSandboxProfile
libstore: check additionalSandboxProfile
2024-05-06 17:59:25 +02:00
Théophane Hufschmitt
9bd1191fcc libstore: check additionalSandboxProfile
Make sure that `extraSandboxProfile` is set before we check whether it's
empty or not (in the `sandbox=true` case).

Also adds a test case for this.

Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
2024-05-06 17:37:08 +02:00
HaeNoe
feb1d10f60
_not_ round-trip tests for fetchers::PublicKey default type (#10637)
Another continuation of #10602
2024-05-06 09:50:26 -04:00
Eelco Dolstra
20558e0462 Remove FSInputAccessor 2024-05-03 12:30:28 +02:00
Eelco Dolstra
ba5929c7be Merge InputAccessor into SourceAccessor
After the removal of the InputAccessor::fetchToStore() method, the
only remaining functionality in InputAccessor was `fingerprint` and
`getLastModified()`, and there is no reason to keep those in a
separate class.
2024-05-03 12:14:01 +02:00
Robert Hensing
e17aad23d6
Merge pull request #10555 from jlesquembre/jl/c-api_check-init
Add isValid to nix::Value
2024-05-01 16:33:01 +02:00
Eelco Dolstra
1f41682217
Update tests/functional/flakes/flakes.sh
Co-authored-by: John Ericson <git@JohnEricson.me>
2024-04-30 18:10:16 +02:00
Eelco Dolstra
f29a220b70 Test that the root of a tree produces /nix/store/<hash1>-<hash2>-source 2024-04-30 17:25:35 +02:00
Eelco Dolstra
503be57bbd Test baseNameOf behaviour on the root of a flake 2024-04-30 15:43:33 +02:00
Eelco Dolstra
458441c637 Test dirOf behaviour on the root of a flake 2024-04-30 15:34:38 +02:00
John Ericson
4722b0c9e9
Merge pull request #10602 from haenoe/json-infra-tests-misc
Json infra tests misc
2024-04-25 16:53:51 -04:00
HaeNoe
c73172e986
add unit tests for getNullable 2024-04-24 18:22:53 +02:00
HaeNoe
4ff7f5aa9c
refactor fetchers::PublicKey tests 2024-04-24 18:22:41 +02:00
Max
5ecd820c18 Merge remote-tracking branch 'nixos/master' 2024-04-22 23:29:11 +02:00
Robert Hensing
750bcaa330 Fix fetchGit nested submodules 2024-04-22 16:41:40 +02:00
José Luis Lafuente
6acf02b32a
C API: source argument to nix_copy_value should be const 2024-04-21 22:46:18 +02:00
José Luis Lafuente
8d70db3251
C API: add check_value_[in,out] helper functions 2024-04-21 22:44:14 +02:00
José Luis Lafuente
ff76dd2211
C API: fix test, nix float is a double internally 2024-04-21 22:44:14 +02:00
José Luis Lafuente
ccad6e94e2
C API: add (un)initialized value checks 2024-04-21 22:44:14 +02:00
José Luis Lafuente
9d7dee4a8f
nix::Value: Use more descriptive names 2024-04-21 22:44:13 +02:00
José Luis Lafuente
5cc4af5231
Add isInitialized to nix::Value
Add a method to check if a value has been initialized. This helps avoid
segfaults when calling `type()`.
Useful in the context of the new C API.

Closes #10524
2024-04-21 22:44:13 +02:00
Robert Hensing
6fd2f42c2d
Merge pull request #10573 from RCoeurjoly/Rename_SearchPath
Rename SearchPath to LookupPath and searchPath to lookupPath
2024-04-21 17:33:46 +02:00
Robert Hensing
1b6cd1d2af Revert "tests/test-libstoreconsumer: Ignore config"
This reverts commit 62feb5ca09263c78ddb692836228223e5b58d3ae.

It runs as part of the functional tests, which control the environment,
solving some of the problems a default config has when run in the
sandbox.
2024-04-20 01:45:04 +02:00
Robert Hensing
bcaa2e4a85 tests/libstore-support: Ignore config 2024-04-20 01:45:04 +02:00
Robert Hensing
0ecf7dac3b tests/test-libstoreconsumer: Ignore config 2024-04-20 01:45:04 +02:00
Max
2d0b0537fe Merge remote-tracking branch 'nixos/master' 2024-04-19 16:42:33 +02:00
Robert Hensing
538eb2617a
Merge pull request #10537 from hercules-ci/c-api-nix-value-init-apply
C API: Add nix_init_apply
2024-04-18 20:09:24 +02:00
Robert Hensing
ad643cde58 C API: Add nix_init_apply
Thunks are relevant when initializing attrsets and lists, passing
arguments. This is an important way to produce them.
2024-04-18 19:13:38 +02:00
John Ericson
8433027e35 Build a minimized Nix with MinGW
At this point many features are stripped out, but this works:

- Can run libnix{util,store,expr} unit tests
- Can run some Nix commands

Co-Authored-By volth <volth@volth.com>
Co-Authored-By Brian McKenna <brian@brianmckenna.org>
2024-04-17 12:26:10 -04:00
Eelco Dolstra
6a3ecdaa39 Merge remote-tracking branch 'origin/master' into finish-value 2024-04-17 16:02:44 +02:00
José Luis Lafuente
c75b143b6c
C API: nix_get_string now accepts a callback to return the value 2024-04-16 22:36:39 +02:00
Eelco Dolstra
74e4bc9b1d
Merge pull request #10486 from tweag/jl/c-api_function-pointer
C API: Safer function pointer casting
2024-04-16 18:56:08 +02:00
Théophane Hufschmitt
d2a07a96ba
Merge pull request #10467 from edolstra/nix-shell-symlink
nix shell: Handle output paths that are symlinks
2024-04-16 12:31:14 +02:00
John Ericson
65cc237b3a
Merge pull request #10482 from tweag/fix-symlink-in-sandbox
Fix the access of symlinks to host files in the sandbox
2024-04-15 09:29:00 -04:00
Eelco Dolstra
d084c1cb41 Remove the "locked" flag from the fetcher cache
This also reworks the Mercurial fetcher (which was still using the
old cache interface) to have two distinct cache mappings:

* A ref-to-rev mapping, which is store-independent.
* A rev-to-store-path mapping.
2024-04-15 13:03:27 +02:00
José Luis Lafuente
774e7213e8
C API: Use nix_get_string_callback typedef 2024-04-15 12:05:57 +02:00
HaeNoe
ff4c286e80
add tests for optionalValueAt 2024-04-14 22:36:03 +02:00
Roland Coeurjoly
40a6a9fdb8 Rename SearchPath to LookupPath and searchPath to lookupPath 2024-04-13 17:35:15 +02:00
José Luis Lafuente
01bad63c72
C API: Safer function pointer casting
See https://github.com/NixOS/nix/pull/8699#discussion_r1554312181

Casting a function pointer to `void*` is undefined behavior in the C
spec, since there are platforms with different sizes for these two kinds
of pointers. A safe alternative might be `void (*callback)()`
2024-04-12 21:41:15 +02:00
Théophane Hufschmitt
cef677ddbc Test the inclusion of transitive symlinks in the sandbox 2024-04-12 16:10:22 +02:00
Théophane Hufschmitt
acbb1523c1 Fix the access of symlinks to host files in the sandbox
https://github.com/NixOS/nix/pull/10456 fixed the addition of symlink
store paths to the sandbox, but also made it so that the hardcoded
sandbox paths (like `/etc/hosts`) were now bind-mounted without
following the possible symlinks. This made these files unreadable if
there were symlinks (because the sandbox would now contain a symlink to
an unreachable file rather than the underlying file).
In particular, this broke FOD derivations on NixOS as `/etc/hosts` is a
symlink there.

Fix that by canonicalizing all these hardcoded sandbox paths before
adding them to the sandbox.
2024-04-12 15:57:53 +02:00
John Ericson
5b9cb8b372
Merge pull request #10412 from roberth/c-string-context
C API: Add `nix_string_realise`
2024-04-11 12:07:31 -04:00
Robert Hensing
f2522d4ecd libexpr-c: Add nix_store_path_name 2024-04-11 17:39:21 +02:00
Robert Hensing
a512f4eebc test/libutil: Add OBSERVE_STRING macro
Makes string callback easier to pass, without mistakes.
2024-04-11 17:39:21 +02:00
Robert Hensing
876e70bc9a tests/unit/libexpr/local.mk
A proper build system would catch errors like this.
2024-04-11 17:39:21 +02:00
Robert Hensing
48808a5320 tests/unit/libexpr: Enable nix_store_realise test, and add docs 2024-04-11 17:39:21 +02:00
Robert Hensing
94d9819bdc tests/unit/libexpr/main: Fix realisation 2024-04-11 17:39:19 +02:00
Bouke van der Bijl
1e4f902b28 Add gitSubmodules test to github actions 2024-04-11 15:55:09 +02:00
Bouke van der Bijl
cd06193d13 Add nixos test 2024-04-11 15:28:38 +02:00
Théophane Hufschmitt
da1e977bf4
Merge pull request #10456 from NixOS/fixpermdeniedbind
Fix adding symlink to the sandbox paths
2024-04-11 13:41:49 +02:00
Eelco Dolstra
26a4688a86 nix shell: Test that store paths cannot link outside of the store 2024-04-11 11:06:35 +02:00
Eelco Dolstra
9d50f57fa3 Doh 2024-04-11 09:00:47 +02:00
Théophane Hufschmitt
def00d7b52
Merge pull request #10455 from tie/structured-attrs-get-env
Do not rely on $stdenv/setup to set output variables
2024-04-11 06:48:36 +02:00
Eelco Dolstra
85b9f4ef4f nix shell: Handle output paths that are symlinks
This requires moving resolveSymlinks() into SourceAccessor. Also, it
requires LocalStoreAccessor::maybeLstat() to work on parents of the
store (to avoid an error like "/nix is not in the store").

Fixes #10375.
2024-04-10 23:49:19 +02:00
Ivan Trubach
664532c533 Do not rely on $stdenv/setup to set output variables
Instead of relying on setup script to set output variables when
structured attributes are enabled, iterate over the values of an
outputs associative array.

See also
374fa3532e/pkgs/stdenv/generic/setup.sh (L23-L26)
2024-04-10 19:50:19 +03:00