Commit graph

1800 commits

Author SHA1 Message Date
John Ericson
2291232dc1
Merge pull request #8387 from layus/fix-tests
Check exact error codes in linux-sandbox.sh
2023-06-22 13:56:03 -04:00
Guillaume Maudoux
3a20c7c46c
Update tests/linux-sandbox.sh
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-06-20 22:51:29 +02:00
Ben Radford
6ae35534b7
Support opening local store with database on read-only filesystem (#8356)
Previously it was not possible to open a local store when its database is on a read-only filesystem. Obviously a store on a read-only filesystem cannot be modified, but it would still be useful to be able to query it.

This change adds a new read-only setting to LocalStore. When set to true, Nix will skip operations that fail when the database is on a read-only filesystem (acquiring big-lock, schema migration, etc), and the store database will be opened in immutable mode.

Co-authored-by: Ben Radford <benradf@users.noreply.github.com>
Co-authored-by: cidkidnix <cidkidnix@protonmail.com>
Co-authored-by: Dylan Green <67574902+cidkidnix@users.noreply.github.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-20 11:34:09 +02:00
Eelco Dolstra
e503eadafc
Merge pull request #8477 from edolstra/tarball-flake-redirects
Tarball flake improvements
2023-06-16 18:03:50 +02:00
Eelco Dolstra
713836112c
Merge pull request #8517 from hercules-ci/fix-build-hook-error-for-lib-users
Fix build hook error for libstore library users
2023-06-16 13:20:50 +02:00
Robert Hensing
d2696cdd1e Fix build hook error for libstore library users
A library shouldn't require changes to the caller's argument handling,
especially if it doesn't have to, and indeed we don't have to.

This changes the lookup order to prioritize the hardcoded path to nix
if it exists. The static executable still finds itself through /proc
and the like.
2023-06-15 14:32:00 +02:00
Valentin Gagarin
c453719d6e rename files referring to antiquotation
since we renamed this to string interpolation, file names should be
fixed up as well
2023-06-15 02:29:31 +02:00
John Ericson
946cd9e3f9
Merge pull request #8351 from obsidiansystems/delete-profiles-tests-docs
Expanding tests and docs relating to deleting profiles
2023-06-15 01:47:21 +02:00
John Ericson
d4a2ced9cb Split out nix-collect-garbage -d test to new file
Good for test parallelism, and separation of concerns (core GC vs
profiles deleting).
2023-06-14 19:01:07 -04:00
John Ericson
ca5752d4fa Add another case to the nix-collect-garbage -d test 2023-06-14 19:01:07 -04:00
Michal Sojka
a0c4d58549 Remove RegisterPrimOp constructor without support for documentation
The remaining constructor RegisterPrimOp::RegisterPrimOp(Info && info)
allows specifying the documentation in .args and .doc members of the
Info structure.

Commit 8ec1ba0210 removed all uses of the removed constructor in the
nix binary. Here, we remove the constructor completely as well as its
use in a plugin test. According to #8515, we didn't promis to maintain
compatibility with external plugins.

Fixes #8515
2023-06-14 22:37:52 +02:00
John Ericson
05eb06a1de
Merge pull request #8490 from flox/stdin_handling
fix: Do not apply default installables when using --stdin
2023-06-14 20:41:22 +02:00
John Ericson
ff905cb796
Merge pull request #4803 from ShamrockLee/nix-channel-list-generations
Add `nix-channel --list-generations`
2023-06-14 18:30:35 +02:00
John Ericson
61a3e1f2e2
Merge pull request #4282 from tweag/fix-ca-hash-rewriting
fix the hash rewriting for ca-derivations
2023-06-14 18:25:00 +02:00
Eelco Dolstra
1ad3328c5e Allow tarball URLs to redirect to a lockable immutable URL
Previously, for tarball flakes, we recorded the original URL of the
tarball flake, rather than the URL to which it ultimately
redirects. Thus, a flake URL like
http://example.org/patchelf-latest.tar that redirects to
http://example.org/patchelf-<revision>.tar was not really usable. We
couldn't record the redirected URL, because sites like GitHub redirect
to CDN URLs that we can't rely on to be stable.

So now we use the redirected URL only if the server returns the
`x-nix-is-immutable` or `x-amz-meta-nix-is-immutable` headers in its
response.
2023-06-13 14:17:45 +02:00
David McFarland
5454fdcceb Add test of explicit ssh control path in nix-copy test
This highlights a problem caused by SSHMaster::isMasterRunning returning
false when NIX_SSHOPTS contains -oControlPath.
2023-06-13 00:54:52 -03:00
Max
7f56f0aed9 Merge remote-tracking branch 'nixos/master' 2023-06-12 17:43:09 +02:00
Tom Bereknyei
0e3849dc65 test: add test for non-defaulting for stding installable input 2023-06-12 08:40:17 -04:00
Eelco Dolstra
381a32981b
Merge branch 'master' into angerman/mac-fix-recursive-nix 2023-06-09 13:06:47 +02:00
Andrea Bedini
3c78920f73
Parse TOML timestamps (#8120)
Currently `fromTOML` throws an exception when encountering a timestamp
since the Nix language lacks a way to represent them.

This patch changes this beaviour and makes `fromTOML` parse timestamps as
attrsets of the format

  { _type = "timestamp"; value = "1979-05-27T07:32:00Z"; }

This is guarded by an experimental feature flag to leave room for iterating on the representation.
2023-06-09 11:53:18 +02:00
Eelco Dolstra
f5c6b29940 Fix SourcePath::resolveSymlinks()
This fixes handling of symlinks that start with '..', and symlink
targets that contain symlinks themselves.
2023-06-06 11:24:10 +02:00
Max
518ce77742 Merge remote-tracking branch 'nixos/master' 2023-06-04 15:46:00 +02:00
Théophane Hufschmitt
0101ce0d96 Test nix-channel --list-generations
Rough test, but the feature is a fairly trivial addition on top of
`nix-profile --list-generations`, so it should be enough
2023-06-02 10:21:30 +02:00
polykernel
a382919d7d
primops: lazy evaluation of replaceStrings replacements
The primop `builtins.replaceStrings` currently always strictly evaluates the
replacement strings, however time and space are wasted for their computation
if the corresponding pattern do not occur in the input string. This commit
makes the evaluation of the replacement strings lazy by deferring their
evaluation to when the corresponding pattern are matched and memoize the result
for efficient retrieval on subsequent matches.

The testcases for replaceStrings was updated to check for lazy evaluation
of the replacements. A note was also added in the release notes to
document the behavior change.
2023-05-25 18:35:23 -04:00
Théophane Hufschmitt
d0cecbe877 Disable the fetchClosure test for old daemons
Broken because of the change introduced by #4282
2023-05-24 15:35:46 +02:00
Guillaume Maudoux
f0233f3a3f Further refactor linux-sandbox.sh and fix tee usage 2023-05-23 10:05:56 +02:00
Peter Becich
a420ccc6a8
nix flake check: skip derivations for foreign systems (#7759)
`nix flake show` now skips derivations for foreign systems: https://github.com/NixOS/nix/pull/6988

This commit borrows from that to implement the same behavior for `nix flake check`.

See "nix flake check breaks on IFD in multi-platform flake" https://github.com/NixOS/nix/issues/4265
2023-05-23 06:59:44 +02:00
Guillaume Maudoux
b9c2f834ee Check exact error codes in linux-sandbox.sh 2023-05-22 23:39:31 +02:00
Max
16e733360f Merge remote-tracking branch 'nixos/master' 2023-05-22 16:28:21 +02:00
Guillaume Maudoux
5a98dd0b39 Add tests for bind mount of SSL certs in sandbox 2023-05-22 02:32:09 +02:00
Max
80c164b001 Merge remote-tracking branch 'nixos/master' 2023-05-19 15:07:50 +02:00
John Ericson
32dc77ba5d
Merge pull request #8349 from tweag/fix-control-master
Fix ControlMaster behaviour
2023-05-17 12:17:09 -04:00
Alexander Bantyev
992e2ed0cf
Add a test for ControlMaster 2023-05-17 11:34:45 +04:00
John Ericson
5fd161189d
Merge pull request #8346 from tweag/fix-nix-profile-install-conflict-segfault
Fix the segfault on `nix profile install` with conflict
2023-05-16 15:49:43 -04:00
John Ericson
0a715ff9cf
Merge pull request #8154 from tweag/delete-old-on-all-profiles-dir
undefined
2023-05-16 09:51:41 -04:00
Théophane Hufschmitt
e997512523 Fix the printing of the installables on nix profile install conflict
- If the element comes from a flake, print the full flakeref (with the
  fragment part) and not just the reference to the flake itself
- If the element doesn't come from a flake, print its store path(s)

This is a bit too verbose, but has the advantages of being correct (and
not crashing), so it's strictly better than the previous situation

Fix https://github.com/NixOS/nix/issues/8284
2023-05-16 11:41:58 +02:00
John Ericson
754ced4a3f Avoid out links in supplementary groups test
This gets in the way of the tests running in parallel.
2023-05-15 17:49:28 -04:00
John Ericson
d8ef0c9495 Add some tests for drop-supplementary-groups 2023-05-15 17:41:51 -04:00
John Ericson
d2162e7acd Make more string values work as installables
As discussed in #7417, it would be good to make more string values work
as installables. That is to say, if an installable refers to a value,
and the value is a string, it used to not work at all, since #7484, it
works somewhat, and this PR make it work some more.

The new cases that are added for `BuiltPath` contexts:

- Fixed input- or content-addressed derivation:

  ```
  nix-repl> hello.out.outPath
  "/nix/store/jppfl2bp1zhx8sgs2mgifmsx6dv16mv2-hello-2.12"

  nix-repl> :p builtins.getContext hello.out.outPath
  { "/nix/store/c7jrxqjhdda93lhbkanqfs07x2bzazbm-hello-2.12.drv" = { outputs = [ "out" ]; }; }

  The string matches the specified single output of that derivation, so
  it should also be valid.

- Floating content-addressed derivation:

  ```
  nix-repl> (hello.overrideAttrs (_: { __contentAddressed = true; })).out.outPath
  "/1a08j26xqc0zm8agps8anxpjji410yvsx4pcgyn4bfan1ddkx2g0"

  nix-repl> :p builtins.getContext (hello.overrideAttrs (_: { __contentAddressed = true; })).out.outPath
  { "/nix/store/qc645pyf9wl37c6qvqzaqkwsm1gp48al-hello-2.12.drv" = { outputs = [ "out" ]; }; }
  ```

  The string is not a path but a placeholder, however it also matches
  the context, and because it is a CA derivation we have no better
  option. This should also be valid.

We may also want to think about richer attrset based values (also
discussed in that issue and #6507), but this change "completes" our
string-based building blocks, from which the others can be desugared
into or at least described/document/taught in terms of.

Progress towards #7417

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-05-15 09:03:38 -04:00
Théophane Hufschmitt
e97e9e9f00 test the garbage collection with the old profile dir
Regression test for https://github.com/NixOS/nix/issues/8294
2023-05-15 11:58:49 +02:00
John Ericson
a93110ab19 Fix nix print-dev-env & nix develop with drv paths
Fixes #8309

This regression was because both `CmdDevelop` and `CmdPrintDevEnv` were
switched to be `InstallableValueCommand` subclasses, but actually
neither should have been.

The `nixpkgsFlakeRef` method should indeed not be on the base
installable class, because "flake refs" and "nixpkgs" are not
installable-wide notions, but that doesn't mean these commands should
only accept installable values.
2023-05-10 11:29:45 -04:00
John Ericson
53a1354acf
Merge pull request #3959 from obsidiansystems/ca-drv-exotic
Derivations can output "text-hashed" data
2023-05-10 10:41:59 -04:00
Théophane Hufschmitt
85ff212051
Merge pull request #7721 from yorickvP/post-build-hook
Also pass unwanted outputs to post-build-hook
2023-05-10 14:30:42 +02:00
Alex Ameen
82d1d74a85
quote subshell expansion in tests/eval.sh 2023-05-09 10:06:26 -05:00
Alex Ameen
82296f8113
prevent double quotation 2023-05-09 09:59:18 -05:00
Alex Ameen
b72bc4a972
libexpr: quote reserved keys when printing
This fixes a bug in commands like `nix eval' which would emit invalid attribute
sets if they contained reserved keywords such as "assert", "let", etc.

These keywords will not be quoted when printed, making them valid expressions.
All keywords recognized by the lexer are quoted except "or", which does not
require quotation.
2023-05-09 09:45:12 -05:00
John Ericson
b5d9ef0a4c
Merge pull request #3921 from obsidiansystems/trustless-remote-builder-simple
Trustless remote building for input-addressed drvs
2023-05-08 10:43:37 -04:00
John Ericson
278c94d607 Rename a few things in new tests
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-05-08 08:01:58 -04:00
Yorick van Pelt
d1ff33d2d6
tests/post-hook: remove TODO and --derivation upload 2023-05-08 12:58:59 +02:00
Yorick van Pelt
5e332aa503
tests: copying only the out paths is not enough information for CA 2023-05-08 12:58:59 +02:00
Yorick van Pelt
869fb1a2f6
tests/post-hook: test to see if all outputs are passed
fe5509df caused only wanted outputs to be passed to the
post-build-hook, which resulted in paths being built
without ever going into the hook.

This commit adds a (currently failing) test for this.
2023-05-08 12:43:56 +02:00
Max
aaba1f91e7 README: update 2023-05-05 00:41:06 +02:00
Max
3822d33277 tests: disable some tests that fail with Nix Super 2023-05-04 23:37:56 +02:00
Max
a6bf675bc8 Merge remote-tracking branch 'nixos/master' 2023-05-04 01:55:40 +02:00
Moritz Angermann
0e18254aa8
Fix shutdown behavior and resource management for recursive-nix on macOS
Previously, we relied on the `shutdown()` function to terminate `accept()`
calls on a listening socket. However, this approach did not work on macOS as
the waiting `accept()` call is not considered a connected socket, resulting in
an `ENOTCONN` error. Instead, we now close the listening socket to terminate
the `accept()` call.

Additionally, we fixed a resource management issue where we set the
`daemonSocket` variable to -1, triggering resource cleanup and causing the
`stopDaemon` function to be called twice. This resulted in errors as the socket
was already closed by the time the second `stopDaemon` call was made. Instead of
setting `daemonSocket` to -1, we now release the socket using the `release()`
method on a unique pointer. This properly transfers ownership and allows for
correct resource cleanup.

These changes ensure proper behavior and resource management for the
recursive-nix feature on macOS.
2023-04-25 09:39:05 +08:00
Eelco Dolstra
01232358ff Merge remote-tracking branch 'origin/master' into source-path 2023-04-24 13:20:36 +02:00
John Ericson
969def696a Fix typo in tests 2023-04-19 20:47:23 -04:00
John Ericson
e26662709e Add a more interesting test
In this one, we don't just output an existing derivation as is, but
modify it first.
2023-04-19 20:36:33 -04:00
John Ericson
3eb343754e Move test/recursive.sh nix expr to file
I found it hard to read as a big string literal.
2023-04-19 19:36:05 -04:00
John Ericson
f3a31b14db Make tests/dyn-drv test dir 2023-04-19 18:49:50 -04:00
John Ericson
61d3e64fd0 Require daemon version for text hashing test 2023-04-19 17:24:55 -04:00
John Ericson
76baaeb341 Merge remote-tracking branch 'upstream/master' into ca-drv-exotic 2023-04-19 11:32:14 -04:00
John Ericson
668377f217 TextHashMethod -> TextIngestionMethod, gate with XP feature
I suppose we can use `dynamic-derivations` for the few things we neeed.
2023-04-17 19:02:45 -04:00
John Ericson
f56c4a5bdf Merge remote-tracking branch 'upstream/master' into ca-drv-exotic 2023-04-17 18:10:12 -04:00
John Ericson
aa74c7b0bc Gate experimental features in DerivationOutput::fromJSON
This is an entry point for outside data, so we need to check enabled
experimental features here.
2023-04-17 17:36:12 -04:00
John Ericson
ab5ca608bf Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2023-04-17 13:54:48 -04:00
Robert Hensing
64ee02890c
Merge pull request #8230 from obsidiansystems/daemon-trust-override
Experimentally allow forcing `nix-daemon` trust; use this to test
2023-04-17 19:43:41 +02:00
John Ericson
d41e1bed5e Experimentally allow forcing nix-daemon trust; use this to test
We finally test the status quo of remote build trust in a number of
ways. We create a new experimental feature on `nix-daemon` to do so.

PR #3921, which improves the situation with trustless remote building,
will build upon these changes. This code / tests was pull out of there
to make this, so everything is easier to review, and in particular we
test before and after so the new behavior in that PR is readily apparent
from the testsuite diff alone.
2023-04-17 13:06:21 -04:00
John Ericson
2c8475600d Fix some issues with experimental config settings
Issues:

1. Features gated on disabled experimental settings should warn and be
   ignored, not silently succeed.

2. Experimental settings in the same config "batch" (file or env var)
   as the enabling of the experimental feature should work.

3. For (2), the order should not matter.

These are analogous to the issues @roberth caught with my changes for
arg handling, but they are instead for config handling.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-04-17 12:41:04 -04:00
John Ericson
e12efa3654 Merge remote-tracking branch 'upstream/master' into ca-drv-exotic 2023-04-17 10:16:57 -04:00
John Ericson
e95db8f2b9 nix-testing -> daemon-trust-override
And only enable in the tests that need it. This makes it less of a
sledgehammer.
2023-04-17 09:35:43 -04:00
John Ericson
b1343e8ad1 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2023-04-17 09:27:17 -04:00
Robert Hensing
cb2615cf47 Merge remote-tracking branch 'upstream/master' into source-path 2023-04-17 11:41:50 +02:00
Robert Hensing
9af9c260fc
Merge pull request #8193 from hercules-ci/dry-strings
Deduplicate string literal rendering, fix 4909
2023-04-17 11:19:40 +02:00
John Ericson
615c25b0dd Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2023-04-12 00:00:12 -04:00
John Ericson
450e5ec618 Do not gate or hide experimental settings
This is somewhat hacky fix just for 2.15. I unintentionally hid them
from the manual, when no one wanted to hide them that (including
myself). I also required the experimental feature to be enabled in an
order-dependent way, which is not good.

The simplest fix for this immanent release is just to always show them,
and always allow them to be set.

Effectively undoes some changes from aa663b7e89
2023-04-11 10:56:48 -04:00
Eelco Dolstra
45ca4e6432
Merge pull request #8158 from tweag/harden-profiles-test
Fix the flaky `nix-profile` test
2023-04-11 12:12:09 +02:00
Robert Hensing
4e0804c920 Deduplicate string literal rendering, fix 4909 2023-04-09 22:42:20 +02:00
John Ericson
51c8ffbc28 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2023-04-08 18:49:26 -04:00
John Ericson
38ae7c2891 Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2023-04-07 11:13:35 -04:00
John Ericson
6e1e15ffec Fix it! 2023-04-07 11:13:23 -04:00
John Ericson
c036de086e Merge remote-tracking branch 'upstream/master' into trustless-remote-builder-simple 2023-04-07 09:40:36 -04:00
John Ericson
59e0728714 Create nix derivation add command
Also refine `nix derivation show`'s docs very slightly.
2023-04-07 08:34:58 -04:00
John Ericson
2b98af2e62 nix show-derivation -> nix derivation show 2023-04-07 08:34:58 -04:00
matthewcroughan
9207f94582 Add Store::isTrustedClient()
This function returns true or false depending on whether the Nix client
is trusted or not. Mostly relevant when speaking to a remote store with
a daemon.

We include this information in `nix ping store` and `nix doctor`

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2023-04-06 19:59:57 -04:00
Eelco Dolstra
94812cca98 Backport SourcePath from the lazy-trees branch
This introduces the SourcePath type from lazy-trees as an abstraction
for accessing files from inputs that may not be materialized in the
real filesystem (e.g. Git repositories). Currently, however, it's just
a wrapper around CanonPath, so it shouldn't change any behaviour. (On
lazy-trees, SourcePath is a <InputAccessor, CanonPath> tuple.)
2023-04-06 13:15:50 +02:00
Théophane Hufschmitt
7076d37047 Fix the flaky nix-profile test
Exclude the `error (ignored)` from the message that is checked by the
install conflict test.

Fix https://github.com/NixOS/nix/issues/8140
2023-04-03 20:47:21 +02:00
Eelco Dolstra
81491e1379
Merge pull request #8042 from lheckemann/alt-lockfiles
Allow specifying alternative paths for reading/writing flake locks
2023-04-03 19:28:09 +02:00
John Ericson
4a0b893d5e Stuctured command stability
Prior to this, there was an ad-hoc whitelist in `main.cc`. Now, every
command states its stability.

In a future PR, we will adjust the manual to take advantage of this new
information in the JSON.
(It will be easier to do that once we have some experimental feature
docs to link too; see #5930 and #7798.)
2023-04-03 11:48:21 -04:00
Robert Hensing
f3a6de6ba9
Merge pull request #8148 from hercules-ci/fix-issue-8119-printValue-tBlackhole-abort
eval: Fix crash on missing printValue tBlackhole case
2023-04-03 16:05:55 +02:00
Robert Hensing
1c55544a42 eval: Fix crash on missing printValue tBlackhole case
Fixes #8119
2023-04-03 15:32:42 +02:00
Théophane Hufschmitt
70bb7b7289
Merge pull request #7610 from obsidiansystems/gate-default-settings
Punt on improper global flags for now
2023-04-03 14:02:45 +02:00
John Ericson
f7f44f7c96 Merge commit 'aa99005004bccc9be506a2a2f162f78bad4bcb41' into ca-drv-exotic 2023-04-01 15:15:32 -04:00
Théophane Hufschmitt
3f362f1c07 test: Make the installer work on old rhel versions
`scp -r` doesn't seem to behave properly there, so tarpipe ftw!
2023-03-31 15:29:50 +02:00
Théophane Hufschmitt
4e7a78ca97 test: Don't add a channel for the force-no-daemon installer variant
Was probably an overlook of when the tests were first added, but that
now messes-up with the `nix-channel --update` that happens down the line
(and can't access the network since we're inside a Nix build)
2023-03-31 15:27:02 +02:00
Théophane Hufschmitt
e32ca3cf16
Merge pull request #8018 from tweag/ssh-password-prompt
SSH: don't erase password prompt if it is displayed
2023-03-31 12:06:10 +02:00
Eelco Dolstra
237587bc0a
Merge pull request #8084 from edolstra/store-docs
Auto-generate store documentation
2023-03-27 15:46:18 +02:00
John Ericson
1d539aa287
Merge pull request #8073 from tweag/fix-root-channels-location
Fix root channels location
2023-03-27 09:37:53 -04:00
John Ericson
7c4dea3cf3 Punt on improper global flags for now
See the note in the test.

We don't want these flags showing up for commands where they are
irrelevant.

Eventually, this needs a proper fix, but it need not be a blocker for
stabilize: for a quick-n-dirty punt, just put these flags behind the
`nix-command` unstable feature.

This is fine because they are only relevant for commands which we don't
need to stabilize for a while.
2023-03-27 09:21:50 -04:00
John Ericson
570829d67e
Merge pull request #7609 from obsidiansystems/hide-experimental-settings
Hide experimental settings
2023-03-27 09:19:29 -04:00