Commit graph

542 commits

Author SHA1 Message Date
Eelco Dolstra
3026613893
Merge pull request #10675 from edolstra/zip-symlinks
Handle zip files containing symlinks
2024-05-15 22:07:22 +02:00
Cole Helbling
06e13465c5 tests/functional: test both clis warn on unknown settings 2024-05-15 12:25:03 -07:00
Eelco Dolstra
1da18e85ba tests/functional/common/init.sh: Make $TEST_ROOT writable before removing it
$TEST_ROOT typically contains read-only files/directories (e.g. the
Nix store). So we have to make it writable first.
2024-05-14 16:23:08 +02:00
Valentin Gagarin
7822ecbadf tests: always clean the test directory
previously the test directory could have been left untouched before executing
a test when `init.sh` was not run - and sometimes it isn't
supposed to be run - which made the test suite highly stateful and thus
behaving surprisingly on multiple runs.
2024-05-13 15:19:49 +02:00
Valentin Gagarin
33ca905cdb tests: simplify initialisation and wiring
pararameterisation is not actually needed the way things are currently
set up, and it confused me when trying to understand what the code does.

all but one test sources vars-and-functions.sh, which nominally only
defines variables, but in practice is always coupled with the actual
initialisation. while the cleaner way of making this more legible would
be to source variables and initialisation separately, this would produce
a huge diff.

the change requires a few small fixes to keep the tests working:

- only create test home directory during initialisation

  that vars-and-functions.sh wrote to the file system seems not write

- fix creation of the test directory

  due to statefulness, the test home directory was implicitly creating
  the test root, too. decoupling that made it apparent that this was
  probably not intentional, and certainly confusing.

- only source vars-and-functions.sh if init.sh is not needed

  there is one test case that only needs a helper function but no
  initialisation side effects

- remove some unnecessary cleanups and split parts of re-used test code

  there were confusing bits in how initialisation code was repurposed,
  which break if trying to refactor the outer layers naively...
2024-05-13 15:19:49 +02:00
Eelco Dolstra
9951e14ae0 Handle zip files containing symlinks
In streaming mode, libarchive doesn't handle symlinks in zip files
correctly. So write the entire file to disk so libarchive can access
it in random-access mode.

Fixes #10649. This was broken in cabee98152.
2024-05-09 19:33:09 +02:00
Eelco Dolstra
79c7d6205c Support unit prefixes in configuration settings
E.g. you can now say `--min-free 1G`.
2024-05-08 21:11:09 +02:00
Eelco Dolstra
da3381d51f
Merge pull request #10465 from edolstra/remove-locked
Fetcher cache cleanups
2024-05-06 21:32:26 +02:00
Théophane Hufschmitt
9bd1191fcc libstore: check additionalSandboxProfile
Make sure that `extraSandboxProfile` is set before we check whether it's
empty or not (in the `sandbox=true` case).

Also adds a test case for this.

Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
2024-05-06 17:37:08 +02:00
Eelco Dolstra
1f41682217
Update tests/functional/flakes/flakes.sh
Co-authored-by: John Ericson <git@JohnEricson.me>
2024-04-30 18:10:16 +02:00
Eelco Dolstra
f29a220b70 Test that the root of a tree produces /nix/store/<hash1>-<hash2>-source 2024-04-30 17:25:35 +02:00
Eelco Dolstra
503be57bbd Test baseNameOf behaviour on the root of a flake 2024-04-30 15:43:33 +02:00
Eelco Dolstra
458441c637 Test dirOf behaviour on the root of a flake 2024-04-30 15:34:38 +02:00
Eelco Dolstra
ff107d9d03 Input::fetchToStore(): Don't try to substitute
Having a narHash doesn't mean that we have the other attributes
returned by the fetcher (such as lastModified or rev). For instance,

   $ nix flake metadata github:NixOS/patchelf/7c2f768bf9601268a4e71c2ebe91e2011918a70f
   Last modified: 2024-01-15 10:51:22

but

   $ nix flake metadata github:NixOS/patchelf/7c2f768bf9601268a4e71c2ebe91e2011918a70f?narHash=sha256-PPXqKY2hJng4DBVE0I4xshv/vGLUskL7jl53roB8UdU%3D
   (does not print a "Last modified")

The latter only happens if the store path already exists or is
substitutable, which made this impure behaviour unpredictable.

Fixes #10601.
2024-04-26 16:41:33 +02:00
Robert Hensing
750bcaa330 Fix fetchGit nested submodules 2024-04-22 16:41:40 +02:00
Max
2d0b0537fe Merge remote-tracking branch 'nixos/master' 2024-04-19 16:42:33 +02:00
Théophane Hufschmitt
d2a07a96ba
Merge pull request #10467 from edolstra/nix-shell-symlink
nix shell: Handle output paths that are symlinks
2024-04-16 12:31:14 +02:00
Eelco Dolstra
d084c1cb41 Remove the "locked" flag from the fetcher cache
This also reworks the Mercurial fetcher (which was still using the
old cache interface) to have two distinct cache mappings:

* A ref-to-rev mapping, which is store-independent.
* A rev-to-store-path mapping.
2024-04-15 13:03:27 +02:00
Théophane Hufschmitt
cef677ddbc Test the inclusion of transitive symlinks in the sandbox 2024-04-12 16:10:22 +02:00
Théophane Hufschmitt
acbb1523c1 Fix the access of symlinks to host files in the sandbox
https://github.com/NixOS/nix/pull/10456 fixed the addition of symlink
store paths to the sandbox, but also made it so that the hardcoded
sandbox paths (like `/etc/hosts`) were now bind-mounted without
following the possible symlinks. This made these files unreadable if
there were symlinks (because the sandbox would now contain a symlink to
an unreachable file rather than the underlying file).
In particular, this broke FOD derivations on NixOS as `/etc/hosts` is a
symlink there.

Fix that by canonicalizing all these hardcoded sandbox paths before
adding them to the sandbox.
2024-04-12 15:57:53 +02:00
Théophane Hufschmitt
da1e977bf4
Merge pull request #10456 from NixOS/fixpermdeniedbind
Fix adding symlink to the sandbox paths
2024-04-11 13:41:49 +02:00
Eelco Dolstra
26a4688a86 nix shell: Test that store paths cannot link outside of the store 2024-04-11 11:06:35 +02:00
Eelco Dolstra
9d50f57fa3 Doh 2024-04-11 09:00:47 +02:00
Théophane Hufschmitt
def00d7b52
Merge pull request #10455 from tie/structured-attrs-get-env
Do not rely on $stdenv/setup to set output variables
2024-04-11 06:48:36 +02:00
Eelco Dolstra
85b9f4ef4f nix shell: Handle output paths that are symlinks
This requires moving resolveSymlinks() into SourceAccessor. Also, it
requires LocalStoreAccessor::maybeLstat() to work on parents of the
store (to avoid an error like "/nix is not in the store").

Fixes #10375.
2024-04-10 23:49:19 +02:00
Ivan Trubach
664532c533 Do not rely on $stdenv/setup to set output variables
Instead of relying on setup script to set output variables when
structured attributes are enabled, iterate over the values of an
outputs associative array.

See also
374fa3532e/pkgs/stdenv/generic/setup.sh (L23-L26)
2024-04-10 19:50:19 +03:00
Théophane Hufschmitt
a268c0de71
Merge pull request #10413 from Ma27/path-info-all-binary-cache
path-info: print correct path when using `nix path-info --store file://... --all --json`
2024-04-10 15:25:50 +02:00
Théophane Hufschmitt
872d93eb13 Add a test for depending on a symlink store path
Regression test for https://github.com/NixOS/nix/issues/9579
2024-04-10 15:17:39 +02:00
John Ericson
93d68e18e5 Make outputHashAlgo accept "nar", stay in sync
Now that we have a few things identifying content address methods by
name, we should be consistent about it.

Move up the `parseHashAlgoOpt` for tidiness too.

Discussed this change for consistency's sake as part of #8876

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-04-09 16:17:23 -04:00
Eelco Dolstra
737ce5e81f Actually run the Mercurial tests 2024-04-08 15:21:57 +02:00
Théophane Hufschmitt
bd8c276ddb Improve the config check output for stores that don't know about trust
Make it proper english
2024-04-08 11:02:39 +02:00
Jade Lovelace
dea23c3c9b "but doctor, I AM the untrusted store": nix doctor had wrong trustedness
This probably snuck in in a refactor using truthiness or so. The
trustedness flag was having the optional fullness checked, rather than
the actual contained trust level.

Also adds some tests.

```
m1@6876551b-255d-4cb0-af02-8a4f17b27e2e ~ % nix store ping
warning: 'nix store ping' is a deprecated alias for 'nix store info'
Store URL: daemon
Version: 2.20.4
Trusted: 0
m1@6876551b-255d-4cb0-af02-8a4f17b27e2e ~ % nix doctor
warning: 'doctor' is a deprecated alias for 'config check'
[PASS] PATH contains only one nix version.
[PASS] All profiles are gcroots.
[PASS] Client protocol matches store protocol.
[INFO] You are trusted by store uri: daemon
```
2024-04-07 22:43:02 -07:00
John Ericson
bd7c26bc7b Add comment explaining LIBMOUNT_FORCE_MOUNT2=always 2024-04-07 21:55:19 -04:00
cidkidnix
e73dc0e938 Use LIBMOUNT_FORCE_MOUNT2=always to workaround new mount API issues 2024-04-05 16:43:14 -05:00
Max
05c190487d Merge remote-tracking branch 'nixos/master' 2024-04-05 22:42:41 +02:00
John Ericson
c99c80f075 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2024-04-05 16:32:02 -04:00
Maximilian Bosch
c80cd6bb06
path-info: print correct path when using nix path-info --store file://... --all --json
When querying all paths in a binary cache store, the path's representation
is `<hash>-x` (where `x` is the value of `MissingName`) because the .narinfo
filenames only contain the hash.

Before cc46ea1630 this worked correctly,
because the entire path info was read and the path from this
representation was printed, i.e. in the form `<hash>-<name>`. Since then
however, the direct result from `queryAllValidPaths()` was used as `path`.

Added a regression test to make sure the behavior remains correct.
2024-04-05 21:29:55 +02:00
Eelco Dolstra
bf86b939f8
Merge pull request #10391 from edolstra/home-symlink
Handle the case where a parent of ~/.nix-defexpr is a symlink
2024-04-03 17:20:35 +02:00
John Ericson
36d92dc80a
Merge pull request #10316 from dottharun/derivation-output-test
test: for derivation output selection with `.`
2024-04-03 10:48:04 -04:00
Eelco Dolstra
09551fabd0 Handle the case where a parent of ~/.nix-defexpr is a symlink
Fixes https://github.com/DeterminateSystems/nix-installer/issues/912 and probably #10247.
2024-04-03 16:46:23 +02:00
Tharun T
8d84de455e outputSpecified doesnt exit in top attr-set 2024-04-03 08:26:42 +05:30
Cameron Dart
ba0bd8fae6 Add functional tests for include directive in nix config file 2024-04-02 18:10:36 -07:00
Théophane Hufschmitt
290be6c906
Merge pull request #10345 from edolstra/fix-chroot-flakes
Fix flake evaluation in chroot stores
2024-04-02 17:49:45 +02:00
Théophane Hufschmitt
b72e1c79da
Merge pull request #10299 from detroyejr/remove-repl-flake
flakes: remove experimental repl-flake
2024-03-29 15:06:23 +01:00
Eelco Dolstra
8930772864 Add regression test for #10331, #10267 2024-03-29 13:20:50 +01:00
Eelco Dolstra
dffc22f30f Rename local-store.sh -> chroot-store.sh 2024-03-29 12:51:14 +01:00
Théophane Hufschmitt
de101417eb
Merge pull request #10305 from hercules-ci/addErrorContext-is-great
Always print addErrorContext
2024-03-29 09:10:09 +01:00
Robert Hensing
ef26133df3
Merge pull request #10337 from Cyclic4179/patch-1
`nix shell` shebang: support O'Caml comments
2024-03-28 19:54:03 +01:00
Tharun T
52359ca00a move test to correct file 2024-03-28 02:20:33 +05:30
John Ericson
77205b2042 Allow for ergnomically putting Unix-only files in subdirs by creating INLCUDE_$(pkg) vars
Separate platform-specific files will allow avoiding a lot of CPP.
2024-03-27 12:01:59 -04:00
Robert Hensing
981c309057 Remove trace item: while calling the 'addErrorContext' builtin 2024-03-27 16:28:04 +01:00
Robert Hensing
bebacc475c Always print addErrorContext traces 2024-03-27 16:28:04 +01:00
Cyclic4179
9f7b2b93ce
fixup! add flakes.sh test for shebang supported types of comments 2024-03-27 15:03:54 +01:00
Cyclic4179
6227cd06bd
add flakes.sh test for shebang supported types of comments 2024-03-27 14:55:02 +01:00
Théophane Hufschmitt
6c10cc0eda
Merge pull request #10312 from hercules-ci/add-build-dir
Add build-dir setting
2024-03-26 13:51:50 +01:00
Théophane Hufschmitt
1394d4e9c5
Merge pull request #10306 from hercules-ci/baseNameOf
Test and document `builtins.baseNameOf`, improve internal `baseNameOf()`
2024-03-26 09:59:27 +01:00
Robert Hensing
8b16cced18 Add build-dir setting 2024-03-25 19:48:59 +01:00
Robert Hensing
6221770c9d tests/functional: Add count() 2024-03-25 19:46:34 +01:00
Max
06eac000db fix search test 2024-03-25 18:46:03 +01:00
Max
ee50c7d033 adjust test output for eval-fail-function-apply-dollar 2024-03-25 18:33:38 +01:00
Max
fd78c9b000 Merge remote-tracking branch 'nixos/master' 2024-03-25 17:43:50 +01:00
Tharun T
53c15336b0 derivation output selection test 2024-03-25 07:43:31 +05:30
John Ericson
2a44b11f55 Fix git-hashing/simple.sh
I realized it was checking NAR hashes before of added objects, which
makes little sense --- we don't really care about ancillary NAR hashes.

Now, the bottom `nix store add` tests compare the CA field with a git
hash to hashes calculated by Git. This matches top `nix hash path` ones
in using git as a source of truth.
2024-03-23 22:08:52 -04:00
Robert Hensing
175afc7106 Test and document builtins.baseNameOf 2024-03-24 01:26:17 +01:00
detroyejr
b11dd58fe4 flakes: test to ensure we get an error if --file isn't used 2024-03-22 21:58:39 -04:00
John Ericson
18945e3f44 Merge branch 'master' into overlayfs-store 2024-03-18 16:43:33 -04:00
Bouke van der Bijl
60c2d15f5a git fetcher: use resolveRef for getting revision of reference
* Add regression test
* Fix 'no repo' test so it doesn't succeed if the data is still in cache
* Use git_revparse_single inside git-utils instead of reimplementing the same logic.
2024-03-14 14:04:51 +01:00
tomberek
a200ee6bf7
Merge pull request #9874 from pennae/error-reports
improve error reports somewhat
2024-03-08 10:51:14 -05:00
Eelco Dolstra
ae2bd460a8
Merge pull request #10166 from bobvanderlinden/profile-regex-all
profile: introduce --regex and --all
2024-03-08 15:59:24 +01:00
Bob van der Linden
91f068c193
profile: make --all exclusive 2024-03-07 21:07:03 +01:00
Bob van der Linden
7a4d5e89d3
profile: add --all option to match any package 2024-03-07 21:07:03 +01:00
Bob van der Linden
fb391ebc77
profile: add tests for not matching any packages 2024-03-07 21:07:03 +01:00
Bob van der Linden
9fac62435c
tests/functional: add assertStderr function
Currently there isn't a convenient way to check for multiline output. In
addition, these outputs will easily change and having a diff between the
expected an the actual output upon failures is convenient.
2024-03-07 21:07:03 +01:00
Bob van der Linden
87741dbd21
profile: add --regex option to match packages 2024-03-07 21:07:03 +01:00
Robert Hensing
73cf23275e
Merge pull request #10107 from tweag/repl-file-argument
Properly fail on flakerefs that don't point to a directory
2024-03-07 10:04:35 +01:00
pennae
855fd5a1bb diagnose "unexpected EOF" at EOF
this needs a string comparison because there seems to be no other way to
get that information out of bison. usually the location info is going to
be correct (pointing at a bad token), but since EOF isn't a token as
such it'll be wrong in that this case.

this hasn't shown up much so far because a single line ending *is* a
token, so any file formatted in the usual manner (ie, ending in a line
ending) would have its EOF position reported correctly.
2024-03-06 23:11:12 +01:00
pennae
2be6b14328 match line endings used by parser and error reports
the parser treats a plain \r as a newline, error reports do not. this
can lead to interesting divergences if anything makes use of this
feature, with error reports pointing to wrong locations in the input (or
even outside the input altogether).
2024-03-06 23:11:12 +01:00
pennae
1edd6fada5 report inherit attr errors at the duplicate name
previously we reported the error at the beginning of the binding
block (for plain inherits) or the beginning of the attr list (for
inherit-from), effectively hiding where exactly the error happened.

this also carries over to runtime positions of attributes in sets as
reported by unsafeGetAttrPos. we're not worried about this changing
observable eval behavior because it *is* marked unsafe, and the new
behavior is much more useful.
2024-03-06 23:11:12 +01:00
pennae
4147ecfb1c normalize formal order on ExprLambda::show
we already normalize attr order to lexicographic, doing the same for
formals makes sense. doubly so because the order of formals would
otherwise depend on the context of the expression, which is not quite as
useful as one might expect.
2024-03-06 23:11:12 +01:00
pennae
d384ecd553 keep copies of parser inputs that are in-memory only
the parser modifies its inputs, which means that sharing them between
the error context reporting system and the parser itself can confuse the
reporting system. usually this led to early truncation of error context
reports which, while not dangerous, can be quite confusing.
2024-03-06 23:11:12 +01:00
Eelco Dolstra
6b976a1898
Merge pull request #10143 from fricklerhandwerk/test-help
add tests for showing help
2024-03-04 20:29:03 +01:00
Valentin Gagarin
8d23847571
fix indentation
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-04 19:25:44 +01:00
Valentin Gagarin
8a6ef3bae5
less scary jq
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-04 19:25:28 +01:00
Valentin Gagarin
4b15ca2ffb add tests for showing help 2024-03-04 16:14:00 +01:00
Eelco Dolstra
438855952b
Merge pull request #10122 from edolstra/arg-from-file
Add --arg-from-file and --arg-from-stdin
2024-03-04 13:29:58 +01:00
Théophane Hufschmitt
8a9eec375c
Merge pull request #10073 from olmokramer/nix-flake-update-multiple-inputs
Accept multiple inputs in `nix flake update`
2024-03-04 09:13:26 +01:00
Olmo Kramer
b1ad729add
Add test for nix flake update with multiple inputs 2024-03-03 13:59:39 +01:00
Théophane Hufschmitt
11a1dcc43b Properly fail on flakerefs that don't point to a directory
Directly fail if a flakeref points to something that isn't a directory
instead of falling back to the logic of trying to look up the hierarchy
to find a valid flake root.

Fix https://github.com/NixOS/nix/issues/9868
2024-03-02 10:24:20 +01:00
Eelco Dolstra
ba9b6b29b7
Merge pull request #10111 from obsidiansystems/git-objects
Support symlinks properly with `git-hashing` experimental feature
2024-03-01 15:00:43 +01:00
Eelco Dolstra
8ce1f6800b Add --arg-from-stdin to read an argument from stdin 2024-03-01 14:39:42 +01:00
Eelco Dolstra
291b10c607 Add --arg-from-file for reading a string from a file 2024-03-01 14:35:27 +01:00
John Ericson
5a2985431c Revert "Revert "Merge pull request #9546 from NixOS/nixos-23.11""
This reverts commit d6d7d2cb46.
2024-02-29 14:52:31 -05:00
John Ericson
d6d7d2cb46 Revert "Merge pull request #9546 from NixOS/nixos-23.11"
This reverts commit 587c7dcb2b, reversing
changes made to 864fc85fc8.
2024-02-29 14:39:29 -05:00
John Ericson
9b506ff0c1 Activate hermetic.nix variation only for new layered store tests 2024-02-29 10:06:53 -05:00
John Ericson
cb4f85f11c Merge branch 'master' into overlayfs-store 2024-02-29 09:54:21 -05:00
Eelco Dolstra
b1586a6799
Merge pull request #10090 from bobvanderlinden/profile-error-install-twice
profile install: warn on installing package twice
2024-02-29 10:04:08 +01:00
Bob van der Linden
14adff1711
profile install: skip and warn on installing package twice 2024-02-29 08:29:38 +01:00
John Ericson
bcb5f235f9 Support symlinks properly with git-hashing experimental feature
Before, they would not be written to a file `FileSystemObjectSink`
correctly.
2024-02-28 12:24:45 -05:00
Théophane Hufschmitt
6147d27afb Bump the required daemon version for the git hashing tests
The required version check was a bit too lenient, and
`nixpkgs#nixUnstable` was considered valid while it didn't have the fix.
2024-02-28 07:11:22 +01:00
Théophane Hufschmitt
bbef03872b Bump the required daemon version for the impure-env test
The required version check was a bit too lenient, and
`nixpkgs#nixUnstable` was considered valid while it didn't have the fix.
2024-02-28 07:08:21 +01:00
John Ericson
201551c937 Add Git object hashing to the store layer
Part of RFC 133

Extracted from our old IPFS branches.

Co-Authored-By: Matthew Bauer <mjbauer95@gmail.com>
Co-Authored-By: Carlo Nucera <carlo.nucera@protonmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Florian Klink <flokli@flokli.de>
2024-02-27 11:27:34 -05:00
Robert Hensing
4c7f0ef6ca
Merge pull request #9847 from pennae/inherit-from-dedup
deduplicate inherit-from source expr work
2024-02-26 20:25:58 +01:00
pennae
cefd0302b5 evaluate inherit (from) exprs only once per directive
desugaring inherit-from to syntactic duplication of the source expr also
duplicates side effects of the source expr (such as trace calls) and
expensive computations (such as derivationStrict).
2024-02-26 19:07:08 +01:00
Rebecca Turner
fe6408b5df
Update snapshots 2024-02-22 17:58:55 -08:00
Théophane Hufschmitt
6a5210f48e
Merge pull request #9815 from NixOS/nix-hash-path
`nix hash path`, text hashing for `nix store add`, and preparatory refactors
2024-02-22 17:15:34 +01:00
Eelco Dolstra
f183eef4a0
Merge pull request #10060 from bobvanderlinden/pr-simple-profile-remove-suggestion
profile: suggest removal using profile entry name
2024-02-22 11:02:12 +01:00
Bob van der Linden
4ae5091716
nix profile: suggest removal using entry name
When a file conflict arises during a package install a suggestion is
made to remove the old entry. This was previously done using the
installable URLs of the old entry. These URLs are quite verbose and
often do not equal the URL of the existing entry.

This change uses the recently introduced profile entry name for the
suggestion, resulting in a simpler output.

The improvement is easily seen in the change to the functional test.
2024-02-21 21:58:36 +01:00
John Ericson
2080d89b87
Merge pull request #10038 from edolstra/tarball-git-cache
Use the Git cache for tarball flakes
2024-02-21 15:47:02 -05:00
John Ericson
efd36b49e8 nix hash path, and preperatory refactors
- `nix store add` supports text hashing

  With functional test ensuring it matches `builtins.toFile`.

- Factored-out flags for both commands

- Move all common reusable flags to `libcmd`

  - They are not part of the *definition* of the CLI infra, just a usag
    of it.

  - The `libstore` flag couldn't go in `args.hh` in libutil anyways,
    would be awkward for it to live alone

- Shuffle around `Cmd*` hierarchy so flags for deprecated commands don't
  end up on the new ones
2024-02-21 12:11:25 -05:00
Eelco Dolstra
071dd2b3a4 Input: Replace 'locked' bool by isLocked() method
It's better to just check whether the input has all the attributes
needed to consider itself locked (e.g. whether a Git input has an
'rev' attribute).

Also, the 'locked' field was actually incorrect for Git inputs: it
would be set to true even for dirty worktrees. As a result, we got
away with using fetchTree() internally even though fetchTree()
requires a locked input in pure mode. In particular, this allowed
'--override-input' to work by accident.

The fix is to pass a set of "overrides" to call-flake.nix for all the
unlocked inputs (i.e. the top-level flake and any --override-inputs).
2024-02-20 16:59:15 +01:00
Eelco Dolstra
0acd783190 Don't send settings that depend on disabled experimental features to the daemon
This fixes warnings like

   warning: Ignoring setting 'auto-allocate-uids' because experimental feature 'auto-allocate-uids' is not enabled
   warning: Ignoring setting 'impure-env' because experimental feature 'configurable-impure-env' is not enabled

when using the daemon and the user didn't actually set those settings.

Note: this also hides those settings from `nix config show`, but that
seems a good thing.
2024-02-20 15:28:42 +01:00
Théophane Hufschmitt
d2c6a93bd5
Merge pull request #10044 from edolstra/empty-git-repos
Handle empty Git repositories / workdirs
2024-02-20 14:01:23 +01:00
Eelco Dolstra
930b9c8269 PosixSourceAccessor: Support roots that are not directories
We have to support this for `fetchTree { type = "file" }` (and
probably other types of trees that can have a non-directory at the
root, like NARs).
2024-02-20 12:51:15 +01:00
Eelco Dolstra
b00f412f81 Remove bad.tar.xz check, since libarchive doesn't care 2024-02-20 12:51:15 +01:00
Eelco Dolstra
db012d1e63 tests/functional/tarball.sh: Fix invalid file:// URLs 2024-02-20 12:51:15 +01:00
Théophane Hufschmitt
6f4bb1b584
Merge pull request #10043 from edolstra/fix-readonly-fetchToStore
fetchToStore(): Don't always respect settings.readOnlyMode
2024-02-20 12:07:48 +01:00
Eelco Dolstra
7cb4d0c5b7 fetchToStore(): Don't always respect settings.readOnlyMode
It's now up to the caller whether readOnlyMode should be applied. In
some contexts (like InputScheme::fetch()), we always need to fetch.
2024-02-20 11:46:49 +01:00
Eelco Dolstra
9e762454cf Support empty Git repositories / workdirs
Fixes #10039.
2024-02-20 11:40:02 +01:00
John Ericson
d53c8901ef
Merge pull request #9985 from alois31/symlink-resolution
Restore `builtins.pathExists` behavior on broken symlinks
2024-02-16 09:24:03 -05:00
Théophane Hufschmitt
d857914e1a
Merge pull request #9931 from 9999years/pretty-printer
Pretty-print values in the REPL
2024-02-14 13:32:58 +01:00
Eelco Dolstra
2b946aad8e
Merge pull request #10006 from edolstra/fix-impure-fetchurl
<nix/fetchurl.nix>: Restore support for "impure = true"
2024-02-13 22:50:55 +01:00
Alois Wohlschlager
89e21ab4bd
Restore builtins.pathExists behavior on broken symlinks
Commit 83c067c0fa changed `builtins.pathExists`
to resolve symlinks before checking for existence. Consequently, if the path
refers to a symlink itself, existence of the target of the symlink (instead of
the symlink itself) was checked. Restore the previous behavior by skipping
symlink resolution in the last component.
2024-02-13 18:09:55 +01:00
John Ericson
8bebf9607c Split hash.sh test in two
Converting hashes and hashing files are pretty separate tasks, and more
test parallelism is better.
2024-02-13 09:44:14 -05:00
Eelco Dolstra
bb63bd50e6 <nix/fetchurl.nix>: Restore support for "impure = true" 2024-02-13 14:14:20 +01:00
pennae
ecf8b12d60 group inherit by source during Expr::show
for plain inherits this is really just a stylistic choice, but for
inherit-from it actually fixes an exponential size increase problem
during expr printing (as may happen during assertion failure reporting,
on during duplicate attr detection in the parser)
2024-02-12 13:58:29 +01:00
pennae
6c08fba533 use the same bindings print for ExprAttrs and ExprLet
this also has the effect of sorting let bindings lexicographically
rather than by symbol creation order as was previously done, giving a
better canonicalization in the process.
2024-02-12 13:35:00 +01:00
pennae
73065a400d add test for inherit expr printing 2024-02-12 13:32:33 +01:00
pennae
8669c02468 add test for inherit-from semantics 2024-02-12 13:32:33 +01:00
Théophane Hufschmitt
1ba9780cf5
Merge pull request #9834 from 9999years/structured-errors
Towards structured error classes
2024-02-08 20:00:25 +01:00
Théophane Hufschmitt
9b8b486091
Merge pull request #9933 from pennae/debugger-fix
fix debugger crashing while printing envs
2024-02-08 10:48:02 +01:00
Rebecca Turner
c0a15fb7d0
Pretty-print values in the REPL
Pretty-print values in the REPL by printing each item in a list or
attrset on a separate line. When possible, single-item lists and
attrsets are printed on one line, as long as they don't contain a nested
list, attrset, or thunk.

Before:
```
{ attrs = { a = { b = { c = { }; }; }; }; list = [ 1 ]; list' = [ 1 2 3 ]; }
```

After:
```
{
  attrs = {
    a = {
      b = {
        c = { };
      };
    };
  };
  list = [ 1 ];
  list' = [
    1
    2
    3
  ];
}
```
2024-02-05 13:23:38 -08:00
Eelco Dolstra
e190c20c33
Merge pull request #9926 from 9999years/fix-cycle-detection-in-nix-repl
Fix cycle detection in `nix repl`
2024-02-05 14:24:28 +01:00
pennae
5ccb06ee1b fix debugger crashing while printing envs
fixes #9932
2024-02-04 17:12:04 +01:00
Rodney Lorrimar
e1131b5927
print-dev-env: Avoid using unbound shellHook variable
Some tools which consume the "nix print-dev-env" rc script (such as
"nix-direnv") are sensitive to the use of unbound variables. They use
"set -u".

The "nix print-dev-env" rc script initially unsets "shellHook", then
loads variables from the derivation, and then evaluates "shellHook".
However, most derivations don't have a "shellHook" attribute.

So users get the error "shellHook: unbound variable". This can be
demonstrated with the command:

    nix print-dev-env nixpkgs#hello | bash -u

This commit changes the rc script to provide an empty fallback value
for the "shellHook" variable.

Closes: #7951 #8253
2024-02-04 13:57:13 +08:00
Rebecca Turner
770d2bc779
Key repeated values on attribute binding pointers, not value pointers
Closes #8672
2024-02-03 21:23:06 -08:00
Eelco Dolstra
e67458e5b8 Better test fix 2024-02-02 13:34:56 +01:00
Eelco Dolstra
05535be03a Fix test 2024-02-02 13:07:08 +01:00
Rebecca Turner
c6a89c1a16
libexpr: Support structured error classes
While preparing PRs like #9753, I've had to change error messages in
dozens of code paths. It would be nice if instead of

    EvalError("expected 'boolean' but found '%1%'", showType(v))

we could write

    TypeError(v, "boolean")

or similar. Then, changing the error message could be a mechanical
refactor with the compiler pointing out places the constructor needs to
be changed, rather than the error-prone process of grepping through the
codebase. Structured errors would also help prevent the "same" error
from having multiple slightly different messages, and could be a first
step towards error codes / an error index.

This PR reworks the exception infrastructure in `libexpr` to
support exception types with different constructor signatures than
`BaseError`. Actually refactoring the exceptions to use structured data
will come in a future PR (this one is big enough already, as it has to
touch every exception in `libexpr`).

The core design is in `eval-error.hh`. Generally, errors like this:

    state.error("'%s' is not a string", getAttrPathStr())
      .debugThrow<TypeError>()

are transformed like this:

    state.error<TypeError>("'%s' is not a string", getAttrPathStr())
      .debugThrow()

The type annotation has moved from `ErrorBuilder::debugThrow` to
`EvalState::error`.
2024-02-01 16:39:38 -08:00
Eelco Dolstra
1ee42c5b88 builtin:fetchurl: Ensure a fixed-output derivation
Previously we didn't check that the derivation was fixed-output, so
you could use builtin:fetchurl to impurely fetch a file.
2024-02-01 21:46:01 +01:00
John Ericson
31881d651a Merge remote-tracking branch 'upstream/master' into overlayfs-store 2024-02-01 11:07:47 -05:00
Eelco Dolstra
b36ff47e7c Resolve symlinks in a few more places
Fixes #9882.
2024-01-30 15:35:31 +01:00
Rebecca Turner
772897a1cd
Color diff output in tests/functional/lang tests
Use `diff --color=always` to print colored output for language test
failures. I've also flipped the arguments so that expected lines missing
from the actual output will be marked with a red `-` and additional
lines found in the actual output will be marked with a green `+`.
Previously it was the other way around, which was very confusing.
2024-01-26 10:08:56 -08:00
Robert Hensing
212ba69e6f
Merge pull request #9843 from hercules-ci/test-pr-9626
tests/functional/fetchGit.sh: Test fetchGit/fetchTree error message
2024-01-24 13:55:41 +01:00
Robert Hensing
6532dd50fc tests/functional/fetchGit.sh: Test fetchGit/fetchTree error message
Follow-up for https://github.com/NixOS/nix/pull/9626
176dcd5c61
2024-01-24 13:19:02 +01:00
Robert Hensing
5b7bfd2d6b
Merge pull request #9754 from 9999years/print-value-when-coercion-fails
Print the value in `error: cannot coerce` messages
2024-01-24 12:48:39 +01:00
tomberek
1c260fa6d1
Merge pull request #9481 from iFreilicht/disallow-nix-search-without-search-terms
nix search: Disallow empty regex
2024-01-23 20:59:52 -05:00
Rebecca Turner
83bb494a30
Print the value in error: cannot coerce messages
This extends the `error: cannot coerce a TYPE to a string` message
to print the value that could not be coerced. This helps with debugging
by making it easier to track down where the value is being produced
from, especially in errors with deep or unhelpful stack traces.
2024-01-23 15:15:41 -08:00
Rebecca Turner
cb7fbd4d83
Print value on type error
Adds the failing value to `value is <TYPE> while a <TYPE> is expected`
error messages.
2024-01-22 08:56:02 -08:00
John Ericson
202c5e2afc Start standardizing hash algo flags
Do this if we want to do `--hash-algo` everywhere, and not `--algo` for
hash commands.

The new `nix hash convert` is updated. Deprecated new CLI commands are
left as-is (`nix hash path` needs to be redone and is also left as-is).
2024-01-20 17:29:35 -05:00
John Ericson
356352c370 Add missing --hash-algo flag to nix store add 2024-01-19 23:11:18 -05:00
Eelco Dolstra
e6e160a075
Merge pull request #9632 from cole-h/nix-daemon-testing
Use `nix daemon` in the test suite
2024-01-19 16:21:51 +01:00
Robert Hensing
2a3c5e6b8b
Merge pull request #9480 from NixOS/libfetchers-git-exportIgnore
libfetchers/git: Support export-ignore
2024-01-16 23:03:46 +01:00
Eelco Dolstra
302625e83b Merge remote-tracking branch 'origin/master' into handle-missing-gc-socket 2024-01-16 13:18:58 +01:00
Robert Hensing
cbc319e9be tests/functional/lang: Test substring with negative length 2024-01-16 12:19:31 +01:00
Robert Hensing
b2deff1947
Merge pull request #9747 from awakesecurity/mz/fix-quadratic-splitString
Fix performance of builtins.substring for empty substrings
2024-01-16 12:18:59 +01:00
John Ericson
03a6ca9b25 tests/functional/nix-profile.sh: Add missing --no-link
Otherwise we get a stray `tests/functional/result`, which can cause
spurious failures later.

(I got a failure because the test temp dir effecting the store dir
changed. This caused a test later because Nix didn't want to remove the
old `result` because it wasn't pointing inside the new Nix store.)
2024-01-14 15:27:44 -05:00
Cole Helbling
bbcd9fcfc1 Arbitrarily bring back some nix-daemon calls
This means that both `nix daemon` and `nix-daemon` will be (somewhat)
tested.
2024-01-13 11:27:04 -08:00
Mel Zuser
1996105e91 added test for empty substring special case 2024-01-12 09:40:21 -08:00
Eelco Dolstra
52f949bbf5
Merge pull request #9656 from edolstra/nix-profile-stable-names
Make profile element names stable
2024-01-12 16:59:36 +01:00
Eelco Dolstra
72560f7bbe Add profile migration test 2024-01-12 16:33:15 +01:00
Robert Hensing
692e9197bc fetchTree: Disallow combination of submodules and exportIgnore for now 2024-01-12 15:31:53 +01:00
Robert Hensing
99bd12f0b1 fetchGit/fetchTree: Improve exportIgnore, submodule interaction
Also fingerprint and some preparatory improvements.

Testing is still not up to scratch because lots of logic is duplicated
between the workdir and commit cases.
2024-01-12 15:31:53 +01:00
Robert Hensing
ce6d58a97c git fetcher: Add exportIgnore parameter
Enabled for fetchGit, which historically had this behavior,
among other behaviors we do not want in fetchGit.

fetchTree disables this parameter by default. It can choose the
simpler behavior, as it is still experimental.

I am not confident that the filtering implementation is future
proof. It should reuse a source filtering wrapper, which I believe
Eelco has already written, but not merged yet.
2024-01-12 15:31:52 +01:00
DavHau
4d0ecda33e fetchTree/fetchGit: add test for .gitattributes
...with the intention to prevent future regressions in fetchGit
2024-01-12 15:31:52 +01:00
Eelco Dolstra
0d55d660d5
Merge pull request #8043 from bobvanderlinden/pr-shell-env
nix: develop: use SHELL from rc script
2024-01-12 13:42:42 +01:00
Eelco Dolstra
e21b3cf9db Merge remote-tracking branch 'origin/master' into nix-profile-stable-names 2024-01-12 13:36:27 +01:00
Eelco Dolstra
66bd1b0298 Merge remote-tracking branch 'origin/master' into pr-shell-env 2024-01-12 12:56:26 +01:00
Eelco Dolstra
3e23759834 gc-non-blocking.sh: Add explanation
Also name the _NIX_TEST_GC_SYNC environment variables logically.
2024-01-12 12:38:55 +01:00
Eelco Dolstra
5703c31325 Merge remote-tracking branch 'origin/master' into handle-missing-gc-socket 2024-01-12 12:26:25 +01:00
Rebecca Turner
df84dd4d8d
Restore ambiguous value printer for nix-instantiate
The Nix team has requested that this output format remain unchanged.
I've added a warning to the man page explaining that `nix-instantiate
--eval` output will not parse correctly in many situations.
2024-01-11 16:34:36 -08:00
Rebecca Turner
0fa08b4516
Unify and refactor value printing
Previously, there were two mostly-identical value printers -- one in
`libexpr/eval.cc` (which didn't force values) and one in
`libcmd/repl.cc` (which did force values and also printed ANSI color
codes).

This PR unifies both of these printers into `print.cc` and provides a
`PrintOptions` struct for controlling the output, which allows for
toggling whether values are forced, whether repeated values are tracked,
and whether ANSI color codes are displayed.

Additionally, `PrintOptions` allows tuning the maximum number of
attributes, list items, and bytes in a string that will be displayed;
this makes it ideal for contexts where printing too much output (e.g.
all of Nixpkgs) is distracting. (As requested by @roberth in
https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735)

Please read the tests for example output.

Future work:
- It would be nice to provide this function as a builtin, perhaps
  `builtins.toStringDebug` -- a printing function that never fails would
  be useful when debugging Nix code.
- It would be nice to support customizing `PrintOptions` members on the
  command line, e.g. `--option to-string-max-attrs 1000`.
2024-01-11 16:34:36 -08:00
John Ericson
423484ad26 Only link with -pthread on Unix
We don't want this with MinGW.
2024-01-10 20:38:39 -05:00
John Ericson
f9e5eb5f0a Make indentation in makesfiles consistent
Tab (as required) for rules, two spaces for `if`...`endif`.
2024-01-10 20:26:34 -05:00
Eelco Dolstra
b91c935c2f
Merge pull request #9648 from cole-h/nix-shell-ordering
nix shell: reflect command line order in PATH order
2024-01-09 19:00:33 +01:00
John Ericson
ff6de4a9ee
Merge pull request #9662 from shlevy/flat-fixed-references-assert
Improve error message for fixed-outputs with references.
2024-01-08 10:46:37 -05:00
John Ericson
c4c636284e
Only test bug fix with new enough deamon 2024-01-08 10:17:28 -05:00
Shea Levy
eeb2f083c5
Improve error message for fixed-outputs with references.
This codepath is possible, e.g. with a dockerTools.pullImage of an image with a Nix store.
2024-01-07 07:32:31 -05:00
Rebecca Turner
7434caca05
Fix segfault on infinite recursion in some cases
This fixes a segfault on infinite function call recursion (rather than
infinite thunk recursion) by tracking the function call depth in
`EvalState`.

Additionally, to avoid printing extremely long stack traces, stack
frames are now deduplicated, with a `(19997 duplicate traces omitted)`
message. This should only really be triggered in infinite recursion
scenarios.

Before:

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)'
    Segmentation fault: 11

After:

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)'
    error: stack overflow

           at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)' --show-trace
    error:
           … from call site
             at «string»:1:1:
                1| (x: x x) (x: x x)
                 | ^

           … while calling anonymous lambda
             at «string»:1:2:
                1| (x: x x) (x: x x)
                 |  ^

           … from call site
             at «string»:1:5:
                1| (x: x x) (x: x x)
                 |     ^

           … while calling anonymous lambda
             at «string»:1:11:
                1| (x: x x) (x: x x)
                 |           ^

           … from call site
             at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^

           (19997 duplicate traces omitted)

           error: stack overflow
           at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^
2023-12-29 22:16:44 -08:00
Shea Levy
c3942ef85f
Build IFD in the build store when using eval-store.
Previously, IFDs would be built within the eval store, even though one
is typically using `--eval-store` precisely to *avoid* local builds.

Because the resulting Nix expression must be copied back to the eval
store in order to be imported, this requires the eval store to trust
the build store's signatures.
2023-12-23 21:33:56 -05:00
Shea Levy
9cb287657b
remote-store test: Break out IFD expression into a separate file 2023-12-23 17:15:09 -05:00
Eelco Dolstra
6268a45b65 nix profile: Make profile element names stable
The profile manifest is now an object keyed on the name returned by
getNameFromURL() at installation time, instead of an array. This
ensures that the names of profile elements don't change when other
elements are added/removed.
2023-12-22 16:21:30 +01:00
Eelco Dolstra
3187bc9ac3 nix profile: Remove indices 2023-12-22 16:21:30 +01:00
Felix Uhl
397cf4e285 nix search: Disallow empty regex
Fixes #4739
Fixes #3553 in spirit IMO
2023-12-21 22:13:43 +01:00
Cole Helbling
1fb43d1eee tests: add a test for command line ordering 2023-12-21 10:23:07 -08:00
Eelco Dolstra
9d9d9ff0de Merge remote-tracking branch 'origin/master' into profile-names-instead-of-index 2023-12-21 16:21:26 +01:00
Rebecca Turner
0cee56db1a
Fix logging.sh test on macOS
On macOS in the `nix develop` shell, `make
tests/functional/logging.sh.test` errors:

    ++(logging.sh:18) mktemp
    +(logging.sh:18) builder=/var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.StuabKUhMh
    +(logging.sh:19) echo -e '#!/bin/sh\nmkdir $out'
    +++(logging.sh:22) mktemp -d
    ++(logging.sh:22) nix-build -E 'with import ./config.nix; mkDerivation { name = "fnord"; builder = /var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.StuabKUhMh; }' --out-link /var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.oaKcy0NXqC/result
    error:
           … while calling the 'derivationStrict' builtin
             at <nix/derivation-internal.nix>:9:12:
                8|
                9|   strict = derivationStrict drvAttrs;
                 |            ^
               10|

           … while evaluating derivation 'fnord'
             whose name attribute is located at «string»:1:42

           … while evaluating attribute 'args' of derivation 'fnord'
             at /Users/wiggles/nix/tests/functional/config.nix:23:7:
               22|       builder = shell;
               23|       args = ["-e" args.builder or (builtins.toFile "builder-${args.name}.sh" ''
                 |       ^
               24|         if [ -e "$NIX_ATTRS_SH_FILE" ]; then source $NIX_ATTRS_SH_FILE; fi;

           error: path '/var' is a symlink
    +(logging.sh:22) outp=
    ++(logging.sh:22) onError
    ++(/Users/wiggles/nix/tests/functional/common/vars-and-functions.sh:237) set +x
    logging.sh: test failed at:
      main in logging.sh:22

This is because `mktemp` returns a path like
`/var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.qDY24l6bIM`,
where `/var` is a symlink to `/private/var`.

Then, we attempt to use that path as a `builder`, which errors because
symlinks are impure or whatever.

Anyways, we can fix this by using `realpath "$(mktemp)"` instead of
`mktemp` directly.

NB: This error doesn't seem to happen when I run the tests through `nix
flake check`. I'm not sure if Nix does something to `TMP` in that case.
2023-12-18 14:04:25 -08:00
Cole Helbling
1f7b62f123 Use nix daemon in the test suite
As part of the CLI stabilization effort, the last remaining checkbox (at
the moment) for `nix daemon` is that it "needs testing". This implements
the proposal of using `nix daemon` in place of `nix-daemon` in the test
suite.
2023-12-18 11:29:55 -08:00
Rebecca Turner
5cb98095ba
Remove some blank lines from stack traces
This keeps hint messages, source location information, and source code
snippets grouped together, while making stack traces shorter (so that
more stack frames can be viewed on the same terminal).

Before:

    error:
           … while evaluating the attribute 'body'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:

                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:

                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed

           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|

After:

    error:
           … while evaluating the attribute 'body'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:
                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:
                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed
           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|
2023-12-15 23:57:26 -08:00
John Ericson
70f50cbb2a Functional Test for builtins.storeDir and builtins.currentSystem
These were under-tested. This tests the status quo and especially
previous commit of this PR better.
2023-12-14 19:47:10 -05:00
John Ericson
e76df87814 Test nix copy --substitute-on-destination
It works with both `ssh://` and `ssh-ng://` now since #9600 (and
`ssh-ng:// didn't work before that).

Also, by making the two tests share code, we nudge ourselves towards
making sure there is feature parity.
2023-12-13 15:22:19 -05:00
Théophane Hufschmitt
0dfa66d120
Merge pull request #9593 from B4dM4n/fix-path-like-flake-query
Fix query parsing for path-like flakes
2023-12-12 17:15:30 +01:00
John Ericson
eae2717e00 tests: Use cp -ar instead of tar-untar pipe 2023-12-11 13:56:18 -05:00
John Ericson
8d0a03b5a2 Fix tests after last rename (path -> pathInLowerStore) 2023-12-11 13:48:42 -05:00
John Ericson
c90e46d3f0
Update tests/functional/local-overlay-store/common.sh
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-12-11 13:45:46 -05:00
John Ericson
c93f78f6fa Fix test a bit from previous commit 2023-12-11 13:36:53 -05:00
John Ericson
6bb13358e6
Update tests/functional/local-overlay-store/redundant-add-inner.sh
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-12-11 13:34:09 -05:00
John Ericson
bf0bf3d1be local-overlay store tests: storeDirs -> setupStoreDirs
Co-Authored-By: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
2023-12-11 13:30:40 -05:00
John Ericson
245af3ea02 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-12-11 13:12:09 -05:00
John Ericson
9f39dda66c Fix building CA derivations with and eval store
I don't love the way this code looks. There are two larger problems:

- eval, build/scratch, destination stores (#5025) should have different
  types to reflect the fact that they are used for different purposes
  and those purposes correspond to different operations. It should be
  impossible to "use the wrong store" in my cases.

- Since drvs can end up in both the eval and build/scratch store, we
  should have some sort of union/layered store (not on the file sytem
  level, just conceptual level) that allows accessing both. This would
  get rid of the ugly "check both" boilerplate in this PR.

Still, it might be better to land this now / soon after minimal cleanup,
so we have a concrete idea of what problem better abstractions are
supposed to solve.
2023-12-11 12:17:36 -05:00
Fabian Möller
994f1b5c0d
Add test cases for flake urls with fragments 2023-12-11 16:05:34 +01:00
Robert Hensing
c8458bd731
Merge pull request #9555 from 9999years/positions-in-errors
Pass positions when evaluating
2023-12-09 03:55:58 +01:00
Robert Hensing
6e8d598314 tests/lang/eval-fail-bad-string-interpolation-4: init 2023-12-09 02:52:49 +01:00
Robert Hensing
9b7b7a7561 Revert "Print the value in error: cannot coerce messages (#9553)"
This reverts commit f0ac2a35d5.

The request from the sibling PR, which also applies here, was not addressed.
https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735
2023-12-09 02:13:32 +01:00
Robert Hensing
d4f6b1d38b
Merge pull request #9497 from edolstra/move-access-control
Move restricted/pure-eval access control out of the evaluator and into the accessor
2023-12-08 22:21:50 +01:00
Rebecca Turner
f0ac2a35d5
Print the value in error: cannot coerce messages (#9553)
* Print the value in `error: cannot coerce` messages

This extends the `error: cannot coerce a TYPE to a string` message
to print the value that could not be coerced. This helps with debugging
by making it easier to track down where the value is being produced
from, especially in errors with deep or unhelpful stack traces.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-08 16:36:57 +00:00
Rebecca Turner
0b80935c22
Pass positions when evaluating
This includes position information in more places, making debugging
easier.

Before:

```
$ nix-instantiate --show-trace --eval tests/functional/lang/eval-fail-using-set-as-attr-name.nix
error:
       … while evaluating an attribute name

         at «none»:0: (source not available)

       error: value is a set while a string was expected
```

After:

```
error:
       … while evaluating an attribute name

         at /pwd/lang/eval-fail-using-set-as-attr-name.nix:5:10:

            4| in
            5|   attr.${key}
             |          ^
            6|

       error: value is a set while a string was expected
```
2023-12-07 10:27:21 -08:00
Peter Kolloch
e9a5365db6 hash.sh: Make failure tests more tolerant of additional output
"warning: you don'\''t have Internet access; disabling some network-dependent features" ...

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch
fc6f29053a Renamed HashFormat::Base32 to HashFormat::Nix32
...and also adjusted parsing accordingly.

Also added CLI completion for HashFormats.

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch
0c2d5f7673 nix hash convert: s/--type/--algo/ + more functional tests
https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch
6bbd900d4f nix hash convert: added
This deviated from the proposal! See comments on the issue.

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:34 +01:00
Eelco Dolstra
53ab5d87c2 Use expectStderr 2023-12-06 14:05:32 +01:00
Eelco Dolstra
83c067c0fa PosixSourceAccessor: Don't follow any symlinks
All path components must not be symlinks now (so the user needs to
call `resolveSymlinks()` when needed).
2023-12-05 23:02:59 +01:00
Max
591f28e586 Merge remote-tracking branch 'nixos/master' 2023-12-05 17:49:26 +01:00
Eelco Dolstra
345f79d016 Check that we can't follow symlinks outside of the allowed paths 2023-12-05 16:33:02 +01:00
wh0
2e5abc0fd0
tests: avoid a chroot store without sandbox support 2023-12-03 17:18:58 -08:00
John Ericson
82359eba6b
Merge pull request #9233 from bouk/bouk/apply-config-inner
config: add included files into parsedContents before applying
2023-12-01 08:23:32 -05:00
Eelco Dolstra
ea95327e72 Move restricted/pure-eval access control out of the evaluator and into the accessor 2023-11-30 16:16:17 +01:00
John Ericson
52e0911302 Use buildprefix in a few more places
`installcheck` doesn't yet work, but the rest of the build can now
happen mostly inside a separate build directory.

Progress on #9342

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-11-29 19:49:07 -05:00
Théophane Hufschmitt
f300e11b05 Rename nix show-config to nix config show
Part of #7672
2023-11-28 14:28:19 +01:00
Tom Bereknyei
9a4641146f tests: ensure nix-shell uses relative paths for expressions 2023-11-27 14:40:38 -05:00
Bob van der Linden
06a745120b
nix: develop: remove test for interactive shell executable 2023-11-26 21:27:46 +01:00
Max
c076362db8 Merge remote-tracking branch 'nixos/master' 2023-11-25 20:40:52 +01:00
Théophane Hufschmitt
bf13943206
Merge pull request #9173 from vkryachko/transitive-input-overrides
Fix transitive input locking.
2023-11-23 08:54:45 +01:00
Max
4f4d3a538e Merge remote-tracking branch 'nixos/master' 2023-11-22 22:10:12 +01:00
Matej Urbas
03c3af1bf9 mounted-ssh-ng store: integration tests 2023-11-21 13:35:12 -05:00
Eelco Dolstra
c185e45ed7
Merge pull request #9410 from edolstra/shallow
Some shallow git fetching improvements
2023-11-21 12:58:01 +01:00
John Ericson
516e7ddc41
Merge pull request #9257 from Artturin/nixenvjsondrvpath
`nix-env --query`: fix `--json` ignoring `--drv-path`
2023-11-20 16:32:35 -05:00
Max
1314205843 Merge remote-tracking branch 'nixos/master' 2023-11-20 20:44:49 +01:00
Eelco Dolstra
e4066c0444 Fetch specific Git revisions
This is more efficient, and necessary when using shallow=1 with a rev.
2023-11-20 17:25:40 +01:00
John Ericson
4ab27e5595 Merge remote-tracking branch 'upstream/master' into libgit2 2023-11-20 08:30:26 -05:00
John Ericson
87ac33f29a
Merge branch 'master' into nixenvjsondrvpath 2023-11-18 13:47:14 -05:00
Eelco Dolstra
31ebc6028b Fix symlink handling
This restores the symlink handling behaviour prior to
94812cca98.

Fixes #9298.
2023-11-16 16:45:14 +01:00
Bob van der Linden
eaa3629532
fixup! nix: develop: add tests for interactive shell 2023-11-16 15:12:31 +01:00
Bob van der Linden
ceab20d056
nix: develop: add tests for interactive shell 2023-11-16 09:37:49 +01:00
Théophane Hufschmitt
84128461b6 Add a new nix store add command
Deprecate `nix store add-file` and `nix store add-path`, and replace
them with a single `nix store add` command.
2023-11-15 19:21:17 +01:00
John Ericson
9afa697ab6 Refactor bash test build system a bit
The basic idea here is to separate a few intertwined notions:

1. Not all "run bash tests" are "install tests"

2. Not all "run bash tests" use `tests/functional/init.sh`, or any
   pre-test initialization at all.

This will used in the next commit when we have a test that check unit
test golden master data.

Also, move our custom `PS4` from the test to the test runner, as it is
part of how we want to display the tests, not the test themselves.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-11-10 11:02:37 -05:00
Artturin
a903f85f84 nix-env --query: fix --json ignoring --drv-path
```json
{
  "AMB-plugins": {
    "drvPath": "/nix/store/l99cb7h2hy8dg005arsjbd9kx0w05d3h-AMB-plugins-0.8.1.drv",
    "name": "AMB-plugins-0.8.1",
    "outputName": "out",
    "outputs": {
      "out": null
    },
    "pname": "AMB-plugins",
    "system": "x86_64-linux",
    "version": "0.8.1"
  },
  "ArchiSteamFarm": {
    "drvPath": "/nix/store/nhplgyjj34fz6hjmnyih25gxscfh8s7b-ArchiSteamFarm-5.4.12.5.drv",
    "name": "ArchiSteamFarm-5.4.12.5",
    "outputName": "out",
    "outputs": {
      "out": null
    },
    "pname": "ArchiSteamFarm",
    "system": "x86_64-linux",
    "version": "5.4.12.5"
  },
...
```
2023-11-09 22:31:32 +02:00
Eelco Dolstra
98a120b8b8 Merge remote-tracking branch 'origin/master' into libgit2 2023-11-09 16:48:41 +01:00
John Ericson
f0adb72c23 Mark fetchTree as unstable again
As discussed in our last meeting, we need a bit more time, but we are
"time boxing" the work left to do to ensure there is not unbounded
delay.

Rather than putting it back underneath `flakes`, though, put it
underneath its own `fetch-tree` experimental feature (which `flakes`
includes/implies). This signals our commitment to the plan to stabilize
it first without waiting to go through the rest of Flakes, and also will
give users a "release candidate" when we get closer to stabilization.

This reverts commit 4112dd1fc9.
2023-11-08 23:23:56 -05:00
Robert Hensing
466271568b nix: Parse --file relative to shebang script 2023-11-07 13:24:01 -05:00
Robert Hensing
198bc22e3b nix: Add command baseDir to parse --expr relative to shebang script 2023-11-07 13:24:01 -05:00
Robert Hensing
20ff61ab25 nix: Reserve shebang line syntax and only parse double backtick quotes
Being restrictive about syntax leaves opportunity to improve the
syntax and functionality later.
2023-11-07 13:24:01 -05:00
Tom Bereknyei
06f3583b1c feat: break out of shebang processing for non-comments 2023-11-07 13:24:01 -05:00
Tom Bereknyei
bfcbf3b5bf doc: shebang release notes, docs, tests
fix: release notes
2023-11-07 13:24:01 -05:00
Tom Bereknyei
eea5a003d9 fix: test to ensure arguments are passed 2023-11-07 13:24:01 -05:00
Tom Bereknyei
74210c12fe Shellbang support with flakes
Enables shebang usage of nix shell. All arguments with `#! nix` get
added to the nix invocation. This implementation does NOT set any
additional arguments other than placing the script path itself as the
first argument such that the interpreter can utilize it.

Example below:

```
    #!/usr/bin/env nix
    #! nix shell --quiet
    #! nix nixpkgs#bash
    #! nix nixpkgs#shellcheck
    #! nix nixpkgs#hello
    #! nix --ignore-environment --command bash
    # shellcheck shell=bash
    set -eu
    shellcheck "$0" || exit 1
    function main {
        hello
        echo 0:"$0" 1:"$1" 2:"$2"
    }
    "$@"
```

fix: include programName usage

EDIT: For posterity I've changed shellwords to shellwords2 in order
      not to interfere with other changes during a rebase.
      shellwords2 is removed in a later commit. -- roberth
2023-11-07 13:24:01 -05:00
Bob van der Linden
9c0a09f09f allow ^ in URLs
Users may select specific outputs using the ^output syntax or selecting
any output using ^*.

URL parsing currently doesn't support these kinds of output references:
parsing will fail.

Currently `queryRegex` was reused for URL fragments, which didn't
include support for ^. Now queryRegex has been split from fragmentRegex,
where only the fragmentRegex supports ^.
2023-11-06 21:21:20 +01:00
Felix Uhl
257b768436 Enable using human-readable name in nix profile 2023-11-06 21:21:20 +01:00
Mel Zuser
61d6fe059e
Fix boost::bad_format_string exception in builtins.addErrorContext (#9291)
* Fix boost::bad_format_string exception in builtins.addErrorContext

The message passed to addTrace was incorrectly being used as a format
string and this this would cause an exception when the string contained
a '%', which can be hit in places where arbitrary file paths are
interpolated.

* add test
2023-11-06 19:13:40 +00:00
Théophane Hufschmitt
06d0d51895
Merge pull request #9242 from obsidiansystems/path-info-map
Make `nix path-info --json` return an object not array
2023-11-06 18:00:42 +01:00
John Ericson
cc46ea1630 Make nix path-info --json return an object not array
Before it returned a list of JSON objects with store object information,
including the path in each object. Now, it maps the paths to JSON
objects with the metadata sans path.

This matches how `nix derivation show` works.

Quite hillariously, none of our existing functional tests caught this
change to `path-info --json` though they did use it. So just new
functional tests need to be added.
2023-11-06 11:06:31 -05:00
Eelco Dolstra
28dddde0ac
Merge pull request #9225 from drupol/nixpkgs-pr-107251/make-gnutar-reproducible
fix: make sure `tar` reproducibility flags are set
2023-11-06 10:51:42 +01:00
Valentin Gagarin
8e222fbb12
Merge pull request #8848 from BootRhetoric/flake-authentication 2023-11-04 00:23:28 +01:00
BootRhetoric
271932782d
fetchGit and flake: add commit signature verification tests
This adds simple tests of the commit signature verification mechanism of
fetchGit and its flake input wrapper.
OpenSSH is added to the build dependencies since it's needed to create
a key when testing the functionality. It is neither a built- nor a
runtime dependency.
2023-11-03 20:23:31 +01:00
Eelco Dolstra
e5908212e2 Fix nar-access test on macOS 2023-11-03 11:03:58 +01:00
Eelco Dolstra
39ea46abb1 Merge remote-tracking branch 'origin/master' into libgit2 2023-11-02 16:16:37 +01:00
Eelco Dolstra
d26c317b14
Use expect
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-11-02 13:40:54 +01:00
Eelco Dolstra
1a902f5fa7 Merge FSAccessor into SourceAccessor 2023-11-01 17:09:28 +01:00
Eelco Dolstra
cdb27c1519 SourceAccessor: Change the main interface from lstat() to maybeLstat() 2023-11-01 15:26:07 +01:00
Eelco Dolstra
0c5eac9c45 Git fetcher: Handle submodules for workdirs 2023-10-31 15:59:25 +01:00
Felix Uhl
f282ef5a56 fix: segfault in positional arg completion
Adding the inputPath as a positional feature uncovered this bug.
As positional argument forms were discarded from the `expectedArgs`
list, their closures were not. When the `.completer` closure was then
called, part of the surrounding object did not exist anymore.

This didn't cause an issue before, but with the new call to
`getEvalState()` in the "inputs" completer in nix/flake.cc, a segfault
was triggered reproducibly on invalid memory access to the `this`
pointer, which was always 0.

The solution of splicing the argument forms into a new list to extend
their lifetime is a bit of a hack, but I was unable to get the "nicer"
iterator-based solution to work.
2023-10-31 15:33:57 +01:00
Felix Uhl
c7dcdb8325 Overhaul nix flake update and lock commands
Closes #5110
2023-10-31 15:33:57 +01:00
Eelco Dolstra
d88106df24 Git fetcher: Improve submodule handling
Instead of making a complete copy of the repo, fetching the
submodules, and writing the result to the store (which is all
superexpensive), we now fetch the submodules recursively using the Git
fetcher, and return a union accessor that "mounts" the accessors for
the submodules on top of the root accessor.
2023-10-31 14:52:21 +01:00
Eelco Dolstra
ee36a44bf2 GitInputScheme: Use libgit2
This replaces most calls to the "git" binary with libgit2.
2023-10-31 14:52:21 +01:00
Max
e5a5fbc0fb Merge remote-tracking branch 'nixos/master' 2023-10-31 00:05:37 +01:00
John Ericson
250c3541bb Use local-overlay:// not local-overlay for store URL
This is a bit uglier, but allows us to avoid an ad-hoc special case in
`store-api.cc`.
2023-10-25 15:29:11 -04:00
John Ericson
8434f23c97 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-10-25 14:23:20 -04:00
Bouke van der Bijl
1d28d613b1 config: add included files into parsedContents before applying
Fixes #8719
2023-10-25 11:39:18 +02:00
Pol Dellaiera
7f71fc7540
fix: make sure tar reproducibility flags are set 2023-10-24 22:14:35 +02:00
John Ericson
a58d7f143e
Merge pull request #9216 from obsidiansystems/addDrvOutputDependencies-pre
Add `builtins.addDrvOutputDependencies`
2023-10-23 13:26:51 -04:00
John Ericson
5c1cb0b696 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-10-23 13:13:37 -04:00
John Ericson
765436e300 Add builtins.addDrvOutputDependencies
End goal: make `(mkDerivation x).drvPath` behave like a non-DrvDeep
context.

Problem: users won't be able to recover the DrvDeep behavior when
nixpkgs makes this change.

Solution: add this primop.

The new primop is fairly simple, and is supposed to complement other
existing ones (`builtins.storePath`, `builtins.outputOf`) so there are
simple ways to construct strings with every type of string context
element.

(It allows nothing we couldn't already do with `builtins.getContext` and `builtins.appendContext`, which is also true of those other two primops.)

This was originally in #8595, but then it was proposed to land some doc
changes separately. So now the code changes proper is just moved to
this, and the doc will be done in that.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.nore
github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io
2023-10-23 12:49:14 -04:00
Naïm Favier
e053eeb272 tests: test nix-shell shebang quoting 2023-10-23 17:32:50 +02:00
Eelco Dolstra
955bbe53c5
Merge pull request #9177 from edolstra/input-accessors
Backport FSInputAccessor and MemoryInputAccessor from lazy-trees
2023-10-23 11:42:04 +02:00
Eelco Dolstra
7a086a32bc fetchToStore(): Handle flat ingestion method and add test 2023-10-20 13:32:15 +02:00
Eelco Dolstra
173abec0bc coerceToPath(): Handle __toString, add tests 2023-10-20 13:06:44 +02:00
Yueh-Shun Li
5088e6563a primops: add builtins.convertHash
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-10-19 00:58:56 +08:00
Vladimir Kryachko
311e2ad024 Address review comments 2023-10-18 10:37:06 -04:00
Théophane Hufschmitt
c1a1766c46
Merge pull request #9169 from vkryachko/follow_cycle
Detect cycles in flake follows.
2023-10-18 07:34:03 +02:00
John Ericson
9d1f42db52
Merge pull request #9150 from vicky1999/fix/8914
`nix store ping` -> `nix store info`
2023-10-17 22:52:28 -04:00
vicky1999
891dfb4359 updated store ping to store info in files 2023-10-18 00:14:11 +05:30
vicky1999
a0f071f1d3 store info sh renamed 2023-10-18 00:12:10 +05:30
vicky1999
dcc5f801f4 Store info command help updates 2023-10-17 09:39:59 +05:30
Vladimir Kryachko
b3fd7db63f Detect cycles in flake follows.
This change results in an error thrown as opposed to segfaulting due to
stack overflow.

Fixes #9144
2023-10-16 13:00:49 -04:00
John Ericson
f7a36f9812 Fix language tests a bit
- Remove some stray saved error messages that didn't correspond to any
  test, because they were renamed in
  d11faa01b5.

- Need `--eval` in test failure test in order to get in "read-only" mode
  where we don't try to write to the store. (The other tests already do
  this.)

- Need `--strict` so top-level attribute sets are still forced, like
  they are without `--eval`.
2023-10-16 12:52:59 -04:00
John Ericson
c27d2f8da9 Add two more completions tests
Thanks @ncfavier for catching these regressions in my PR.

Co-Authored-By: Naïm Favier <n@monade.li>
2023-10-16 10:09:10 -04:00
Max
ba035e1ea3 Merge remote-tracking branch 'nixos/master' 2023-10-15 19:51:25 +02:00
Eelco Dolstra
856fe13533 fetchTree cleanup
Two changes:

* The (probably unintentional) hack to handle paths as tarballs has
  been removed. This is almost certainly not what users expect and is
  inconsistent with flakeref handling everywhere else.

* The hack to support scp-style Git URLs has been moved to the Git
  fetcher, so it's now supported not just by fetchTree but by flake
  inputs.
2023-10-13 14:34:23 +02:00
Max
90e32c2a42 Merge remote-tracking branch 'nixos/master' 2023-10-12 20:09:14 +02:00
Ninlives
94e91566ed
Allow CLI to pass environment variables to FOD builder (#8830)
Add a new experimental `impure-env` setting that is a key-value list of
environment variables to inject into FOD derivations that specify the
corresponding `impureEnvVars`.

This allows clients to make use of this feature (without having to change the
environment of the daemon itself) and might eventually deprecate the current
behaviour (pick whatever is in the environment of the daemon) as it's more
principled and might prevent information leakage.
2023-10-11 11:58:42 +00:00
John Ericson
47b3508665 Use positive source filtering for the standalone functional tests job
Additionally this skipping of the building is reimplemented to be a bit
more robust and use the same idioms as the functionality for skipping
the tests. In particular, it will now work even if the source files
exist, so we can do this during development too.
2023-10-09 08:29:27 -04:00
John Ericson
68c81c7375 Put functional tests in tests/functional
I think it is bad for these reasons when `tests/` contains a mix of
functional and integration tests

 - Concepts is harder to understand, the documentation makes a good
   unit vs functional vs integration distinction, but when the
   integration tests are just two subdirs within `tests/` this is not
   clear.

 - Source filtering in the `flake.nix` is more complex. We need to
   filter out some of the dirs from `tests/`, rather than simply pick
   the dirs we want and take all of them. This is a good sign the
   structure of what we are trying to do is not matching the structure
   of the files.

With this change we have a clean:
```shell-session
$ git show 'HEAD:tests'
tree HEAD:tests

functional/
installer/
nixos/
```
2023-10-06 09:05:56 -04:00