Commit graph

1263 commits

Author SHA1 Message Date
Eelco Dolstra
32a62b0d25 Re-allow inputs.x.url = "/path/to/file" 2021-12-13 20:59:33 +01:00
Eelco Dolstra
df1edd143f gc-non-blocking.sh: Don't hang indefinitely if the GC roots server crashes 2021-12-13 19:01:38 +01:00
regnat
55e3a1108c Properly take the last assignment in the REPL
When a variable is assigned in the REPL, make sure to remove any possible reference to the old one so that we correctly pick the new one afterwards

Fix #5706
2021-12-09 16:35:27 +01:00
Eelco Dolstra
853ef1304c Inline file 2021-12-07 21:45:09 +01:00
Alexander Bantyev
33926ed1e7
nix repl: add :log
Add a :log command that shows logs for a derivation.

Closes https://github.com/NixOS/nix/issues/3504

Co-authored-by: Taeer Bar-Yam <taeer@bar-yam.me>
2021-12-06 16:52:50 +03:00
Tom Bereknyei
b6cc0a704d flakes: search up to git or filesystem boundary
While parsing a flakeref, upon not finding a flake.nix, search upwards
until git or filesystem boundary.
2021-12-03 10:55:30 -05:00
Alexander Bantyev
8cbf862e6f
Flakes: computeLocks: pass correct LockParent when reusing oldLock
Previously, when we were attempting to reuse the old lockfile
information in the computeLocks function, we have passed the parent of
the current input to the next computeLocks call. This was incorrect,
since the follows are resolved relative to the parent. This caused
issues when we tried to reuse oldLock but couldn't for some
reason (read: mustRefetch is true), in that case the follows were
resolved incorrectly.

Fix this by passing the correct parent, and adding some tests to
prevent this particular regression from happening again.

Closes https://github.com/NixOS/nix/issues/5697
2021-12-02 12:02:34 +03:00
Alexander Sosedkin
a9bd06d0ea Make libcpuid dependency optional with --disable-cpuid 2021-11-28 00:52:35 +01:00
regnat
05081bedc1 Make nix-shell work when nixpkgs is content-addressed
Fix #5259
2021-11-26 09:56:48 +01:00
regnat
c47027f3a1 Fix the error when accessing a forbidden path in pure eval
If we’re in pure eval mode, then tell that in the error message rather
than (wrongly) speaking about restricted mode.

Fix https://github.com/NixOS/nix/issues/5611
2021-11-25 14:48:01 +01:00
Silvan Mosberger
09471d2680 Make lists be comparable
Makes lists comparable using lexicographic comparison.

Increments builtins.langVersion in order for this change to be
detectable
2021-11-24 13:40:46 +01:00
John Ericson
06fb6aecea Fix testing the other daemon
The eventual PATH entry needs the `.../bin` or we will not use the right
daemon.
2021-11-18 22:23:36 +00:00
Eelco Dolstra
79f27500a4 Test that untrusted config is ignored without --accept-flake-config 2021-11-18 13:32:52 +01:00
Eelco Dolstra
0961c1068a Merge branch 'accept_conf' of https://github.com/tomberek/nix 2021-11-18 12:56:09 +01:00
regnat
83af9550a1 Add a test for the --accept-flake-config option 2021-11-12 16:02:32 +01:00
Alexander Bantyev
07bffe7998
Flakes: refetch the input when a follows disappears
When an input follows disappears, we can't just reuse the old lock
file entries since we may be missing some required ones. Refetch the
input when this happens.

Closes https://github.com/NixOS/nix/issues/5289
2021-11-11 12:05:18 +03:00
Eelco Dolstra
736d6ab721
Merge pull request #5504 from NixOS/flake-options-and-daemon
Make the flake options work when using the daemon
2021-11-08 13:54:55 +01:00
Eelco Dolstra
c9ecc0948b
Merge pull request #5506 from NixOS/fix-post-hook-test-with-different-daemon
Make the post-build-hook use the daemon Nix package
2021-11-08 13:53:30 +01:00
Théophane Hufschmitt
3d6ee223d6
Merge pull request #5476 from NixOS/ca/fix-build-in-nix-repl
Fix `nix repl`’s building of CA derivations
2021-11-05 17:35:18 +01:00
regnat
1f3c3a3785 Make the flake options work when using the daemon
When setting flake-local options (with the `nixConfig` field), forward
these options to the daemon in case we’re using one.

This is necessary in particular for options like `binary-caches` or
`post-build-hook` to make sense.

Fix <343239fc8a (r44356843)>
2021-11-05 16:19:16 +01:00
regnat
93eadd5803 Make the post-build-hook use the daemon Nix package
Having the `post-build-hook` use `nix` from the client package can lead
to a deadlock in case there’s a db migration to do between both, as a
`nix` command running inside the hook will run as root (and as such will
bypass the daemon), so might trigger a db migration, which will get
stuck trying to get a global lock on the DB (as the daemon that ran the
hook already has a lock on it).
2021-11-05 11:11:33 +01:00
regnat
1a4c9ba50b Fix nix repl’s building of CA derivations
When running a `:b` command in the repl, after building the derivations
query the store for its outputs rather than just assuming that they are
known in the derivation itself (which isn’t true for CA derivations)

Fix #5328
2021-11-05 11:04:14 +01:00
Eelco Dolstra
05560f6350 Fix function-trace test case 2021-11-04 15:04:04 +01:00
Eelco Dolstra
a7d4f3411e Merge remote-tracking branch 'origin/master' into non-blocking-gc 2021-10-28 14:56:55 +02:00
regnat
af99941279 Make experimental-features a proper type
Rather than having them plain strings scattered through the whole
codebase, create an enum containing all the known experimental features.

This means that
- Nix can now `warn` when an unkwown experimental feature is passed
  (making it much nicer to spot typos and spot deprecated features)
- It’s now easy to remove a feature altogether (once the feature isn’t
  experimental anymore or is dropped) by just removing the field for the
  enum and letting the compiler point us to all the now invalid usages
  of it.
2021-10-26 07:02:31 +02:00
Eelco Dolstra
a594d1afd5 Revert "Fix referrers test"
This reverts commit e31a48366f. Unnecessary after 0be8cc1466.
2021-10-15 16:58:21 +02:00
Eelco Dolstra
10f9a8e77d Add a test for the non-blocking GC 2021-10-15 16:52:37 +02:00
regnat
0b55c8767d Disable the eval-store test when using the daemon
Using the daemon will break most of the assumptions of this test, so
it’s as simple to just disable it
2021-10-15 14:15:43 +02:00
Eelco Dolstra
4d014221d4 Fix test against old daemon 2021-10-15 12:52:21 +02:00
regnat
b598e5c47c Fix the min bound for the structured-attrs test
The min bound written corresponds to the date of the commit that
introduced the change, but it only got merged on master some weeks
later. Since the version is essentially the commit date, that means that
there’s a whole range of commits on master (including the current
`nixUnstable`) that have a higher version but don’t contain the required
change.
2021-10-14 16:00:59 +02:00
regnat
3a2fc9ce1d Fix the isDaemonNewer check
- Don’t hardcode the “newer” version
- Remove an ill-placed `return`
2021-10-14 16:00:45 +02:00
Eelco Dolstra
0be8cc1466 pathInfoCache: Use the entire base name as the cache key
This fixes a bug in the garbage collector where if a path
/nix/store/abcd-foo is valid, but we do a
isValidPath("/nix/store/abcd-foo.lock") first, then a negative entry
for /nix/store/abcd is added to pathInfoCache, so /nix/store/abcd-foo
is subsequently considered invalid and deleted.
2021-10-14 13:28:22 +02:00
Eelco Dolstra
e31a48366f Fix referrers test
This test broke the assumption that the hash parts of store paths are
unique.
2021-10-13 12:12:44 +02:00
Eelco Dolstra
c24b9d68c5 tests/multiple-outputs.sh: Assert empty store 2021-10-13 12:12:44 +02:00
Eelco Dolstra
8614cf1334 Non-blocking garbage collector
The garbage collector no longer blocks other processes from
adding/building store paths or adding GC roots. To prevent the
collector from deleting store paths just added by another process,
processes need to connect to the garbage collector via a Unix domain
socket to register new temporary roots.
2021-10-13 12:12:44 +02:00
Tom Bereknyei
03bb8f84e0 Add compression level for NARs
Based off on @dtzWill's #2276
2021-10-12 02:14:36 -04:00
Arthur Gautier
e33f74495b Adds a test for nss preload mechanism
This tests for the fix implemented in #5224

Signed-off-by: Arthur Gautier <baloo@superbaloo.net>
2021-10-07 17:25:41 +00:00
Eelco Dolstra
4806f2f6b0 Allow builtins.{path,filterSource} on paths with a context
We now build the context (so this has the side-effect of making
builtins.{path,filterSource} work on derivations outputs, if IFD is
enabled) and then check that the path has no references (which is what
we really care about).
2021-10-07 13:43:17 +02:00
Eelco Dolstra
dcb3bc614c Add a test for builtins.path in a flake 2021-10-06 20:53:29 +02:00
Eelco Dolstra
faeab0d5d5 Make Mercurial optional for the flakes tests 2021-10-06 18:29:20 +02:00
Eelco Dolstra
7cc220825d
Merge pull request #5167 from Ma27/keep-failed-on-ssh-remote-build
nix-store --serve: pass on `settings.keepFailed` from SSH store
2021-10-01 16:35:02 +02:00
Eelco Dolstra
6b5efeafd1
Merge pull request #5295 from Ma27/bmcheck-fod-mismatch
build: also throw hash-mismatch errors if `buildMode == bmCheck`
2021-09-29 10:13:45 +02:00
Maximilian Bosch
bb1a851bcf
build: also throw hash-mismatch errors if buildMode == bmCheck
This actually bit me quite recently in `nixpkgs` because I assumed that
`nix-build --check` would also error out if hashes don't match anymore[1]
and so I wrongly assumed that I couldn't reproduce the mismatch error.

The fix is rather simple, during the output registration a so-called
`delayedException` is instantiated e.g. if a FOD hash-mismatch occurs.
However, in case of `nix-build --check` (or `--rebuild` in case of `nix
build`), the code-path where this exception is thrown will never be
reached.

By adding that check to the if-clause that causes an early exit in case
of `bmCheck`, the issue is gone. Also added a (previously failing)
test-case to demonstrate the problem.

[1] https://github.com/NixOS/nixpkgs/pull/139238, the underlying issue
    was that `nix-prefetch-git` returns different hashes than `fetchgit`
    because the latter one fetches submodules by default.
2021-09-27 15:44:39 +02:00
Eelco Dolstra
9c766a40cb Fix 'error: reading a line: Input/output error' in startBuilder()
With -vvvv, the ProgressBar was polluting the stderr of the child,
messing up its \2 message to the parent.
2021-09-27 14:44:21 +02:00
Eelco Dolstra
d1bf7431bb Revert "Merge pull request #4922 from nrdxp/default-submodules"
This reverts commit 6678e98411, reversing
changes made to 90b2dd570c.
2021-09-22 17:25:25 +02:00
Eelco Dolstra
5cbb9c5406 path fetcher: Fix relative path check 2021-09-21 13:19:26 +02:00
Eelco Dolstra
c17f3c5e69 Merge branch 'mh/fix-chroot-eval' of https://github.com/obsidiansystems/nix 2021-09-15 18:37:58 +02:00
Eelco Dolstra
f3259af73e Merge branch 'tomberek/show_json' of https://github.com/tomberek/nix 2021-09-14 16:58:35 +02:00
regnat
7c8c42dfdc tests: Fix the recursive test with the daemon
Add the `recursive-nix` experimental-feature to the daemon, as the test
will otherwise fail
2021-09-13 14:29:42 +02:00
Tom Bereknyei
dc25856d74 Ensure nix flake show produces valid json 2021-09-08 19:38:22 -04:00
Shea Levy
6678e98411
Merge pull request #4922 from nrdxp/default-submodules
libfetchers/git: fetch submodules by default
2021-09-07 05:48:23 -04:00
Eelco Dolstra
b2e8120d25 parseInstallables(): Parse store paths as store paths
If the store path contains a flake, this means that a command like
"nix path-info /path" will show info about /path, not about the
default output of the flake in /path. If you want the latter, you can
explicitly ask for it by doing "nix path-info path:/path".

Fixes #4568.
2021-09-02 14:18:04 +02:00
Stéphan Kochen
b2f966f487 Failing test case for flake in store 2021-09-02 13:53:09 +02:00
Eelco Dolstra
7ee639f9db
Merge pull request #5066 from Radvendii/master
add antiquotations to paths
2021-09-01 12:55:04 +02:00
Robert Hensing
ecf9a3e128 tests/fetchurl.sh: Be verbose to help debug darwin eof bug #2794 2021-08-31 13:17:01 +02:00
Maximilian Bosch
50edbc4ddf
nix-store --serve: pass on settings.keepFailed from SSH store
When doing e.g.

    nix-build -A package --keep-failed --option \
      builders \
      'ssh://mfhydra?remote-store=/home/bosch/store x86_64-linux - 10 4 big-parallel'

this doesn't work properly because this build-setting is ignored.

I changed this behavior by passing the `settings.keepFailed` through the
serve-protocol to remote machines to make sure that I can introspect the
build-directory (which is particularly helpful when I have to look at a
`config.log` from a failed build for instance).
2021-08-31 13:11:46 +02:00
Alex Zero
57b9ba0ad0
Add tests for flake follow paths 2021-08-17 15:25:55 +01:00
Madeline Haraj
d56ddbb999 Fix disabled case in local store test 2021-08-06 12:36:03 -04:00
Taeer Bar-Yam
a6bfda7d95 path antiquotations: rename confusing test 2021-08-06 07:38:52 -04:00
Taeer Bar-Yam
624162c729 add path antiqutations test 2021-08-06 07:06:52 -04:00
ldesgoui
6eeb6f9c84 nix-shell -p: pass --arg values as nixpkgs params 2021-07-31 17:25:10 +02:00
regnat
c2c0dba792 Remove the right socket before starting the daemon
For some reason, an old socket occasionally stays here on OSX, causing
the subsequent tests to fail
2021-07-27 17:06:11 +02:00
regnat
addacfce4a Allow running all the tests with the daemon
When `NIX_DAEMON_PACKAGE` is set, make all the tests use the Nix daemon.
That way we can test every piece of Nix functionality both with and
without the daemon.

Tests for which using the daemon isn’t possible or doesn’t make sens can
selectively be disabled with `needLocalStore`
2021-07-27 17:06:11 +02:00
Eelco Dolstra
29e4913f79 Add --eval-store test 2021-07-27 11:17:56 +02:00
Eelco Dolstra
fc248cf59e Merge branch 'balsoft/no-registries' of https://github.com/serokell/nix 2021-07-21 10:11:40 +02:00
regnat
8707773965 Properly lock the builds of CA derivations
Make sure that we can’t build the same derivation twice at the same
time.

Fix https://github.com/NixOS/nix/issues/5029
2021-07-20 06:57:56 +02:00
Alexander Bantyev
3e57e3480b
Add use-registries config option (and deprecate --no-registries flag)
Some people want to avoid using registries at all on their system; Instead
of having to add --no-registries to every command, this commit allows to
set use-registries = false in the config. --no-registries is still allowed
everywhere it was allowed previously, but is now deprecated.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-07-16 23:26:04 +03:00
regnat
c05bdef020 Forward the whole Nix config to the repl subprocesses
Fill `NIX_CONFIG` with the value of the current Nix configuration before
calling the nix subprocesses in the repl
That way the whole configuration (including the possible
`experimental-features`, a possibly `--store` option or whatever) will
be made available.

This is required for example to make `nix repl` work with a custom
`--store`
2021-07-16 09:48:45 +02:00
Maximilian Bosch
04cd2da84c
Merge branch 'master' into structured-attrs-shell
Conflicts:
        src/nix/develop.cc
        src/nix/get-env.sh
        tests/shell.nix
2021-07-12 15:49:39 +02:00
Eelco Dolstra
86fb01c4be nix print-dev-env: Add --json flag 2021-07-09 12:10:48 +02:00
Eelco Dolstra
e50408bd31 nix develop: Add a test for bash functions 2021-07-09 01:18:44 +02:00
Eelco Dolstra
9fc7da1e08 Add test for #4992 2021-07-09 01:18:44 +02:00
Eelco Dolstra
b1cfe8f984 nix develop: Add a test for arrays 2021-07-09 00:32:55 +02:00
Eelco Dolstra
8648143120
Merge pull request #4988 from NixOS/fetchgit-name-attribute
Add a name attribute to the fetchers
2021-07-08 14:33:49 +02:00
regnat
7e5c79a2d2 Forbid the name attribute for fetchTree
We need to support it for the “old” fetch* functions for backwards
compatibility, but we don’t need it for fetchTree (as it’s a new
function).
Given that changing the `name` messes-up the content hashing, we can
just forbid passing a custom `name` argument to it
2021-07-08 13:53:52 +02:00
Eelco Dolstra
6060ea1b0e Add tests/dummy 2021-07-07 11:30:35 +02:00
Eelco Dolstra
02dd6bb610 tests/check.sh: Fix a race
Fixes this random failure:

    error: hash mismatch in fixed-output derivation '/tmp/nix-shell.EUgAVU/nix-test/tests/check/store/sfps3l3c5n7dabpx34kigxnfhmrwk2h6-dummy.drv':
             specified: sha256-47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=
                got:    sha256-0qhPS4tlCTfsj3PNi+LHSt1akRumTfJ0WO2CKdqASiY=

which happens because multiple tests were writing to ./dummy.
2021-07-07 10:48:47 +02:00
Eelco Dolstra
e37ecd1282 Merge branch 'balsoft/registry' of https://github.com/serokell/nix 2021-07-07 09:57:15 +02:00
regnat
e4b082a52b allow fetchMercurial to take a name argument 2021-07-06 08:43:06 +02:00
regnat
a487a652ed allow fetchTarball to take a name argument 2021-07-06 08:42:47 +02:00
regnat
83615fcf8f Allow fetchGit to take a name argument
Fix #3388
2021-07-06 08:41:52 +02:00
Michael Fellinger
087c5f5325
Fix devShell handling of env values including @ and % 2021-07-05 12:05:06 +02:00
Alexander Bantyev
d8ad6f1c10
Add tests for --registry and second arg for nix registry pin 2021-07-02 16:10:57 +03:00
Eelco Dolstra
f5320299dd
Merge pull request #4937 from NixOS/ca/make-the-tests-useful
Make the CA tests actually test something
2021-06-28 16:06:49 +02:00
Eelco Dolstra
bf68c693dc tests: Get rid of some result symlinks
Fixes

  error: cannot create symlink '/home/eelco/Dev/nix/tests/result'; already exists
2021-06-25 11:17:19 +02:00
regnat
7746cb13dc Make CA derivations compatible with recursive Nix
Add an access-control list to the realisations in recursive-nix (similar
to the already existing one for store paths), so that we can build
content-addressed derivations in the restricted store.

Fix #4353
2021-06-24 14:53:10 +02:00
regnat
be7a4a6a13 Make the post-build-hook also run for unresolved CA derivations
Fix #4837
2021-06-24 11:41:57 +02:00
regnat
01a3f4d7ec Fix the CA gc test
Broken by https://github.com/NixOS/nix/issues/4936
2021-06-23 17:37:29 +02:00
regnat
fd3f5e9085 Make the CA tests actuall test CA derivations
Fix a mistake in config.nix that was preventing
`NIX_TESTS_CA_BY_DEFAULT` from having any meaningful effect
2021-06-23 17:36:50 +02:00
Eelco Dolstra
0a535dd5ac
Merge pull request #4839 from NixOS/ca/gracefully-handle-duplicate-realisations
Gracefully handle duplicate realisations
2021-06-23 11:50:18 +02:00
Eelco Dolstra
f9f773b332
Merge pull request #4908 from NixOS/ca/fix-nix-develop
Make `nix develop` work with CA derivations
2021-06-23 11:27:58 +02:00
Eelco Dolstra
7945055c63
Merge pull request #4842 from NixOS/ca/fix-nix-shell
Make `nix-shell` support content-addressed derivations
2021-06-23 11:26:22 +02:00
regnat
a5df669bc6 Add a test for the “two glibc” issue 2021-06-23 11:18:31 +02:00
Eelco Dolstra
4a5aa1dbf6
Merge pull request #4838 from NixOS/ca/recursively-substitute-realisations
Recursively substitute the realisations
2021-06-23 10:33:25 +02:00
Eelco Dolstra
26d2c62225
Merge pull request #4906 from NixOS/collect-garbage-ca
Make `computeFSClosure` ca-aware
2021-06-23 10:07:08 +02:00
Eelco Dolstra
d9a43d3137
Merge pull request #4905 from NixOS/ca-derivations-machine-feature
Add a ca-derivations required machine feature
2021-06-23 10:05:53 +02:00
Maximilian Bosch
27ce722638
Prefix env vars for attrs.* files with NIX_ 2021-06-22 19:45:08 +02:00
Maximilian Bosch
3504c811a5
Add testcase for nix develop with __structuredAttrs 2021-06-22 19:15:57 +02:00
Maximilian Bosch
3944a120ec
Set environment variables for .attrs.json & .attrs.sh
This way no derivation has to expect that these files are in the `cwd`
during the build. This is problematic for `nix-shell` where these files
would have to be inserted into the nix-shell's `cwd` which can become
problematic with e.g. recursive `nix-shell`.

To remain backwards-compatible, the location inside the build sandbox
will be kept, however using these files directly should be deprecated
from now on.
2021-06-22 19:15:57 +02:00
Maximilian Bosch
3b5429aec1
Source complete env in nix-shell with __structuredAttrs = true;
This is needed to push the adoption of structured attrs[1] forward. It's
now checked if a `__json` exists in the environment-map of the derivation
to be openend in a `nix-shell`.

Derivations with structured attributes enabled also make use of a file
named `.attrs.json` containing every environment variable represented as
JSON which is useful for e.g. `exportReferencesGraph`[2]. To
provide an environment similar to the build sandbox, `nix-shell` now
adds a `.attrs.json` to `cwd` (which is mostly equal to the one in the
build sandbox) and removes it using an exit hook when closing the shell.

To avoid leaking internals of the build-process to the `nix-shell`, the
entire logic to generate JSON and shell code for structured attrs was
moved into the `ParsedDerivation` class.

[1] https://nixos.mayflower.consulting/blog/2020/01/20/structured-attrs/
[2] https://nixos.org/manual/nix/unstable/expressions/advanced-attributes.html#advanced-attributes
2021-06-22 19:15:57 +02:00