Commit graph

6716 commits

Author SHA1 Message Date
Eelco Dolstra
02dff9e529 Style 2021-07-12 17:32:23 +02:00
Eelco Dolstra
e06c272c12 Merge branch 'structured-attrs-shell' of https://github.com/Ma27/nix 2021-07-12 17:13:05 +02:00
Maximilian Bosch
04cd2da84c
Merge branch 'master' into structured-attrs-shell
Conflicts:
        src/nix/develop.cc
        src/nix/get-env.sh
        tests/shell.nix
2021-07-12 15:49:39 +02:00
Eelco Dolstra
91d2e8d5ad Error -> UsageError 2021-07-12 15:04:46 +02:00
Eelco Dolstra
8d97030bfd Merge branch 'days' of https://github.com/nielsegberts/nix 2021-07-12 15:02:24 +02:00
Niels Egberts
2cf14db857 Throw on unexpected input for --delete-older-than
'--delete-older-than 10' deletes the generations older than a single day, and '--delete-older-than 12m' deletes all generations older than 12 days.

This changes makes it throw on those invalid inputs, and gives an example of a valid input.
2021-07-11 11:24:43 +01:00
Niels Egberts
ae0ed53b09 toString also coerces a set with an outPath attribute to a string
nix-repl> builtins.toString { outPath = "somestring"; }
"somestring"
2021-07-09 21:50:10 +01:00
Eelco Dolstra
ceda58d112 Formatting 2021-07-09 14:03:51 +02:00
Eelco Dolstra
07790fdddf ref: Add equality operators 2021-07-09 14:03:48 +02:00
Eelco Dolstra
86fb01c4be nix print-dev-env: Add --json flag 2021-07-09 12:10:48 +02:00
Eelco Dolstra
5f6375a816 nix develop: Filter some bash magic variables 2021-07-09 01:18:44 +02:00
Eelco Dolstra
b1f1347ade nix develop: Don't parse bash environment with regexes
Instead have get-env.sh dump the bash environment as JSON. This should
be a lot less error-prone.

Fixes #4992.
2021-07-09 01:18:37 +02:00
Eelco Dolstra
d2b8b23ae9
Merge pull request #4995 from NixOS/fetchgit-name-attribute
Fix the `name` attribute in builtins.fetchGit
2021-07-09 00:32:13 +02:00
regnat
a654c1d81c Restore the possibility to use a name parameter in fetchGit
Accidentally broken by 7e5c79a2d2
2021-07-08 14:57:48 +02:00
Eelco Dolstra
8648143120
Merge pull request #4988 from NixOS/fetchgit-name-attribute
Add a name attribute to the fetchers
2021-07-08 14:33:49 +02:00
regnat
7e5c79a2d2 Forbid the name attribute for fetchTree
We need to support it for the “old” fetch* functions for backwards
compatibility, but we don’t need it for fetchTree (as it’s a new
function).
Given that changing the `name` messes-up the content hashing, we can
just forbid passing a custom `name` argument to it
2021-07-08 13:53:52 +02:00
Eelco Dolstra
2172e60f7a
Merge pull request #4935 from alyssais/host_os
Apply OS checks to host platform, not build
2021-07-08 12:40:54 +02:00
Eelco Dolstra
156666de3d
Merge pull request #4969 from serokell/balsoft/fix-nixConfig-flake-registry
flake.nixConfig: fix flake-registry config settings
2021-07-08 12:35:08 +02:00
Eelco Dolstra
2c853e2a58 Merge branch 'master' of github.com:NixOS/nix 2021-07-07 10:03:04 +02:00
Eelco Dolstra
3b3e6bb1e5 Style tweaks 2021-07-07 10:02:55 +02:00
Eelco Dolstra
e37ecd1282 Merge branch 'balsoft/registry' of https://github.com/serokell/nix 2021-07-07 09:57:15 +02:00
regnat
2c8240677e allow fetchFromGitHub to take a name argument 2021-07-06 08:43:19 +02:00
regnat
e4b082a52b allow fetchMercurial to take a name argument 2021-07-06 08:43:06 +02:00
regnat
a487a652ed allow fetchTarball to take a name argument 2021-07-06 08:42:47 +02:00
regnat
83615fcf8f Allow fetchGit to take a name argument
Fix #3388
2021-07-06 08:41:52 +02:00
Eelco Dolstra
0a8845720e
Merge pull request #4980 from Ma27/file-attr-completion
libcmd/installables: implement completion for Nix expressions passed via `-f`
2021-07-05 21:50:15 +02:00
Maximilian Bosch
3c5f69bb60
completeInstallable: also match for already typed prefixes 2021-07-05 21:37:33 +02:00
Eelco Dolstra
1f93084149
Merge pull request #4978 from NixOS/scp-uris
Allow scp-style uris in `fetchgit`
2021-07-05 14:07:04 +02:00
Eelco Dolstra
cee426cc01
Merge pull request #4938 from tomcur/store-uri-parsing
Improve machine store URI parsing
2021-07-05 14:00:35 +02:00
Eelco Dolstra
24bc935462 Merge branch 'disable_gc' of https://github.com/Pamplemousse/nix 2021-07-05 13:53:52 +02:00
Michael Fellinger
087c5f5325
Fix devShell handling of env values including @ and % 2021-07-05 12:05:06 +02:00
Maximilian Bosch
70cb2ffacc
libcmd/installables: implement completion for Nix expressions passed via -f 2021-07-03 14:19:10 +02:00
Domen Kožar
f9d72855ae
Merge pull request #4967 from Pamplemousse/specific_errors
Prefer to throw specific errors
2021-07-03 10:34:56 +02:00
regnat
ec2c6bd470 Allow scp-style uris in fetchgit
Fix #5303
2021-07-02 19:20:07 +02:00
Yestin L. Harrison
0fe84bef72 Add $NO_COLOR check to ANSI escape conditions 2021-07-02 09:33:54 -06:00
Yestin L. Harrison
20cce079f2 Respect TERM=dumb more consistently 2021-07-01 18:19:01 -06:00
Pamplemousse
4a7a8b87cd Prefer to throw specific errors
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-07-01 11:09:31 -07:00
Alexander Bantyev
e756a59c72
fixup! flake.nixConfig: fix flake-registry config settings 2021-07-01 17:54:22 +03:00
Alexander Bantyev
ef1e7ab840
flake.nixConfig: fix flake-registry config settings
Before this commit, nixConfig.flake-registry didn't have any real effect
on the evaluation, since config was applied after inputs were evaluated.
Change this behavior: apply the config in the beginning of flake::lockFile.
2021-07-01 00:23:47 +03:00
Alexander Bantyev
811f3e8605
nix registry pin: add a way to pin to a custom locked 2021-06-30 22:16:40 +03:00
Alexander Bantyev
093ed47636
nix registry: add --registry flag 2021-06-30 22:13:32 +03:00
Pamplemousse
5be17a4b96 Allow to compile after ./configure --enable-gc=no
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-06-30 09:45:21 -07:00
Eelco Dolstra
139f7af5ec
Merge pull request #4958 from NixOS/nix-develop-remove-override-input-warning
develop: Discard the input{Overrides,Updates} when getting bash
2021-06-30 17:57:01 +02:00
Eelco Dolstra
c92fbdb654
Merge pull request #4960 from NixOS/nix-repl-experimental-features
Forward the experimental features to the nix repl subprocesses
2021-06-29 15:10:02 +02:00
regnat
7daf0c6ef1 Forward the experimental features to the nix repl subprocesses
Pass the current experimental features using `NIX_CONFIG` to the various
Nix subprocesses that `nix repl` invokes.

This is quite a hack, but having `nix repl` call Nix with a subprocess
is a hack already, so I guess that’s fine.
2021-06-29 14:52:46 +02:00
Eelco Dolstra
5a18e2a533
Merge pull request #4959 from NixOS/override-non-flake-inputs
Keep the `isFlake` attribute for overriden inputs
2021-06-29 14:23:50 +02:00
regnat
4cff8188a5 Keep the isFlake attribute for overriden inputs
When overriding an input that isn’t a flake, mark the override as also not
being a flake.

Fix #3774
2021-06-29 13:56:48 +02:00
Eelco Dolstra
f14c3b6f68
Merge pull request #4944 from hercules-ci/fix-gc-crash
Fix gc crash
2021-06-29 13:52:14 +02:00
regnat
bf7960a4ed develop: Discard the input{Overrides,Updates} when getting bash
`nix develop` is getting bash from an (assumed existing) `nixpkgs`
flake. However, when doing so, it reuses the `lockFlags` passed to the
current flake, including the `--input-overrides` and `--input-update`
which generally don’t make sense anymore at that point (and trigger a
warning because of that)

Clear these overrides before getting the nixpkgs flake to get rid of the
warning.
2021-06-29 10:13:42 +02:00
Florian Franzen
decc14d4b7
man: fix formatting of nix3-profile-remove 2021-06-28 16:27:03 +02:00
Eelco Dolstra
6182ae6898
Merge pull request #4942 from NixOS/ca/remove-lock-files
Eventually delete the CA paths lock files
2021-06-28 16:12:19 +02:00
Eelco Dolstra
f5320299dd
Merge pull request #4937 from NixOS/ca/make-the-tests-useful
Make the CA tests actually test something
2021-06-28 16:06:49 +02:00
regnat
8b6fba2b63 Eventually delete the CA paths lock files
Mark the lockfiles as having to eventually be deleted so that they don’t
stay laying around in the store at the end of the build

Fix #4936
2021-06-24 15:45:05 +02:00
Maximilian Bosch
644415d391
Perform input rewrites only in LocalDerivationGoal 2021-06-24 15:25:21 +02:00
regnat
7746cb13dc Make CA derivations compatible with recursive Nix
Add an access-control list to the realisations in recursive-nix (similar
to the already existing one for store paths), so that we can build
content-addressed derivations in the restricted store.

Fix #4353
2021-06-24 14:53:10 +02:00
Robert Hensing
bec83a6f95 BoehmGCStackAllocator: ignore stack protection page
This fixes a crash that looks like:

```
Thread 1 "nix-build" received signal SIGSEGV, Segmentation fault.
0x00007ffff7ad22a0 in GC_push_all_eager () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
(gdb) bt
0  0x00007ffff7ad22a0 in GC_push_all_eager () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
1  0x00007ffff7adeefb in GC_push_all_stacks () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
2  0x00007ffff7ad5ac7 in GC_mark_some () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
3  0x00007ffff7ad77bd in GC_stopped_mark () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
4  0x00007ffff7adbe3a in GC_try_to_collect_inner.part.0 () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
5  0x00007ffff7adc2a2 in GC_collect_or_expand () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
6  0x00007ffff7adc4f8 in GC_allocobj () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
7  0x00007ffff7adc88f in GC_generic_malloc_inner () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
8  0x00007ffff7ae1a04 in GC_generic_malloc_many () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
9  0x00007ffff7ae1c72 in GC_malloc_kind () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
10 0x00007ffff7e003d6 in nix::EvalState::allocValue() () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
11 0x00007ffff7e04b9c in nix::EvalState::callPrimOp(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
12 0x00007ffff7e0a773 in nix::EvalState::callFunction(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
13 0x00007ffff7e0a91d in nix::ExprApp::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
14 0x00007ffff7e0a8f8 in nix::ExprApp::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
15 0x00007ffff7e0e0e8 in nix::ExprOpNEq::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
16 0x00007ffff7e0d708 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
17 0x00007ffff7e0d695 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
18 0x00007ffff7e0d695 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
19 0x00007ffff7e0d695 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
20 0x00007ffff7e0d695 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
21 0x00007ffff7e09e19 in nix::ExprOpNot::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
22 0x00007ffff7e0a792 in nix::EvalState::callFunction(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
23 0x00007ffff7e8cba0 in nix::addPath(nix::EvalState&, nix::Pos const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Value*, nix::FileIngestionMethod, std::optional<nix::Hash>, nix::Value&)::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
24 0x00007ffff752e6f9 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
25 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
26 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
27 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
28 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
29 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
30 0x00007ffff757f8c0 in void boost::context::detail::fiber_entry<boost::context::detail::fiber_record<boost::context::fiber, nix::VirtualStackAllocator, boost::coroutines2::detail::pull_coroutine<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::control_block::control_block<nix::VirtualStackAllocator, nix::sinkToSource(std::function<void (nix::Sink&)>, std::function<void ()>)::SinkToSource::read(char*, unsigned long)::{lambda(boost::coroutines2::detail::push_coroutine<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >&)#1}>(boost::context::preallocated, nix::VirtualStackAllocator&&, nix::sinkToSource(std::function<void (nix::Sink&)>, std::function<void ()>)::SinkToSource::read(char*, unsigned long)::{lambda(boost::coroutines2::detail::push_coroutine<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >&)#1}&&)::{lambda(boost::context::fiber&&)#1}> >(boost::context::detail::transfer_t) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
31 0x00007ffff6f331ef in make_fcontext () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libboost_context.so.1.69.0
32 0x0000000000000000 in ?? ()
```
2021-06-24 12:20:39 +02:00
regnat
be7a4a6a13 Make the post-build-hook also run for unresolved CA derivations
Fix #4837
2021-06-24 11:41:57 +02:00
Thomas Churchman
e3d11f9a9c Improve machine store URI parsing 2021-06-23 22:09:22 +01:00
Alyssa Ross
4f80464645
Apply OS checks to host platform, not build
Previously, the build system used uname(1) output when it wanted to
check the operating system it was being built for, which meant that it
didn't take into-account cross-compilation when the build and host
operating systems were different.

To fix this, instead of consulting uname output, we consult the host
triple, specifically the third "kernel" part.

For "kernel"s with stable ABIs, like Linux or Cygwin, we can use a
simple ifeq to test whether we're compiling for that system, but for
other platforms, like Darwin, FreeBSD, or Solaris, we have to use a
more complicated check to take into account the version numbers at the
end of the "kernel"s.  I couldn't find a way to just strip these
version numbers in GNU Make without shelling out, which would be even
more ugly IMO.  Because these checks differ between kernels, and the
patsubst ones are quite fiddly, I've added variables for each host OS
we might want to check to make them easier to reuse.
2021-06-23 15:00:36 +00:00
Eelco Dolstra
0a535dd5ac
Merge pull request #4839 from NixOS/ca/gracefully-handle-duplicate-realisations
Gracefully handle duplicate realisations
2021-06-23 11:50:18 +02:00
Eelco Dolstra
f9f773b332
Merge pull request #4908 from NixOS/ca/fix-nix-develop
Make `nix develop` work with CA derivations
2021-06-23 11:27:58 +02:00
regnat
c878cee895 Assert that compatible realisations have the same dependencies
Should always hold, but that’s not necessarily obvious, so better
enforce it
2021-06-23 11:27:16 +02:00
regnat
16fb7d8d95 Display the diverging paths in case of a realisation mismatch 2021-06-23 11:27:16 +02:00
regnat
40f925b2da Fix indentation 2021-06-23 11:27:16 +02:00
regnat
d32cf0c17a Gracefully ignore a substituter if it holds an incompatible realisation 2021-06-23 11:27:16 +02:00
regnat
b8f7177a7b Properly fail when trying to register an incoherent realisation 2021-06-23 11:27:14 +02:00
Eelco Dolstra
7945055c63
Merge pull request #4842 from NixOS/ca/fix-nix-shell
Make `nix-shell` support content-addressed derivations
2021-06-23 11:26:22 +02:00
Eelco Dolstra
4a5aa1dbf6
Merge pull request #4838 from NixOS/ca/recursively-substitute-realisations
Recursively substitute the realisations
2021-06-23 10:33:25 +02:00
Eelco Dolstra
26d2c62225
Merge pull request #4906 from NixOS/collect-garbage-ca
Make `computeFSClosure` ca-aware
2021-06-23 10:07:08 +02:00
Eelco Dolstra
d9a43d3137
Merge pull request #4905 from NixOS/ca-derivations-machine-feature
Add a ca-derivations required machine feature
2021-06-23 10:05:53 +02:00
Eelco Dolstra
3e4126b67c
Merge pull request #4926 from NixOS/ca/build-remote-signal-6
Fix the remote build of CA derivations
2021-06-23 10:04:35 +02:00
Eelco Dolstra
4f9508c3b5
Merge pull request #4836 from NixOS/ca/track-drvoutput-dependencies-2-le-retour
Track the dependencies of CA realisations
2021-06-23 10:03:31 +02:00
regnat
ed0e21a88d Fix indentation 2021-06-23 08:16:34 +02:00
regnat
7c96a76dd7 Reformat the sql statements 2021-06-23 08:16:34 +02:00
Théophane Hufschmitt
8d09a4f9a0 Remove a useless string split
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-06-23 08:16:34 +02:00
Théophane Hufschmitt
c13d7d0b97 Pass more values by reference
Rather than copying them around everywhere

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-06-23 08:16:34 +02:00
Maximilian Bosch
6f206549ba
Move writeStructuredAttrsShell out of ParsedDerivation class 2021-06-22 21:14:20 +02:00
Maximilian Bosch
27ce722638
Prefix env vars for attrs.* files with NIX_ 2021-06-22 19:45:08 +02:00
Maximilian Bosch
a92245b110
Remove now-obsolete typedef 2021-06-22 19:15:57 +02:00
Maximilian Bosch
3504c811a5
Add testcase for nix develop with __structuredAttrs 2021-06-22 19:15:57 +02:00
Maximilian Bosch
f1e281c4fe
Split shell & json creation for build environments with structured attrs 2021-06-22 19:15:57 +02:00
Maximilian Bosch
447928bdb5
Fix usage of structured attrs for nix develop 2021-06-22 19:15:57 +02:00
Maximilian Bosch
3944a120ec
Set environment variables for .attrs.json & .attrs.sh
This way no derivation has to expect that these files are in the `cwd`
during the build. This is problematic for `nix-shell` where these files
would have to be inserted into the nix-shell's `cwd` which can become
problematic with e.g. recursive `nix-shell`.

To remain backwards-compatible, the location inside the build sandbox
will be kept, however using these files directly should be deprecated
from now on.
2021-06-22 19:15:57 +02:00
Maximilian Bosch
3b5429aec1
Source complete env in nix-shell with __structuredAttrs = true;
This is needed to push the adoption of structured attrs[1] forward. It's
now checked if a `__json` exists in the environment-map of the derivation
to be openend in a `nix-shell`.

Derivations with structured attributes enabled also make use of a file
named `.attrs.json` containing every environment variable represented as
JSON which is useful for e.g. `exportReferencesGraph`[2]. To
provide an environment similar to the build sandbox, `nix-shell` now
adds a `.attrs.json` to `cwd` (which is mostly equal to the one in the
build sandbox) and removes it using an exit hook when closing the shell.

To avoid leaking internals of the build-process to the `nix-shell`, the
entire logic to generate JSON and shell code for structured attrs was
moved into the `ParsedDerivation` class.

[1] https://nixos.mayflower.consulting/blog/2020/01/20/structured-attrs/
[2] https://nixos.org/manual/nix/unstable/expressions/advanced-attributes.html#advanced-attributes
2021-06-22 19:15:57 +02:00
regnat
3b58dbb356 nix-shell: Replace resolving failure error by an assertion
This shouldn’t happen in practice, so better make it explicit
2021-06-22 11:29:55 +02:00
regnat
dcabb46124 Shorten a stupidly long sql query name 2021-06-21 16:28:06 +02:00
Eelco Dolstra
610baf359a
Merge pull request #4833 from NixOS/ca/json-realisations-in-worker-protocol
Always send the realisations as JSON
2021-06-21 16:12:17 +02:00
Eelco Dolstra
24e7353232
Merge pull request #4928 from NixOS/ca/remove-existing-invalid-store-path
Remove a possible existing store path when building CA derivations
2021-06-21 16:09:21 +02:00
Théophane Hufschmitt
ce674cb2cf
Properly set the output env variables
Co-authored-by: John Ericson <git@JohnEricson.me>
2021-06-21 15:52:01 +02:00
regnat
608434722b Only symlink the requested outputs in nix build
Fix #4925
2021-06-21 15:47:47 +02:00
regnat
3784c66a46 Remove a possible existing store path when building CA derivations
In case a previous interrupted build left a garbage path laying around,
remove it before trying to move the path to its final location.

Fix #4858
2021-06-21 15:29:15 +02:00
regnat
498677cbed Fix the remote build of CA derivations
Make sure that the derivation we send to the remote builder is exactly
the one that we want to build locally so that the output ids are exactly
the same

Fix #4845
2021-06-21 14:18:33 +02:00
Eelco Dolstra
4202a3bc4e UDSRemoteStore: Support the 'root' store parameter
Useful when we're using a daemon with a chroot store, e.g.

  $ NIX_DAEMON_SOCKET_PATH=/tmp/chroot/nix/var/nix/daemon-socket/socket nix-daemon --store /tmp/chroot

Then the client can now connect with

  $ nix build --store unix:///tmp/chroot/nix/var/nix/daemon-socket/socket?root=/tmp/chroot nixpkgs#hello
2021-06-18 17:04:11 +02:00
Eelco Dolstra
e6150de90d
nix develop: Filter out NIX_REMOTE
When recursive Nix is enabled, NIX_REMOTE is set to
unix:///build/.nix-socket, which doesn't work outside of the sandbox.
2021-06-15 12:06:01 +02:00
regnat
96d7170e12 Don’t check the deriver field on computeFSClosure
That doesn’t really make sense with CA derivations (and wasn’t even
really correct before because of FO derivations, though that probably
didn’t matter much in practice)
2021-06-12 12:24:53 +02:00
regnat
2cf591a134 Make nix develop work with CA derivations
Fix #4823
2021-06-11 13:35:13 +02:00
regnat
56605b4688 Make nix-shell support content-addressed derivations
Resolve the derivation before trying to load its environment −
essentially reproducing what the build loop does − so that we can
effectively access our dependencies (and not just their placeholders).

Fix #4821
2021-06-11 13:32:49 +02:00
regnat
7ac038fa4b Make computeFSClosure ca-aware
Fix #4820 by preventing nix-collect garbage from crashing if
`keep-outputs` or `keep-derivations` is true
2021-06-11 09:26:49 +02:00
regnat
7c077d2a0f Add a ca-derivations required machine feature
Make ca-derivations require a `ca-derivations` machine feature, and
ca-aware builders expose it.

That way, a network of builders can mix ca-aware and non-ca-aware
machines, and the scheduler will send them in the right place.
2021-06-11 09:12:53 +02:00
keke
50dc88a56c
fix error: 'optional' in namespace 'std' does not name a template type 2021-06-02 18:09:03 +08:00
regnat
838f862f4f doc: Wrap at 80 characters 2021-06-02 11:26:04 +02:00
Théophane Hufschmitt
7565308d04
Fix a documentation typo
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-06-02 11:25:47 +02:00
regnat
7d651f5c3f throw_ -> reportError
This function might or might not throw depending on the value of
`keepGoing`, so naming it `throw_` was a bit confusing
2021-06-02 11:24:31 +02:00
regnat
d12b12a15b Let nix flake check keep going when keep-going is set
When the `keep-going` option is set to `true`, make `nix flake check`
continue as much as it can before failing.

The UI isn’t perfect as-it-is as all the lines currently start with a
mostly useless `error (ignored): error:` prefix, but I’m not sure what
the best output would be, so I’ll leave it as-it-is for the time being

(This is a bit hijacking the `keep-going` flag as it’s supposed to be a
build-time only thing. But I think it’s faire to reuse it here).

Fix https://github.com/NixOS/nix/issues/4450
2021-06-02 11:13:12 +02:00
regnat
5985b8b527 Check the CA hash when importing stuff in the local store
When adding a path to the local store (via `LocalStore::addToStore`),
ensure that the `ca` field of the provided `ValidPathInfo` does indeed
correspond to the content of the path.
Otherwise any untrusted user (or any binary cache) can add arbitrary
content-addressed paths to the store (as content-addressed paths don’t
need a signature).
2021-06-01 15:09:24 +02:00
Eelco Dolstra
48396d940e
Merge pull request #4866 from alyssais/libdl
Only link with libdl on Linux
2021-06-01 11:50:50 +02:00
Eelco Dolstra
b8fbfc80fd
Merge pull request #4864 from jeremyschlatter/fix-typo
Fix typo in documentation
2021-06-01 11:49:46 +02:00
Eelco Dolstra
caef6f4314
Merge pull request #4734 from p01arst0rm/fix-s3-ifdef
unified macro style for ENABLE_S3
2021-06-01 11:30:21 +02:00
Alyssa Ross
c57ab17687
Only link with libdl on Linux
Linux is (as far as I know) the only mainstream operating system that
requires linking with libdl for dlopen.  On BSD, libdl doesn't exist,
so on non-FreeBSD BSDs linking will currently fail.  On macOS, it's
apparently just a symlink to libSystem (macOS libc), presumably
present for compatibility with things that assume Linux.

So the right thing to do here is to only add -ldl on Linux, not to add
it for everything that isn't FreeBSD.
2021-06-01 08:05:21 +00:00
Jeremy Schlatter
f674f7f434
Fix typo in documentation 2021-06-01 00:05:34 -07:00
Timothy Klim
4da9ec772c Add .tar.zst support for TarballInputScheme 2021-05-29 16:03:26 +07:00
Eelco Dolstra
b10256af51
Merge pull request #4849 from NixOS/ca/fix-nix-store--export
Make the Nar hash non modulo
2021-05-28 10:48:36 +02:00
regnat
a22755721b Recursively substitute the realisations
Make sure that whenever we substitute a realisation, we also substitute
its entire closure
2021-05-26 18:44:17 +02:00
regnat
1f3ff0d193 Aso track the output path of the realisation dependencies 2021-05-26 17:09:21 +02:00
regnat
cb46d70794 Add a db migration script 2021-05-26 16:59:09 +02:00
regnat
63ebfc73c5 Make copyPaths copy the whole realisations closure
Otherwise registering the realisations on the remote side might fail as
it now expects a complete closure
2021-05-26 16:59:09 +02:00
regnat
8c30acc3e8 Properly track the drvoutput references when building 2021-05-26 16:59:09 +02:00
regnat
af3afd25ea Add a method to compute the closure of a realisation
Only considers the closure in term of `Realisation`, ignores all the
opaque inputs.

Dunno whether that’s the nicest solution, need to think it through a bit
2021-05-26 16:59:09 +02:00
regnat
eca6ff06d6 Store the realisation deps on the local store 2021-05-26 16:59:09 +02:00
regnat
7ce0441d80 Add a dependencies field to DrvOutputInfo
Currently never used, nor set but will be useful shortly
2021-05-26 16:59:09 +02:00
regnat
7616268812 Always send the realisations as JSON
Align all the worker protocol with `buildDerivation` which inlines the
realisations as one opaque json blob.
That way we don’t have to bother changing the remote store protocol
when the definition of `Realisation` changes, as long as we keep the
json backwards-compatible
2021-05-26 16:59:09 +02:00
regnat
129384bcf3 Remove the remaining occurenceses of a NarHash modulo 2021-05-26 09:39:29 +02:00
regnat
79ae9e4558 Make the Nar hash non modulo
It makes much more sense to have the Nar hash be a plain straight hash
rather than a hash modulo
2021-05-25 10:58:43 +02:00
Michael Adler
9e9a8456d7 fix doc: nix profile info -> nix profile list
Signed-off-by: Michael Adler <therisen06@gmail.com>
2021-05-24 12:36:08 +02:00
regnat
a8416866cf Always send the realisations as JSON
Align all the worker protocol with `buildDerivation` which inlines the
realisations as one opaque json blob.
That way we don’t have to bother changing the remote store protocol
when the definition of `Realisation` changes, as long as we keep the
json backwards-compatible
2021-05-19 11:45:16 +02:00
regnat
184558834a Extract a generic computeClosure function
Move the `closure` logic of `computeFSClosure` to its own (templated) function.

This doesn’t bring much by itself (except for the ability to properly
test the “closure” functionality independently from the rest), but it
allows reusing it (in particular for the realisations which will require
a very similar closure computation)
2021-05-19 11:44:58 +02:00
Eelco Dolstra
7234cf39be
Merge pull request #4831 from matthewbauer/fix-extra-slash-in-canon-path
Fix extra slash in canonPath output
2021-05-19 11:10:43 +02:00
Matthew Bauer
3d90ab9345 Fix extra slash in canonPath output
When you have a symlink like:

  /tmp -> ./private/tmp

you need to resolve ./private/tmp relative to /tmp’s dir: ‘/’. Unlike
any other path output by dirOf, / ends with a slash. We don’t want
trailing slashes here since we will append another slash in the next
comoponent, so clear s like we would if it was a symlink to an absoute
path.

This should fix at least part of the issue in
https://github.com/NixOS/nix/issues/4822, will need confirmation that
it actually fixes the problem to close though.

Introduced in f3f228700a.
2021-05-18 16:38:55 -05:00
regnat
59d0de6ea1 Restore an accidentally suppressed negation
Accidentally removed in ca96f52194. This
caused `nix run` to systematically fail with

```
error: app program '/nix/store/…' is not in the Nix store
```
2021-05-18 13:54:05 +02:00
Eelco Dolstra
de77d1b924
Merge pull request #4827 from matthewbauer/run-bashrc-first-in-nix-develop
Source bashrc first in nix develop
2021-05-18 11:31:52 +02:00
Matthew Bauer
5fd8cf7667 Source bashrc first in nix develop
~/.bashrc should be sourced first in the rc script so that PATH &
other env vars give precedence over the bashrc PATH.

Also, in my bashrc I alias rm as:

  alias rm='rm -Iv'

To avoid running this alias (which shows ‘removed '/tmp/nix-shell.*'),
we can just prefix rm with command.
2021-05-17 15:07:00 -05:00
regnat
ca96f52194 Split the parsing of an App and its resolving
That way things (like `nix flake check`) can evaluate the `app` outputs
without having to build anything
2021-05-17 17:50:41 +02:00
Eelco Dolstra
bd6cf25952
Merge pull request #4819 from NixOS/ca/fix-nix-run
Resolve the program path in `nix run`
2021-05-17 16:28:14 +02:00
Eelco Dolstra
6849ae82de
Merge pull request #4818 from NixOS/ca/cli-use-builtpaths
Enforce the use of properly built paths in libcmd
2021-05-17 16:15:40 +02:00
regnat
f0c0052d4b Resolve the program path in nix run
Fix #4768
2021-05-17 09:48:51 +02:00
regnat
2105084645 Enfore the use of properly built paths in libcmd
Replace `DerivedPathWithHints` by a new `BuiltPath` type that serves as
a proof that the corresponding path has been built.
2021-05-17 08:45:08 +02:00
Emily
559a504da7 sandbox: allow SystemVersionCompat.plist on Darwin
For whatever reason, many programs trying to access SystemVersion.plist
also open SystemVersionCompat.plist; this includes Python code and
coreutils’ `cat(1)` (but not the native macOS `/bin/cat`). Illustratory
`dtruss(1m)` output:

    open("/System/Library/CoreServices/SystemVersion.plist\0", 0x0, 0x0)		 = 3 0
    open("/System/Library/CoreServices/SystemVersionCompat.plist\0", 0x0, 0x0)		 = 4 0

I assume this is a Big Sur change relating to the 10.16.x/11.x
version compatibility divide and that it’s something along the lines of
a hook inside libSystem.

Fixes a lot of sandboxed package builds under Big Sur.
2021-05-17 01:24:31 +01:00
Eelco Dolstra
340f831ebe
Merge pull request #4798 from matthewbauer/relock-wait-for-build-slot-goals
Relock wait for build slot goals
2021-05-13 13:07:35 +02:00
Eelco Dolstra
89a4ede92b
Merge pull request #4801 from matthewbauer/fix-tokenize-output-names
Fix tokenize output names in DerivedPath
2021-05-13 13:06:29 +02:00
Matthew Bauer
8c7e043de2 Fix tokenize output names in drv
This should fix the issue described in
https://discourse.nixos.org/t/derivation-does-not-have-wanted-outputs-dev-out/12905.

Specifically, we get an error of

  error: derivation '/nix/store/_.drv' does not have wanted outputs 'dev,out'

when a path like /nix/store/_.drv!dev,out is sent to the daemon.
2021-05-12 21:40:28 -05:00
Matthew Bauer
9a14335845 Relock wait for build slot goals
When we don’t have enough free job slots to run a goal, we put it in
the waitForBuildSlot list & unlock its output locks. This will
continue from where we left off (tryLocalBuild). However, we need the
locks to get reacquired when/if the goal ever restarts. So, we need to
send it back through tryToBuild to get reqacquire those locks.

I think this bug was introduced in
https://github.com/NixOS/nix/pull/4570. It leads to some builds
starting without proper locks.
2021-05-12 12:15:32 -05:00
regnat
ec613603ba DerivedPathWithHints -> BuiltPath
Just a renaming for now
2021-05-12 16:19:51 +02:00
Lorenzo Manacorda
4029f4b05b doc: mention Priority for substituters 2021-05-12 13:27:05 +02:00
Eelco Dolstra
7f9759b18d
Merge pull request #4781 from NixOS/locally_cache_the_remote_realisations
Add a realisations disk cache
2021-05-10 20:37:57 +02:00
Eelco Dolstra
8768398d5f
Merge pull request #4787 from Ma27/builtins-ceil-floor
Implement `builtins.floor` and `builtins.ceil` using the C library functions internally
2021-05-10 20:37:27 +02:00
regnat
d5d19582ef Simplify the realisations disk cache 2021-05-10 17:47:14 +02:00
regnat
ab96c1ee50 Remove useless parents
I never remember the exact syntax of the `switch` statement
2021-05-10 17:36:49 +02:00
Maximilian Bosch
bec33858bb
primops: math.h -> cmath
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-05-10 16:41:10 +02:00
Eelco Dolstra
b68a85df0a
Merge pull request #4776 from NixOS/fix-ca-normalization
Properly normalize the content-addressed paths
2021-05-10 16:11:38 +02:00
Maximilian Bosch
7f7f99f350
Implement builtins.floor and builtins.ceil using the C library functions internally
Closes #4782

Note: even though the type is internally called `NixFloat`, it's
actually a `double`.
2021-05-10 12:19:32 +02:00