Commit graph

12855 commits

Author SHA1 Message Date
Théophane Hufschmitt
c55bea4204 Fix the html id of the list headers 2022-08-03 14:16:00 +02:00
Félix Baylac-Jacqué
b47b6a418d
tests/check.sh: add nix3-build check test 2022-08-03 12:17:13 +02:00
Valentin Gagarin
ceed4d4142 encode primitive as list with anchors
to make it consistent with builtins and configuration options
2022-08-03 11:25:41 +02:00
Théophane Hufschmitt
d1cda07ce4 Don't use -load_all on darwin
That flag breaks `-lc++fs` (introducing a duplicate symbol for some
reason). Besides, it was apparently needed for bzip2, but we're not using bzip2
anymore.
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
1ba5b3e001 Make moveFile more atomic
Rather than directly copying the source to its dest, copy it first to a
temporary location, and eventually move that temporary.
That way, the move is at least atomic from the point-of-view of the destination
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
90f9680733 Only use renameFile where needed
In most places the fallback to copying isn’t needed and can actually be
bad, so we’d rather not transparently fallback
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
d71d9e9fbf moveFile -> renameFile
`move` tends to have this `mv` connotation of “I will copy it for you if
needs be”
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
a4f0fd633c Link against c++fs on darwin
Required by the old clang version
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
c5db1821a9 Re-implement the recursive directory copy
The recursive copy from the stl doesn’t exactly do what we need because
1. It doesn’t delete things as we go
2. It doesn’t keep the mtime, which change the nars

So re-implement it ourselves. A bit dull, but that way we have what we want
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
6f89fb6008 rename: Fallback to a copy if the filesystems mismatch
In `nix::rename`, if the call to `rename` fails with `EXDEV` (failure
because the source and the destination are in a different filesystems)
switch to copying and removing the source.

To avoid having to re-implement the copy manually, I switched the
function to use the c++17 `filesystem` library (which has a `copy`
function that should do what we want).

Fix #6262
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
c2de0a232c Create a wrapper around stdlib’s rename
Directly takes some c++ strings, and gently throws an exception on error
(rather than having to inline this logic everywhere)
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
8119390abc Move some fs-related functions to their own file
Unclutter `util.cc` a bit
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
780a479386
Merge pull request #6851 from K900/patch-1
doc/distributed-builds: don't use deprecated alias
2022-08-02 07:17:09 +02:00
Eelco Dolstra
71b155b9e6 Move download stuff into tarball.hh 2022-08-01 16:05:58 +02:00
Eelco Dolstra
48012603b3 Move FSInputAccessor into a separate file 2022-08-01 16:00:12 +02:00
Eelco Dolstra
55c63c9b89 Remove InputScheme::fetchToStore()
InputSchemes now only have a getAccessor(). They could be implemented
internally by fetching the input to the store, but in that case they
will just return a FSInputAccessor.
2022-08-01 15:44:40 +02:00
K900
f675ba5331
doc/distributed-builds: don't use deprecated alias
`nix ping-store` -> `nix store ping`.
2022-08-01 13:50:35 +03:00
Félix Baylac-Jacqué
1467a98d4c
derivation-goal.cc: remove bmCheck custom return branch on buildDone
Once a derivation goal has been completed, we check whether or not
this goal was meant to be repeated to check its output.

An early return branch was preventing the worker to reach that repeat
code branch, hence breaking the --check command (#2619).

It seems like this early return branch is an artifact of a passed
refactoring. As far as I can tell, buildDone's main branch also
cleanup the tmp directory before returning.
2022-08-01 11:39:19 +02:00
Jeremy Fleischman
297f6b5d56
Fix link to hacking doc
Right now,
https://hydra.nixos.org/job/nix/master/build.x86_64-linux/latest/download-by-type/doc/manual/contributing/hacking.html
redirects to
https://hydra.nixos.org/build/183877779/download/1/manual/contributing/hacking.html,
which gives me a "500 Internal Server Error". Not super useful =(

Feel free to ignore if someone's working to fix the 500 I was running
into.
2022-07-30 09:12:50 -07:00
Rok Garbas
e9178d7d4a
Merge pull request #6844 from centromere/custom-nix-conf
docker.nix: Allow Nix configuration to be customized
2022-07-29 13:47:24 +02:00
Valentin Gagarin
27138f1ec6
manual: use singular in body, too
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2022-07-28 23:30:07 +02:00
Valentin Gagarin
41a3b315fd manual: values -> data types 2022-07-28 17:25:25 +02:00
Valentin Gagarin
8f4fab8fab manual: use singular for headings 2022-07-28 17:25:18 +02:00
Valentin Gagarin
4ff48854b8 manual: simple values -> primitives
"simple" is a loaded term
2022-07-28 17:23:57 +02:00
Valentin Gagarin
3063e5b94c manual: use subheadings for primitive types
this gives us HTML anchors for each of them
2022-07-28 17:23:57 +02:00
Eelco Dolstra
86fcd4f692
Merge pull request #6845 from fricklerhandwerk/attrset
manual: set -> attribute set
2022-07-28 16:55:03 +02:00
Valentin Gagarin
85cdaebcd6 manual: set -> attribute set
reword description to have shorter sentences.
2022-07-28 16:10:24 +02:00
Valentin Gagarin
be4654c344 manual: fix section title in table of contents 2022-07-28 15:55:41 +02:00
Alex Wied
228028fc1a docker.nix: Allow Nix configuration to be customized 2022-07-28 03:36:39 -04:00
Eelco Dolstra
360a1284db Fix onError 2022-07-27 16:41:26 +02:00
Eelco Dolstra
59a8e05754 Fix test 2022-07-27 14:41:32 +02:00
Eelco Dolstra
f780539406 Add abstract interface for writing files to an Input 2022-07-27 14:26:15 +02:00
Eelco Dolstra
bb4d35dcca Make 'nix edit' etc. work again 2022-07-26 17:06:45 +02:00
Eelco Dolstra
1790698a74 Rename fetch() -> fetchToStore(), lazyFetch() -> getAccessor() 2022-07-25 16:30:50 +02:00
Eelco Dolstra
7a64bb7f1c Rename Input::fetch() to fetchToStore() 2022-07-25 15:48:31 +02:00
Eelco Dolstra
a3427a1478 nix registry pin: Warn if the resolved flake is not locked 2022-07-25 15:48:16 +02:00
Eelco Dolstra
184c6605b0 nix flake pin: Use lazyFetch() 2022-07-25 15:44:45 +02:00
Eelco Dolstra
c0555c8e1f Remove Input::computeStorePath() 2022-07-25 15:35:53 +02:00
Eelco Dolstra
4f8f52ae58 Remove Tree datatype 2022-07-25 15:21:37 +02:00
Eelco Dolstra
e7faf65784 Remove FlakeRef::fetchTree() 2022-07-25 15:12:28 +02:00
Eelco Dolstra
022390af5a Merge remote-tracking branch 'origin/master' into lazy-trees 2022-07-22 15:27:40 +02:00
Eelco Dolstra
f97489e7ab Remove parent hackery from the path fetcher
Relative paths of subflakes are now handled in lockFlake().
2022-07-22 15:24:41 +02:00
Eelco Dolstra
3d27ce36d0 Restore the evaluation cache 2022-07-22 15:19:30 +02:00
Eelco Dolstra
c73a7584fb Fix path display 2022-07-22 15:11:15 +02:00
Théophane Hufschmitt
2805439335
Merge pull request #6814 from amjoseph-nixpkgs/pr/sandbox-error-messages
local-derivation-goal.cc: improve error messages when sandboxing fails
2022-07-22 13:27:52 +02:00
Théophane Hufschmitt
e10807cdbb
Merge pull request #6813 from centromere/cgroup-cpu-detection
libstore/globals.cc: Automatically set cores based on cgroup CPU limit
2022-07-22 10:15:32 +02:00
Eelco Dolstra
7edacb6248 Fix case 2022-07-21 16:06:22 +02:00
Théophane Hufschmitt
7ed91d6c6a
Merge branch 'master' into parallel-nix-copy 2022-07-20 10:05:34 +02:00
Alex Wied
722de8ddcc libstore/globals.cc: Move cgroup detection to libutil 2022-07-19 16:25:53 -04:00
Alex Wied
1af5d798a4 libstore/globals.cc: Automatically set cores based on cgroup CPU limit
By default, Nix sets the "cores" setting to the number of CPUs which are
physically present on the machine. If cgroups are used to limit the CPU
and memory consumption of a large Nix build, the OOM killer may be
invoked.

For example, consider a GitLab CI pipeline which builds a large software
package. The GitLab runner spawns a container whose CPU is limited to 4
cores and whose memory is limited to 16 GiB. If the underlying machine
has 64 cores, Nix will invoke the build with -j64. In many cases, that
level of parallelism will invoke the OOM killer and the build will
completely fail.

This change sets the default value of "cores" to be
ceil(cpu_quota / cpu_period), with a fallback to
std:🧵:hardware_concurrency() if cgroups v2 is not detected.
2022-07-19 16:03:58 -04:00