Commit graph

3158 commits

Author SHA1 Message Date
Eelco Dolstra
0961c1068a Merge branch 'accept_conf' of https://github.com/tomberek/nix 2021-11-18 12:56:09 +01:00
Eelco Dolstra
262a3c7ce3 Simplify 2021-11-18 12:12:31 +01:00
Farid Zakaria
bc14465e08 Fix stack buffer overflow
Fix a stack buffer overflow found by running MemorySanitizer.
2021-11-18 04:05:25 +00:00
Eelco Dolstra
6463eaca14
Merge pull request #5472 from NixOS/async-realisation-substitution
async realisation substitution
2021-11-16 12:54:20 +01:00
Tom Bereknyei
30496af598 Adds an accept-flake-config flag 2021-11-12 09:50:07 -05:00
Eelco Dolstra
bceda30498 Typo 2021-11-12 13:41:15 +01:00
Eelco Dolstra
67179472df
Merge pull request #5494 from tweag/balsoft/allow-references-in-addPath
Allow references in addPath
2021-11-09 15:57:39 +01:00
Alexander Bantyev
0b005bc9d6
addToStore, addToStoreFromDump: refactor: pass refs by const reference
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
2021-11-09 12:24:49 +03:00
Eelco Dolstra
ff2af4d64e Unshare mount namespace in main()
Doing it as a side-effect of calling LocalStore::makeStoreWritable()
is very ugly.

Also, make sure that stopping the progress bar joins the update
thread, otherwise that thread should be unshared as well.
2021-11-08 22:00:45 +01:00
Eelco Dolstra
7a71621b7c Merge branch 'fix-writable-shell' of https://github.com/yorickvP/nix 2021-11-08 21:12:51 +01:00
Alexander Bantyev
9d4dcff37a
addPath: allow paths with references
Since 4806f2f6b0, we can't have paths with
references passed to builtins.{path,filterSource}. This prevents many cases
of those functions called on IFD outputs from working. Resolve this by
passing the references found in the original path to the added path.
2021-11-05 22:41:30 +03:00
regnat
1f3c3a3785 Make the flake options work when using the daemon
When setting flake-local options (with the `nixConfig` field), forward
these options to the daemon in case we’re using one.

This is necessary in particular for options like `binary-caches` or
`post-build-hook` to make sense.

Fix <343239fc8a (r44356843)>
2021-11-05 16:19:16 +01:00
Doron Behar
14fcf17277 libstore: Use unix-dotfile vfs if useSQLiteWAL is false 2021-11-03 14:19:11 +02:00
regnat
f4c869977c Make the DrvOutputSubstitutionGoal more async 2021-11-03 06:51:34 +01:00
regnat
fbc70034b3 Make the realisation fetching from binary caches async
That way we can fetch several realisations from the same cache in
parallel
2021-11-03 06:51:34 +01:00
regnat
96670ed216 Expose an async interface for queryRealisation
Doesn’t change much so far because everything is still using it
synchronously, but should allow the binary cache to fetch stuff in
parallel
2021-11-03 06:51:34 +01:00
Eelco Dolstra
e6795c4350 Style 2021-10-29 14:45:13 +02:00
Eelco Dolstra
6e30d9b69f Merge branch 'master' of https://github.com/alekswn/nix 2021-10-29 14:42:26 +02:00
Eelco Dolstra
33d04e8a8d Use nix::connect() to connect to the garbage collector 2021-10-28 22:51:11 +02:00
Eelco Dolstra
22c35ea5b8 Remove unused variable 2021-10-28 22:51:11 +02:00
Eelco Dolstra
a7d4f3411e Merge remote-tracking branch 'origin/master' into non-blocking-gc 2021-10-28 14:56:55 +02:00
Eelco Dolstra
f2280749b1 If max-jobs == 0, do preferLocalBuild on remote builders 2021-10-27 14:21:31 +02:00
Eelco Dolstra
6e684d1b87 daemon: Accept 'repeat' setting from untrusted users
Fixes #5352.
2021-10-27 13:09:13 +02:00
regnat
af99941279 Make experimental-features a proper type
Rather than having them plain strings scattered through the whole
codebase, create an enum containing all the known experimental features.

This means that
- Nix can now `warn` when an unkwown experimental feature is passed
  (making it much nicer to spot typos and spot deprecated features)
- It’s now easy to remove a feature altogether (once the feature isn’t
  experimental anymore or is dropped) by just removing the field for the
  enum and letting the compiler point us to all the now invalid usages
  of it.
2021-10-26 07:02:31 +02:00
Alexander Bantyev
b9234142f5
addToStore, addToStoreFromDump: add references argument
Allow to pass a set of references to be added as info to the added paths.
2021-10-23 21:30:51 +03:00
Alexey Novikov
e989c83b44 Add error reporting to machine spec paser
Currently machine specification (`/etc/nix/machine`) parser fails
with a vague exception if the file had incorrect format.
This commit adds verbose exceptions and unit-tests for the parser.
2021-10-17 12:45:56 +04:00
Eelco Dolstra
10f9a8e77d Add a test for the non-blocking GC 2021-10-15 16:52:37 +02:00
Eelco Dolstra
ac54c6faa6 Fix main GC thread exiting 2021-10-15 16:36:48 +02:00
Yorick van Pelt
fcb8af550f
Restore parent mount namespace in restoreProcessContext
This ensures any started processes can't write to /nix/store (except
during builds). This partially reverts 01d07b1e, which happened because
of #2646.

The problem was only happening after nix downloads anything, causing
me to suspect the download thread. The problem turns out to be:
"A  process  can't  join a new mount namespace if it is sharing
filesystem-related attributes with another process", in this case this
process is the curl thread.

Ideally, we might kill it before spawning the shell process, but it's
inside a static variable in the getFileTransfer() function. So
instead, stop it from sharing FS state using unshare(). A strategy
such as the one from #5057 (single-threaded chroot helper binary) is
also very much on the table.

Fixes #4337.
2021-10-15 16:25:49 +02:00
Eelco Dolstra
e0936ae38f Fix crash when a GC client disconnects
The client thread can't just delete its own thread object from
connections, it has to detach it.
2021-10-15 16:12:21 +02:00
Eelco Dolstra
304180d0de Memoize queryReferrers() 2021-10-15 12:20:23 +02:00
yvt
a9d9e55551 Add another path where a Rosetta 2 configuration file is possibly located 2021-10-15 09:48:15 +09:00
Eelco Dolstra
17e6ebcc90 Speed up GC by marking entire closures as live 2021-10-14 14:13:57 +02:00
Eelco Dolstra
0154fa30cf Remove GCState 2021-10-14 13:52:49 +02:00
Eelco Dolstra
0317ffdad3 Move deleteFromStore() 2021-10-14 13:34:48 +02:00
Eelco Dolstra
0be8cc1466 pathInfoCache: Use the entire base name as the cache key
This fixes a bug in the garbage collector where if a path
/nix/store/abcd-foo is valid, but we do a
isValidPath("/nix/store/abcd-foo.lock") first, then a negative entry
for /nix/store/abcd is added to pathInfoCache, so /nix/store/abcd-foo
is subsequently considered invalid and deleted.
2021-10-14 13:28:22 +02:00
Eelco Dolstra
eab934cb2a Make the canReachRoots() traversal non-recursive 2021-10-14 12:34:32 +02:00
Eelco Dolstra
09b14ea97a Cleanup 2021-10-14 10:04:13 +02:00
Eelco Dolstra
35c98a59c5 Fix GC when there are cycles in the referrers graph
(where "referrers" includes the reverse of derivation outputs and
derivers). Now we do a full traversal to look if we can reach any
root. If not, all paths reached can be deleted.
2021-10-13 12:12:44 +02:00
Eelco Dolstra
1785ba2980 Simplify 2021-10-13 12:12:44 +02:00
Eelco Dolstra
262520fcfe Use a thread per connection 2021-10-13 12:12:44 +02:00
Eelco Dolstra
ff453b06f9 Fix auto-gc 2021-10-13 12:12:44 +02:00
Eelco Dolstra
8614cf1334 Non-blocking garbage collector
The garbage collector no longer blocks other processes from
adding/building store paths or adding GC roots. To prevent the
collector from deleting store paths just added by another process,
processes need to connect to the garbage collector via a Unix domain
socket to register new temporary roots.
2021-10-13 12:12:44 +02:00
Eelco Dolstra
9947f1646a Remove syncWithGC() 2021-10-13 12:12:44 +02:00
Eelco Dolstra
8eac7dfad4 Remove trash directory 2021-10-13 12:12:44 +02:00
Eelco Dolstra
abd685d373 Merge branch 'feature/comp-level' of https://github.com/tomberek/nix 2021-10-13 10:45:44 +02:00
Eelco Dolstra
5176b072ed Cleanup 2021-10-12 16:43:00 +02:00
Eelco Dolstra
3a778ea8a0 Merge branch 'nix-repl-download-interruption' of https://github.com/Ma27/nix 2021-10-12 16:34:38 +02:00
Tom Bereknyei
03bb8f84e0 Add compression level for NARs
Based off on @dtzWill's #2276
2021-10-12 02:14:36 -04:00
regnat
7466048d39 (partially) Revert "Don't copy in rethrow"
This reverts some parts of commit
8430a8f086 which was trying to rethrow
some exceptions while we weren’t in the context of a `catch` block,
causing some weird “terminate called without an active exception”
errors.

Fix #5368
2021-10-11 10:51:22 +02:00