Commit graph

249 commits

Author SHA1 Message Date
Sergei Trofimovich
d58453f72e gc: don't visit implicit referrers on garbage collection
Before the change garbage collector was not considering
`.drv` and outputs as alive even if configuration says otherwise.
As a result `nix store gc --dry-run` could visit (and parse)
`.drv` files multiple times (worst case it's quadratic).

It happens because `alive` set was populating only runtime closure
without regard for actual configuration. The change fixes it.

Benchmark: my system has about 139MB, 40K `.drv` files.

Performance before the change:

    $ time nix store gc --dry-run
    real    4m22,148s

Performance after the change:

    $ time nix store gc --dry-run
    real    0m14,178s
2022-03-17 18:47:29 +00:00
Sergei Trofimovich
6b1872312f nix store gc: account for auto-optimised store
Before the change on a system with `auto-optimise-store = true`:

    $ nix store gc --verbose --max 1

deleted all the paths instead of one path (we requested 1 byte limit).

It happens because every file in `auto-optimise-store = true` has at
least 2 links: file itself and a link in /nix/store/.links/ directory.

The change conservatively assumes that any file that has one (as before)
or two links (assume auto-potimise mode) will free space.

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2022-03-13 07:24:48 +00:00
Eelco Dolstra
a949673a5b Fix Darwin build
Fixes #6169
2022-02-28 15:48:26 +01:00
Eelco Dolstra
df552ff53e Remove std::string alias (for real this time)
Also use std::string_view in a few more places.
2022-02-25 16:13:02 +01:00
Eelco Dolstra
d17d46cfc2 More debug info 2021-12-13 19:03:08 +01:00
Eelco Dolstra
05fcc681ac Retry on ECONNREFUSED
https://hydra.nixos.org/build/161439235
2021-12-13 19:00:19 +01:00
Eelco Dolstra
6d30f9e6fe Explicitly make GC roots client sockets blocking
On macOS / BSD, these sockets inherit the non-blocking flag of the
server soocket, which is not what we want.

https://github.com/dotnet/runtime/issues/25069
https://bugs.python.org/issue7995

https://hydra.nixos.org/build/161439304
2021-12-13 18:59:50 +01:00
Alex Shabalin
db2e4489a5 Unify #if linux 2021-11-22 13:57:56 +01:00
Alex Shabalin
a18d9269a5 Fix build warnings on MacOS 2021-11-19 15:22:31 +01:00
Eelco Dolstra
33d04e8a8d Use nix::connect() to connect to the garbage collector 2021-10-28 22:51:11 +02:00
Eelco Dolstra
22c35ea5b8 Remove unused variable 2021-10-28 22:51:11 +02:00
Eelco Dolstra
10f9a8e77d Add a test for the non-blocking GC 2021-10-15 16:52:37 +02:00
Eelco Dolstra
ac54c6faa6 Fix main GC thread exiting 2021-10-15 16:36:48 +02:00
Eelco Dolstra
e0936ae38f Fix crash when a GC client disconnects
The client thread can't just delete its own thread object from
connections, it has to detach it.
2021-10-15 16:12:21 +02:00
Eelco Dolstra
304180d0de Memoize queryReferrers() 2021-10-15 12:20:23 +02:00
Eelco Dolstra
17e6ebcc90 Speed up GC by marking entire closures as live 2021-10-14 14:13:57 +02:00
Eelco Dolstra
0154fa30cf Remove GCState 2021-10-14 13:52:49 +02:00
Eelco Dolstra
0317ffdad3 Move deleteFromStore() 2021-10-14 13:34:48 +02:00
Eelco Dolstra
eab934cb2a Make the canReachRoots() traversal non-recursive 2021-10-14 12:34:32 +02:00
Eelco Dolstra
35c98a59c5 Fix GC when there are cycles in the referrers graph
(where "referrers" includes the reverse of derivation outputs and
derivers). Now we do a full traversal to look if we can reach any
root. If not, all paths reached can be deleted.
2021-10-13 12:12:44 +02:00
Eelco Dolstra
1785ba2980 Simplify 2021-10-13 12:12:44 +02:00
Eelco Dolstra
262520fcfe Use a thread per connection 2021-10-13 12:12:44 +02:00
Eelco Dolstra
ff453b06f9 Fix auto-gc 2021-10-13 12:12:44 +02:00
Eelco Dolstra
8614cf1334 Non-blocking garbage collector
The garbage collector no longer blocks other processes from
adding/building store paths or adding GC roots. To prevent the
collector from deleting store paths just added by another process,
processes need to connect to the garbage collector via a Unix domain
socket to register new temporary roots.
2021-10-13 12:12:44 +02:00
Eelco Dolstra
9947f1646a Remove syncWithGC() 2021-10-13 12:12:44 +02:00
Eelco Dolstra
8eac7dfad4 Remove trash directory 2021-10-13 12:12:44 +02:00
Eelco Dolstra
4202a3bc4e UDSRemoteStore: Support the 'root' store parameter
Useful when we're using a daemon with a chroot store, e.g.

  $ NIX_DAEMON_SOCKET_PATH=/tmp/chroot/nix/var/nix/daemon-socket/socket nix-daemon --store /tmp/chroot

Then the client can now connect with

  $ nix build --store unix:///tmp/chroot/nix/var/nix/daemon-socket/socket?root=/tmp/chroot nixpkgs#hello
2021-06-18 17:04:11 +02:00
volth
eee18f88dd
Handle amount of disk space saved by hard linking being negative
Fixes bogus messages like "currently hard linking saves 17592186044416.00 MiB".
2020-10-12 16:06:38 +00:00
John Ericson
6cc1541782 Split out local-fs-store.hh
This matches the already-existing `local-fs-store.cc`.
2020-10-09 20:18:08 +00:00
Eelco Dolstra
236d9ee7f7 lstat() cleanup 2020-09-23 19:17:28 +02:00
regnat
520895b1da Fix garbage collection of CA derivations
Fix #4026
2020-09-17 13:36:58 +02:00
Eelco Dolstra
8a945d6ddb
Remove gc-check-reachability 2020-09-03 11:30:15 +02:00
Eelco Dolstra
82b77a7726
addPermRoot(): Remove indirect flag 2020-09-03 11:29:34 +02:00
Eelco Dolstra
3f6e88a552 unsigned long long -> uint64_t 2020-07-30 13:34:04 +02:00
Eelco Dolstra
c0dd05131e toStorePath(): Return a StorePath and the suffix 2020-07-13 16:25:48 +02:00
Eelco Dolstra
29542865ce Remove StorePath::clone() and related functions 2020-06-16 22:20:18 +02:00
Eelco Dolstra
1fb762d11f Get rid of explicit ErrorInfo constructors 2020-06-15 14:06:58 +02:00
Ben Burdette
ef9dd9f9bc formatting and a few minor changes 2020-05-13 15:56:39 -06:00
Ben Burdette
c79d4addab consistent capitalization 2020-05-13 10:02:18 -06:00
Ben Burdette
bfca5fc395 change status messages to info level 2020-05-13 09:52:36 -06:00
Ben Burdette
59b1f5c701 Merge branch 'master' into errors-phase-2 2020-05-11 14:35:30 -06:00
Ben Burdette
e3901638b5 todo removal 2020-05-06 15:01:13 -06:00
Eelco Dolstra
74a1bfdcab
Merge pull request #3546 from guibou/nix_readfile_on_0_sized_files
builtins.readFile: do not truncate content
2020-05-06 11:33:55 +02:00
Jude Taylor
e2fc575c61 nix auto-gc: use fragment size 2020-05-04 14:42:06 -07:00
Ben Burdette
4b99c09f5c convert some errors 2020-05-01 14:32:06 -06:00
Guillaume Bouchard
7afcb5af98 Remove the drain argument from readFile
Now it is always `drain` (see previous commit).
2020-04-29 18:43:45 +02:00
Ben Burdette
e4fb9a3849 remove 'format' from Error constructor calls 2020-04-21 17:07:07 -06:00
Philipp Middendorf
04bedda0b6 gc.cc: Ignore hidden files in temproots 2020-04-09 17:05:29 +02:00
Eelco Dolstra
22a754c091
Fix GC failures on bad store path names
It failed on names like '/nix/store/9ip48nkc9rfy0a4yaw98lp6gipqlib1a-'.
2020-02-28 18:07:10 +01:00
Daiderd Jordan
66fccd5832
build: fix sandboxing on darwin
Starting ba87b08f85 getEnv now returns an
std::optional which means these getEnv() != "" conditions no longer happen
if the variables are not defined.
2020-01-05 20:23:52 +01:00