Commit graph

185 commits

Author SHA1 Message Date
Guillaume Maudoux
ebc86550f9 Make roots a map of store paths to pinning links
This new structure makes more sense as there may be many sources rooting
the same store path. Many profiles can reference the same path but this
is even more true with /proc/<pid>/maps where distinct pids can and
often do map the same store path.
This implementation is also more efficient as the `Roots` map contains
only one entry per rooted store path.
2019-03-10 00:56:09 +01:00
Guillaume Maudoux
a17f86ce3a Obfuscate memory roots for non-root users 2019-03-10 00:56:09 +01:00
Guillaume Maudoux
43331d6344 detail memory roots 2019-03-10 00:56:09 +01:00
Eelco Dolstra
2e244fb68f
Merge pull request #2234 from symphorien/esrch2
libstore/gc.cc: ignore ESRCH when reading symlinks in /proc
2018-06-18 09:01:55 +02:00
Symphorien Gibol
ef8122064b libstore/gc.cc: ignore ESRCH when reading symlinks in /proc
readlink is also affected by the problem fixed for regular files in
c567afe355df
2018-06-16 18:04:09 +02:00
Eelco Dolstra
3c5d9f478d
std::random_shuffle -> std::shuffle
The former is removed in C++17.
2018-06-13 16:56:19 +02:00
Symphorien Gibol
8c567afe35 libstore/gc.cc: ignore ESRCH when reading /proc
If a process disappears between the time /proc/[pid]/maps is opened and
the time it is read, the read() syscall will return ESRCH. This should be ignored.
2018-06-11 16:29:47 +02:00
Will Dietz
56253bb08f rand() -> random(), since we use srandom().
rand() requires we call srand() instead,
but might as well use random().
2018-03-06 17:44:25 -06:00
Eelco Dolstra
855699855f
Remove obsolete references to manifests
Closes #323.
2018-02-01 10:39:16 +01:00
Eelco Dolstra
308ecf6361
Show the pid of temporary roots 2017-09-14 15:02:52 +02:00
Eelco Dolstra
da1e4fdfb5
nix-store -q --roots / --gc --print-roots: Print temporary / in-memory roots
For example,

  $ nix-store -q --roots /nix/store/7phd2sav7068nivgvmj2vpm3v47fd27l-patchelf-0.8pre845_0315148
  {temp:1}

denotes that the path is only being kept alive by a temporary root
(i.e. /nix/var/nix/temproots/). Similarly,

  $ nix-store --gc --print-roots
  ...
  {memory:9} -> /nix/store/094gpjn9f15ip17wzxhma4r51nvsj17p-curl-7.53.1

shows that curl is being used by some process.
2017-09-14 14:38:36 +02:00
Eelco Dolstra
1277aab219
Fix abort when the GC thread gets an exception 2017-09-06 11:37:58 +02:00
Eelco Dolstra
0b606aad46
Add automatic garbage collection
Nix can now automatically run the garbage collector during builds or
while adding paths to the store. The option "min-free = <bytes>"
specifies that Nix should run the garbage collector whenever free
space in the Nix store drops below <bytes>. It will then delete
garbage until "max-free" bytes are available.

Garbage collection during builds is asynchronous; running builds are
not paused and new builds are not blocked. However, there also is a
synchronous GC run prior to the first build/substitution.

Currently, no old GC roots are deleted (as in "nix-collect-garbage
-d").
2017-09-05 20:43:42 +02:00
Eelco Dolstra
b932ea58ec
GC: Don't delete own temproots file
Since file locks are per-process rather than per-file-descriptor, the
garbage collector would always acquire a lock on its own temproots
file and conclude that it's stale.
2017-09-05 20:39:57 +02:00
Eelco Dolstra
c2154d4c84
Rename a few configuration options
In particular, drop the "build-" and "gc-" prefixes which are
pointless. So now you can say

  nix build --no-sandbox

instead of

  nix build --no-build-use-sandbox
2017-08-31 14:28:25 +02:00
Jörg Thalheim
2fd8f8bb99 Replace Unicode quotes in user-facing strings by ASCII
Relevant RFC: NixOS/rfcs#4

$ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-07-30 12:32:45 +01:00
Eelco Dolstra
c0015e87af
Support base-64 hashes
Also simplify the Hash API.

Fixes #1437.
2017-07-04 15:07:41 +02:00
Eelco Dolstra
b01d62285c
Improve progress indicator 2017-05-16 16:09:57 +02:00
Eelco Dolstra
1bb87c0487
Remove debug statement 2017-04-24 15:01:28 +02:00
Eelco Dolstra
749696e71c
Detect lsof
Also, don't use lsof on Linux since it's not needed.

Fixes #1328.
2017-04-20 19:11:45 +02:00
Eelco Dolstra
105f8ffc98
Minor cleanup
Also, possible fix for #1310 on 32-bit systems.
2017-04-10 11:27:33 +02:00
Eelco Dolstra
8079ab87a2 AutoCloseDir: Use std::unique_ptr 2017-01-16 22:39:27 +01:00
Eelco Dolstra
b30d1e7ada
Don't delete .check directories of running builds
We need to keep them around for diffoscope.
2016-12-08 21:38:58 +01:00
Eelco Dolstra
215b70f51e
Revert "Get rid of unicode quotes (#1140)"
This reverts commit f78126bfd6. There
really is no need for such a massive change...
2016-11-26 00:38:01 +01:00
Guillaume Maudoux
f78126bfd6 Get rid of unicode quotes (#1140) 2016-11-25 15:48:27 +01:00
Eelco Dolstra
c55bf085eb printMsg(lvlError, ...) -> printError(...) etc. 2016-09-21 16:54:53 +02:00
Shea Levy
3c46fe62b8 find-runtime-roots: fold into gc.cc 2016-07-21 07:04:41 -04:00
Shea Levy
cb5e7254b6 Modernize AutoCloseFD 2016-07-11 15:44:44 -04:00
Eelco Dolstra
202683a4fc Use O_CLOEXEC in most places 2016-06-09 16:37:08 +02:00
Eelco Dolstra
4494000e04 LocalStore: Allow the physical and logical store directories to differ
This is primarily to subsume the functionality of the
copy-from-other-stores substituter. For example, in the NixOS
installer, we can now do (assuming we're in the target chroot, and the
Nix store of the installation CD is bind-mounted on /tmp/nix):

  $ nix-build ... --option substituters 'local?state=/tmp/nix/var&real=/tmp/nix/store'

However, unlike copy-from-other-stores, this also allows write access
to such a store. One application might be fetching substitutes for
/nix/store in a situation where the user doesn't have sufficient
privileges to create /nix, e.g.:

  $ NIX_REMOTE="local?state=/home/alice/nix/var&real=/home/alice/nix/store" nix-build ...
2016-06-02 16:02:48 +02:00
Eelco Dolstra
812c0dfbe2 Allow setting the state directory as a store parameter
E.g. "local?store=/tmp/store&state=/tmp/var".
2016-06-02 16:02:48 +02:00
Eelco Dolstra
7850d3d279 Make the store directory a member variable of Store 2016-06-01 16:24:17 +02:00
Eelco Dolstra
c6a21aed07 More GC fixes 2016-05-04 16:16:53 +02:00
Eelco Dolstra
9eba2c3945 Fix "path is not in the Nix store" during GC 2016-04-25 19:18:45 +02:00
Eelco Dolstra
41633f9f73 Improved logging abstraction
This also gets rid of --log-type, since the nested log type isn't
useful in a multi-threaded situation, and nobody cares about the
"pretty" log type.
2016-04-25 19:18:45 +02:00
Eelco Dolstra
21ef1670b3 Fix test failures 2016-04-21 18:21:25 +02:00
Eelco Dolstra
e0204f8d46 Move path info caching from BinaryCacheStore to Store
Caching path info is generally useful. For instance, it speeds up "nix
path-info -rS /run/current-system" (i.e. showing the closure sizes of
all paths in the closure of the current system) from 5.6s to 0.15s.

This also eliminates some APIs like Store::queryDeriver() and
Store::queryReferences().
2016-04-19 18:52:53 +02:00
Eelco Dolstra
f398949b40 Make LocalStore thread-safe
Necessary for multi-threaded commands like "nix verify-paths".
2016-04-08 18:07:13 +02:00
Eelco Dolstra
152b1d6bf9 deletePath(): Succeed if path doesn't exist
Also makes it robust against concurrent deletions.
2016-02-24 17:44:55 +01:00
Eelco Dolstra
28e7e29abd Eliminate reserveSpace flag 2016-02-24 17:44:55 +01:00
Eelco Dolstra
ae4a3cfa03 Move addPermRoot into Store 2016-02-11 16:14:42 +01:00
Eelco Dolstra
fa7cd5369b StoreAPI -> Store
Calling a class an API is a bit redundant...
2016-02-04 14:48:42 +01:00
Eelco Dolstra
c10c61449f Eliminate the "store" global variable
Also, move a few free-standing functions into StoreAPI and Derivation.

Also, introduce a non-nullable smart pointer, ref<T>, which is just a
wrapper around std::shared_ptr ensuring that the pointer is never
null. (For reference-counted values, this is better than passing a
"T&", because the latter doesn't maintain the refcount. Usually, the
caller will have a shared_ptr keeping the value alive, but that's not
always the case, e.g., when passing a reference to a std::thread via
std::bind.)
2016-02-04 14:28:26 +01:00
Eelco Dolstra
6bd2c7bb38 OCD: foreach -> C++11 ranged for 2015-07-17 20:13:56 +02:00
Eelco Dolstra
ff4de4cb27 GC: Handle ENOSPC creating/moving to the trash directory
Issue #564.
2015-06-30 21:41:26 +02:00
Daniel Hahler
e659978ced Fix typos: s/the the/the/ 2015-03-06 16:43:22 +01:00
Eelco Dolstra
28f22b4653 Ensure we're writing to stderr in the builder
http://hydra.nixos.org/build/17862041
2014-12-12 14:35:44 +01:00
Eelco Dolstra
8299aaf079 Disable vacuuming the DB after garbage collection
Especially in WAL mode on a highly loaded machine, this is not a good
idea because it results in a WAL file of approximately the same size
ad the database, which apparently cannot be deleted while anybody is
accessing it.
2014-11-19 18:14:24 +01:00
Eelco Dolstra
1256ab3b44 Clean up temp roots in a more C++ way 2014-11-19 17:07:29 +01:00
Eelco Dolstra
06a86aee15 nix-store --gc: Don't warn about missing manifests directory 2014-11-04 10:41:29 +01:00