Commit graph

110 commits

Author SHA1 Message Date
John Ericson
938650700f Merge branch 'path-info' into ca-drv-exotic 2022-03-10 16:20:01 +00:00
John Ericson
8ba089597f Merge remote-tracking branch 'upstream/master' into path-info 2022-03-10 15:48:14 +00:00
Eelco Dolstra
95bd5da341 queryMissing(): Use a bigger thread pool
This is a temporary hack until we make queryMissing() async.

Closes #5324.
2021-12-13 21:11:37 +01:00
Eelco Dolstra
09b14ea97a Cleanup 2021-10-14 10:04:13 +02:00
John Ericson
edf67e1508 Merge branch 'path-info' into ca-drv-exotic 2021-10-01 17:25:22 +00:00
John Ericson
13b6b64589 Merge remote-tracking branch 'upstream/master' into path-info 2021-10-01 17:12:54 +00:00
John Ericson
9af9ab4212 Merge branch 'path-info' into ca-drv-exotic 2021-09-30 22:42:15 +00:00
John Ericson
f4f3203aa7 Merge remote-tracking branch 'upstream/master' into path-info 2021-09-30 22:41:53 +00:00
John Ericson
242f9bf3dc std::visit by reference
I had started the trend of doing `std::visit` by value (because a type
error once mislead me into thinking that was the only form that
existed). While the optomizer in principle should be able to deal with
extra coppying or extra indirection once the lambdas inlined, sticking
with by reference is the conventional default. I hope this might even
improve performance.
2021-09-30 21:35:09 +00:00
Eelco Dolstra
26d2c62225
Merge pull request #4906 from NixOS/collect-garbage-ca
Make `computeFSClosure` ca-aware
2021-06-23 10:07:08 +02:00
Théophane Hufschmitt
c13d7d0b97 Pass more values by reference
Rather than copying them around everywhere

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-06-23 08:16:34 +02:00
regnat
96d7170e12 Don’t check the deriver field on computeFSClosure
That doesn’t really make sense with CA derivations (and wasn’t even
really correct before because of FO derivations, though that probably
didn’t matter much in practice)
2021-06-12 12:24:53 +02:00
regnat
7ac038fa4b Make computeFSClosure ca-aware
Fix #4820 by preventing nix-collect garbage from crashing if
`keep-outputs` or `keep-derivations` is true
2021-06-11 09:26:49 +02:00
regnat
1f3ff0d193 Aso track the output path of the realisation dependencies 2021-05-26 17:09:21 +02:00
regnat
8c30acc3e8 Properly track the drvoutput references when building 2021-05-26 16:59:09 +02:00
regnat
184558834a Extract a generic computeClosure function
Move the `closure` logic of `computeFSClosure` to its own (templated) function.

This doesn’t bring much by itself (except for the ability to properly
test the “closure” functionality independently from the rest), but it
allows reusing it (in particular for the realisations which will require
a very similar closure computation)
2021-05-19 11:44:58 +02:00
John Ericson
d0ed11ca72 Merge commit '1b6cf0d5f56e166a1cbbf38142375b7a92fc88f2' into ca-drv-exotic 2021-04-05 19:06:43 -04:00
John Ericson
386765e3ff Merge commit 'd5cef6c33a051dfc672cb1e5f4739948b167315b' into ca-drv-exotic 2021-04-05 19:06:37 -04:00
John Ericson
cdc9f34a44 Merge commit 'e12308dd63f0ad27b22dcdb3da89c411eebcad2b' into ca-drv-exotic 2021-04-05 19:06:30 -04:00
John Ericson
1b6cf0d5f5 Merge remote-tracking branch 'upstream/master' into path-info 2021-04-05 18:47:33 -04:00
John Ericson
d5cef6c33a Merge commit '9dfb97c987d8b9d6a3d15f016e40f22f91deb764' into path-info 2021-04-05 18:40:30 -04:00
John Ericson
e12308dd63 Merge branch 'master' into path-info 2021-04-05 18:29:52 -04:00
John Ericson
9b805d36ac Rename Buildable 2021-04-05 09:52:25 -04:00
John Ericson
9dfb97c987 "newtype" BuildableReq
This makes for better types errors and allows us to give it methods.
2021-04-05 09:35:55 -04:00
John Ericson
255d145ba7 Use BuildableReq for buildPaths and ensurePath
This avoids an ambiguity where the `StorePathWithOutputs { drvPath, {}
}` could mean "build `brvPath`" or "substitute `drvPath`" depending on
context.

It also brings the internals closer in line to the new CLI, by
generalizing the `Buildable` type is used there and makes that
distinction already.

In doing so, relegate `StorePathWithOutputs` to being a type just for
backwards compatibility (CLI and RPC).
2021-04-05 08:33:00 -04:00
John Ericson
6212e89bf6 Avoid some StorePath -> Path -> StorePath roundtrips
There were done when StorePath was defined in Rust and there were some
FFI issues. This is no longer an issue.
2021-03-05 00:49:46 +00:00
John Ericson
47f0d7b798 Cleanup tabs 2020-10-13 16:22:30 +00:00
John Ericson
a4e5de1b9d Derivations can output "text-hashed" data
In particular, this means that derivations can output derivations. But
that ramification isn't (yet!) useful as we would want, since there is
no way to have a dependent derivation that is itself a dependent
derivation.
2020-10-13 02:15:48 +00:00
John Ericson
45ca7c3e4b Merge remote-tracking branch 'upstream/master' into path-info 2020-09-23 22:35:41 +00:00
Eelco Dolstra
d51ba43047 Move Callback into its own header
This gets rid of the inclusion of <future> in util.hh, cutting
compilation time by ~20s (CPU time).

Issue #4045.
2020-09-21 18:42:21 +02:00
John Ericson
45a2f1baab Rename drv output querying functions, like master
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap`
 - `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap
2020-08-20 18:14:12 +00:00
John Ericson
e913a2989f Squashed get CA derivations building 2020-08-07 19:51:55 +00:00
John Ericson
e7b0847f2d Make names more consistent 2020-08-05 14:44:39 +00:00
Eelco Dolstra
75f220a595
Merge pull request #3864 from obsidiansystems/more-topo-sort
Abstract out topo sorting logic
2020-08-05 16:07:29 +02:00
John Ericson
0e9e599b00 Merge remote-tracking branch 'upstream/master' into path-info 2020-08-01 00:06:24 +00:00
Matthew Bauer
05ac4db39a Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-30 12:38:24 -05:00
Eelco Dolstra
3f6e88a552 unsigned long long -> uint64_t 2020-07-30 13:34:04 +02:00
John Ericson
8065c6d160 Abstract out topo sorting logic 2020-07-27 20:45:34 +00:00
Matthew Bauer
8e0d0689be Only store hash of fixed derivation output
we don’t need a full storepath for a fixedoutput derivation. So just
putting the ingestion method + the hash is sufficient.
2020-07-08 19:11:39 -04:00
John Ericson
71e4c9c505 WIP: store separate hasValidPath bool 2020-06-25 17:53:31 +00:00
Matthew Bauer
66a62b3189 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-22 13:08:11 -04:00
Matthew Bauer
e8e1f5282f Replace error message in getDerivationCA 2020-06-19 15:19:09 -04:00
Matthew Bauer
7e11cf3399 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-19 14:40:49 -04:00
Matthew Bauer
5e631e3304 Add StorePathCAMap for querySubstitutablePathInfos
I’m not 100% sure this is wanted since it kind of makes everything
have to know about ca even if they don’t really want to. But it also
make things easier in dealing with looking up ca.
2020-06-17 15:03:05 -04:00
Matthew Bauer
7bd88cc1dc Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-17 13:32:29 -04:00
Eelco Dolstra
29542865ce Remove StorePath::clone() and related functions 2020-06-16 22:20:18 +02:00
Matthew Bauer
c214cda940 Correctly substitute from different storeDir
Originally, the test was only checking for different “real” storeDir.
That’s an easy case to handle, but the much harder one is if different
virtual store dirs are used. To do this, we need the SubstitutionGoal
to know about the ca, so it can recalculate the path to copy it over.
An important note here is that the store path passed to copyStorePath
needs to be one for srcStore - so that queryPathInfo works properly.

This also adds an error message when the store path from queryPathInfo
is different from the one we requested.
2020-06-13 00:07:42 -05:00
Matthew Bauer
79c169d1c6 Allow substituting from different storeDir
Substituters can substitute from one store dir to another with a
little bit of help. The store api just needs to have a CA so it can
recompute the store path based on the new store dir. We can only do
this for fixed output derivations with no references, though.
2020-06-12 09:49:09 -05:00
Eelco Dolstra
bbe97dff8b Make the Store API more type-safe
Most functions now take a StorePath argument rather than a Path (which
is just an alias for std::string). The StorePath constructor ensures
that the path is syntactically correct (i.e. it looks like
<store-dir>/<base32-hash>-<name>). Similarly, functions like
buildPaths() now take a StorePathWithOutputs, rather than abusing Path
by adding a '!<outputs>' suffix.

Note that the StorePath type is implemented in Rust. This involves
some hackery to allow Rust values to be used directly in C++, via a
helper type whose destructor calls the Rust type's drop()
function. The main issue is the dynamic nature of C++ move semantics:
after we have moved a Rust value, we should not call the drop function
on the original value. So when we move a value, we set the original
value to bitwise zero, and the destructor only calls drop() if the
value is not bitwise zero. This should be sufficient for most types.

Also lots of minor cleanups to the C++ API to make it more modern
(e.g. using std::optional and std::string_view in some places).
2019-12-10 22:06:05 +01:00
Eelco Dolstra
e5319a87ce
queryPathInfoUncached(): Return const ValidPathInfo 2019-10-29 13:53:04 +01:00