max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2024-11-11 00:36:20 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
11730 commits 7 branches 0 tags 64 MiB
Commit graph

349 commits

Author SHA1 Message Date
Eelco Dolstra
e6b7c7b79c Cleanup 2020-11-17 13:58:55 +01:00
Maximilian Bosch
3a63fc6cd5
Allow substituting paths when building remotely using ssh-ng:// 
Until now, it was not possible to substitute missing paths from e.g.
`https://cache.nixos.org` on a remote server when building on it using
the new `ssh-ng` protocol.

This is because every store implementation except legacy `ssh://`
ignores the substitution flag passed to `Store::queryValidPaths` while
the `legacy-ssh-store` substitutes the remote store using
`cmdQueryValidPaths` when the remote store is opened with `nix-store
--serve`.

This patch slightly modifies the daemon protocol to allow passing an
integer value suggesting whether to substitute missing paths during
`wopQueryValidPaths`. To implement this on the daemon-side, the
substitution logic from `nix-store --serve` has been moved into a
protected method named `Store::substitutePaths` which gets currently
called from `LocalStore::queryValidPaths` and `Store::queryValidPaths`
if `maybeSubstitute` is `true`.

Fixes #2770
 2020-11-05 20:12:37 +01:00
Eelco Dolstra
ff4dea63c9 Generalize extra-* settings 
This removes the extra-substituters and extra-sandbox-paths settings
and instead makes every array setting extensible by setting
"extra-<name> = <value>" in the configuration file or passing
"--<name> <value>" on the command line.
 2020-10-29 18:17:39 +01:00
John Ericson
15fdb7cc6b Split out uds-remote-store.{cc.hh} 2020-10-11 17:37:05 +00:00
Eelco Dolstra
d51ba43047 Move Callback into its own header 
This gets rid of the inclusion of <future> in util.hh, cutting
compilation time by ~20s (CPU time).

Issue #4045.
 2020-09-21 18:42:21 +02:00
John Ericson
f60b380a7f Merge remote-tracking branch 'upstream/master' into remove-storetype-delegate-regStore 2020-09-16 22:35:24 +00:00
Théophane Hufschmitt
77a0e2c5be
Remove useless exception copy 
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
 2020-09-16 14:00:21 +02:00
regnat
7f103dcddd Properly filter the stores according to their declared uriSchemes 
When opening a store, only try the stores whose `uriSchemes()` include
the current one
 2020-09-16 13:53:09 +02:00
regnat
22afa8fb4d Separate store configs from the implems 
Rework the `Store` hierarchy so that there's now one hierarchy for the
store configs and one for the implementations (where each implementation
extends the corresponding config). So a class hierarchy like

```
StoreConfig-------->Store
    |                 |
    v                 v
SubStoreConfig----->SubStore
    |                 |
    v                 v
SubSubStoreConfig-->SubSubStore
```

(with virtual inheritance to prevent DDD).

The advantage of this architecture is that we can now introspect the configuration of a store without having to instantiate the store itself
 2020-09-16 13:53:08 +02:00
regnat
aa4eac3788 fixup! Separate the instantiation and initialisation of the stores 2020-09-16 13:53:08 +02:00
regnat
3b57181f8e Separate the instantiation and initialisation of the stores 
Add a new `init()` method to the `Store` class that is supposed to
handle all the effectful initialisation needed to set-up the store.
The constructor should remain side-effect free and just initialize the
c++ data structure.

The goal behind that is that we can create “dummy” instances of each
store to query static properties about it (the parameters it accepts for
example)
 2020-09-16 13:53:08 +02:00
regnat
fa32560169 Fix the registration of stores 2020-09-16 13:53:08 +02:00
regnat
7d5bdf8b56 Make the store plugins more introspectable 
Directly register the store classes rather than a function to build an
instance of them.
This gives the possibility to introspect static members of the class or
choose different ways of instantiating them.
 2020-09-16 13:53:08 +02:00
John Ericson
c08c9f08c7 Merge remote-tracking branch 'upstream/master' into remove-storetype-delegate-regStore 2020-09-15 14:08:35 +00:00
John Ericson
8017fe7487 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-28 19:59:14 +00:00
Eelco Dolstra
eb75282b8d
Merge pull request #3434 from Ericson2314/derivation-header-include-order 
Revise division of labor in deserialization of derivations
 2020-08-27 16:39:28 +02:00
John Ericson
45a2f1baab Rename drv output querying functions, like master 
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap`
 - `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap
 2020-08-20 18:14:12 +00:00
Eelco Dolstra
4d77513d97
Merge pull request #3859 from obsidiansystems/drv-outputs-map-allow-missing 
`queryDerivationOutputMap` no longer assumes all outputs have a mapping
 2020-08-20 16:49:23 +02:00
John Ericson
9a9d834dc7 Rename drv output querying functions 
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap`
 - `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap`
 2020-08-20 14:12:51 +00:00
John Ericson
d5af5763cf Merge branch 'master' of github.com:NixOS/nix into remove-storetype-delegate-regStore 2020-08-19 15:54:50 +00:00
John Ericson
950ddfdb82 Merge remote-tracking branch 'upstream/master' into derivation-header-include-order 2020-08-18 14:36:44 +00:00
John Ericson
3c8b5b6219 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-14 17:00:13 +00:00
Eelco Dolstra
d81f13f7cb
Merge pull request #3899 from obsidiansystems/make-narHash-not-optional 
Make narHash in ValidPathInfo not optional
 2020-08-14 17:00:18 +02:00
John Ericson
85aacbee64 Use TeeSink and TeeSouce in a few more places 2020-08-13 14:51:17 +00:00
John Ericson
e913a2989f Squashed get CA derivations building 2020-08-07 19:51:55 +00:00
John Ericson
e89b5bd0bf Minimize the usage of Hash::dummy 2020-08-06 18:31:48 +00:00
Carlo Nucera
8241e660ba Remove Hash::operator bool () 
Since the hash is not optional anymore
 2020-08-05 15:30:38 -04:00
Carlo Nucera
be6e1c6457 Merge branch 'master' of github.com:NixOS/nix into make-narHash-not-optional 2020-08-05 15:14:47 -04:00
Carlo Nucera
1d71028f4d Remove optionality in ValidPathInfo::narInfo 2020-08-05 14:42:48 -04:00
John Ericson
cf939055c8 Merge remote-tracking branch 'upstream/master' into drv-outputs-map-allow-missing 2020-08-05 16:43:30 +00:00
John Ericson
839f0fe095 Merge remote-tracking branch 'upstream/master' into misc-ca 2020-08-05 14:40:01 +00:00
John Ericson
1dfcbebc95 Organize and format code a bit 2020-08-04 22:28:10 +00:00
John Ericson
fbeb8695fb Merge remote-tracking branch 'upstream/master' into drv-outputs-map-allow-missing 2020-08-04 18:35:33 +00:00
John Ericson
d3452a5ed6 Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors 2020-08-04 16:13:20 +00:00
John Ericson
54281f3ac1 addToStore in terms of addToStoreFromDump is not local-store-specific 2020-08-03 04:13:45 +00:00
John Ericson
9357512d73 Merge remote-tracking branch 'upstream/master' into derivation-header-include-order 2020-08-01 19:38:35 +00:00
Carlo Nucera
b6d97fdbf4 Merge branch 'master' of github.com:NixOS/nix into drv-outputs-map-allow-missing 2020-07-31 13:12:51 -04:00
Carlo Nucera
eee6ef86cd Merge branch 'master' of github.com:NixOS/nix into better-ca-parse-errors 2020-07-30 18:27:25 -04:00
Matthew Bauer
05ac4db39a Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-30 12:38:24 -05:00
Eelco Dolstra
c159f48a39 Cleanup 2020-07-29 00:24:55 +02:00
John Ericson
699fc89b39 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-27 14:22:45 +00:00
John Ericson
2c7557481b queryDerivationOutputMap no longer assumes all outputs have a mapping 
This assumption is broken by CA derivations. Making a PR now to do the
breaking daemon change as soon as possible (if it is already too late,
we can bump protocol intead).
 2020-07-24 21:14:06 +00:00
John Ericson
66a2067288 Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors 2020-07-23 14:56:35 +00:00
Eelco Dolstra
ff314f186e
Merge pull request #3736 from obsidiansystems/allow-relative-paths-in-store-option 
Allow relative paths in --store option
 2020-07-21 19:43:07 +02:00
John Ericson
6cce32c8e8 Change logic for deciding what is a relative path for the local store 
The was Eelco's prefered logic, and it looks good to me!
 2020-07-21 15:39:47 +00:00
John Ericson
cdb3f39b64 Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors 2020-07-21 13:43:20 +00:00
Carlo Nucera
9aae179f34 Correct bug, thoroughly document addToStoreSlow 2020-07-20 20:18:12 -04:00
John Ericson
ac2fc7ba1f
Apply suggestions from code review 
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
 2020-07-20 11:29:46 -04:00
Carlo Nucera
0aa79dcc6f Remove StoreType abstraction and delegate regStore 
to each Store implementation. The generic regStore implementation will
only be for the ambiguous shorthands, like "" and "auto".

This also could get us close to simplifying the daemon command.
 2020-07-17 17:24:02 -04:00
Carlo Nucera
fefd6c9e5f Merge branch 'master' of github.com:NixOS/nix into allow-relative-paths-in-store-option 2020-07-17 14:34:20 -04:00
John Ericson
cc0d77f8c9 Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-07-16 17:28:52 +00:00
John Ericson
68dfb8c6ae Optimize addToStoreSlow and remove TeeParseSink 2020-07-16 05:09:41 +00:00
Carlo Nucera
455bdee205 Merge branch 'master' of github.com:NixOS/nix into derivation-header-include-order 2020-07-15 17:58:30 -04:00
Carlo Nucera
d090562348 Merge branch 'master' of github.com:NixOS/nix into hash-always-has-type 2020-07-15 17:21:01 -04:00
Eelco Dolstra
926c3a6664 Doh 2020-07-14 11:55:54 +02:00
Eelco Dolstra
43b8e96d30 Fix 'nix verify --all' on a binary cache (cached case) 2020-07-13 20:17:00 +02:00
Eelco Dolstra
400f1a9b59 Store::pathInfoToJSON(): Use consistent format for downloadHash 2020-07-13 16:26:09 +02:00
Eelco Dolstra
c0dd05131e toStorePath(): Return a StorePath and the suffix 2020-07-13 16:25:48 +02:00
Eelco Dolstra
1d01ae816b Fix 'nix verify --all' on a binary cache and add a test 2020-07-13 14:35:01 +02:00
John Ericson
c466cb2091 Merge branch 'hash-always-has-type' into better-ca-parse-errors 2020-07-13 03:02:09 +00:00
John Ericson
4415765385 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-13 03:01:44 +00:00
Matthew Bauer
acb74d4d94 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-10 18:13:45 -04:00
Eelco Dolstra
8efa23bb99 Avoid a redundant hash 2020-07-10 15:56:24 +02:00
Eelco Dolstra
5dff49f661 Factor out commonality between nix-prefetch-url and nix-store --add-fixed 2020-07-10 13:21:37 +02:00
John Ericson
a38ab99d57 Merge remote-tracking branch 'upstream/master' into derivation-header-include-order 2020-07-05 21:49:01 +00:00
John Ericson
3134db1a83 Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-07-03 14:12:38 +00:00
John Ericson
dbffd309fe Merge branch 'master' of github.com:NixOS/nix into hash-always-has-type 2020-07-03 14:11:38 +00:00
Matthew Bauer
fc2ab42e86 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-02 11:14:04 -04:00
Carlo Nucera
263ccdd489 Rename two hash constructors to proper functions 2020-07-01 18:34:18 -04:00
regnat
d38f860c3e Add a way to get all the outputs of a derivation with their label 
Generalize `queryDerivationOutputNames` and `queryDerivationOutputs` by
adding a `queryDerivationOutputMap` that returns the map
`outputName=>outputPath`

(not that this is not equivalent to merging the results of
`queryDerivationOutputs` and `queryDerivationOutputNames` as sets don't
preserve the order, so we would end up with an incorrect mapping).

squash! Add a way to get all the outputs of a derivation with their label

Rename StorePathMap to OutputPathMap
 2020-06-24 20:38:40 +02:00
John Ericson
98e5d1af03 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-06-23 17:03:37 +00:00
Carlo Nucera
e197bc6229 Enable the --store option to take relative paths 
In nix commands which accept --store options, we can now specify a
relative path, which will be canonicalized.
 2020-06-23 11:13:18 -04:00
Matthew Bauer
66a62b3189 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-22 13:08:11 -04:00
John Ericson
8313f0e939 Merge remote-tracking branch 'upstream/master' into derivation-header-include-order 2020-06-21 20:39:10 +00:00
John Ericson
507aa48739 WIP: Make Hash always store a valid hash type 2020-06-19 21:48:57 +00:00
Matthew Bauer
0c9c1b8826 Return map of StorePaths in copyPaths 
This allows the caller to know what values were actually added to the
store.
 2020-06-19 14:48:57 -04:00
Matthew Bauer
2796b01e86 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-19 14:22:06 -04:00
John Ericson
2f0e395c99 Merge remote-tracking branch 'me/no-stringly-typed-derivation-output' into validPathInfo-ca-proper-datatype 2020-06-19 15:26:59 +00:00
John Ericson
01dc8b0bab Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-19 14:59:05 +00:00
John Ericson
3f8dcfe3fd Merge branch 'validPathInfo-temp' into validPathInfo-ca-proper-datatype 2020-06-18 23:01:58 +00:00
John Ericson
669c3992e8 Merge branch 'no-hash-type-unknown' into validPathInfo-temp 2020-06-18 22:33:07 +00:00
John Ericson
15abb2aa2b Revert the enum struct change 
Not a regular git revert as there have been many merges and things.
 2020-06-18 22:11:26 +00:00
John Ericson
40526fbea5 Merge remote-tracking branch 'upstream/master' into enum-class 2020-06-18 21:38:15 +00:00
Matthew Bauer
be50de1142 Make sure references are empty for store path replacing 
also copy info2 instead of casting
 2020-06-17 14:15:08 -04:00
Matthew Bauer
8974755d19 Add assert for replaced storePath 2020-06-17 14:04:46 -04:00
Matthew Bauer
7bd88cc1dc Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-17 13:32:29 -04:00
Matthew Bauer
6438ba1e99 Update strings from review comment 2020-06-17 12:31:01 -04:00
Eelco Dolstra
ccbea8255c
Merge pull request #3657 from obsidiansystems/sligthly-improve-store-path-documentation 
Clarify the description of StorePath inputs
 2020-06-17 14:54:37 +02:00
John Ericson
517f5980e2 Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-17 04:58:43 +00:00
John Ericson
18493fd9c4 Move some Store functions from derivations.cc to store-api.cc 
This further continues with the dependency inverstion. Also I just went
ahead and exposed `parseDerivation`: it seems like the more proper
building block, and not a bad thing to expose if we are trying to be
less wedded to drv files on disk anywas.
 2020-06-17 04:30:16 +00:00
Eelco Dolstra
29542865ce Remove StorePath::clone() and related functions 2020-06-16 22:20:18 +02:00
Eelco Dolstra
759947bf72 StorePath: Rewrite in C++ 
On nix-env -qa -f '<nixpkgs>', this reduces maximum RSS by 20970 KiB
and runtime by 0.8%. This is mostly because we're not parsing the hash
part as a hash anymore (just validating that it consists of base-32
characters).

Also, replace storePathToHash() by StorePath::hashPart().
 2020-06-16 14:28:41 +02:00
Eelco Dolstra
e14e62fddd Remove trailing whitespace 2020-06-15 14:12:39 +02:00
Eelco Dolstra
7a77762961 Merge branch 'errors-phase-2' of https://github.com/bburdette/nix 2020-06-15 11:46:31 +02:00
Matthew Bauer
b2cb288cdd Add makeFixedOutputPathFromCA function 
This puts what we are already doing into a shared method. It just
needs a path name and a ca and produces a store path.
 2020-06-12 16:36:35 -05:00
John Ericson
f6f01416b7 Use std::string_view in a few more places 2020-06-12 21:32:30 +00:00
Matthew Bauer
88120442d2 Debug when storePath changes 
these rewrites should be transparent, but they are important to know
about when debugging
 2020-06-12 15:32:52 -05:00
Matthew Bauer
5b386b05f5 Recompute storePath based on isContentAddressed 2020-06-12 14:57:45 -05:00
Matthew Bauer
85d01e1f65 Don’t use makeStorePath 2020-06-12 14:53:30 -05:00
Matthew Bauer
9077b9e6b2 Separate dstStore path from srcStore path 2020-06-12 14:27:28 -05:00