John Ericson
e6115ca004
Merge remote-tracking branch 'upstream/master' into ca-derivation-data-types
2020-07-27 17:41:43 +00:00
John Ericson
1d7d94ceea
Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors
2020-07-27 16:17:50 +00:00
John Ericson
43f2bd8dc5
Merge remote-tracking branch 'upstream/master' into hash-always-has-type
2020-07-27 16:13:57 +00:00
John Ericson
387699fe7f
Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca
2020-07-27 15:00:28 +00:00
John Ericson
78466bcb2f
Merge branch 'optional-derivation-output-storepath' into ca-derivation-data-types
2020-07-27 14:40:08 +00:00
John Ericson
d5bb67cfa4
Merge remote-tracking branch 'upstream/master' into optional-derivation-output-storepath
2020-07-27 14:29:32 +00:00
John Ericson
699fc89b39
Merge remote-tracking branch 'upstream/master' into hash-always-has-type
2020-07-27 14:22:45 +00:00
John Ericson
2c7557481b
queryDerivationOutputMap no longer assumes all outputs have a mapping
...
This assumption is broken by CA derivations. Making a PR now to do the
breaking daemon change as soon as possible (if it is already too late,
we can bump protocol intead).
2020-07-24 21:14:06 +00:00
John Ericson
58e55c0923
Merge branch 'misc-ca' of github.com:obsidiansystems/nix into derivation-primop-floating-output
2020-07-23 19:17:52 +00:00
John Ericson
a2af25c55a
Merge remote-tracking branch 'upstream/master' into misc-ca
2020-07-23 19:14:17 +00:00
John Ericson
c36916eca0
Merge branch 'misc-ca' of github.com:obsidiansystems/nix into derivation-primop-floating-output
2020-07-23 15:06:56 +00:00
John Ericson
6fcd9966bf
Merge remote-tracking branch 'upstream/master' into misc-ca
2020-07-23 15:02:29 +00:00
John Ericson
66a2067288
Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors
2020-07-23 14:56:35 +00:00
John Ericson
2274f63453
Get rid of basicDerivation::findOutput
...
It's a tiny function which is:
- hardly worth abstrating over, and also only used once.
- doesn't work once we get CA drvs
I rewrote the one callsite to be forwards compatable with CA
derivations, and also potentially more performant: instead of reading in
the derivation it can ust consult the SQLite DB in the common case.
2020-07-23 14:39:02 +00:00
John Ericson
9423f64ee2
Parse CA derivations using new output variants
...
We no longer need `ParsedDerivation` because everything libstore needs
to know about is in the `BasicDerivation` proper.
2020-07-22 23:59:25 +00:00
John Ericson
c56356bacc
Separate concerns in scanForReferences with TeeSink
...
This also will make it easier to use a `HashModuloSink` instead for CA
derivations.
2020-07-22 22:13:44 +00:00
Eelco Dolstra
ff314f186e
Merge pull request #3736 from obsidiansystems/allow-relative-paths-in-store-option
...
Allow relative paths in --store option
2020-07-21 19:43:07 +02:00
Eelco Dolstra
0951330680
Merge pull request #3714 from obsidiansystems/add-body-to-network-errors
...
Add response body to network errors
2020-07-21 18:09:42 +02:00
John Ericson
6cce32c8e8
Change logic for deciding what is a relative path for the local store
...
The was Eelco's prefered logic, and it looks good to me!
2020-07-21 15:39:47 +00:00
Carlo Nucera
922a845ffc
Update chunkSize to the suggested value
...
This was a suggested course of action in a review in one of our earlier
commits, https://github.com/NixOS/nix/pull/3801#discussion_r457557079
2020-07-21 10:24:19 -04:00
John Ericson
54e507a7aa
Merge remote-tracking branch 'upstream/master' into add-body-to-network-errors
2020-07-21 14:17:59 +00:00
John Ericson
5ce95b9529
Update src/libstore/build.cc
2020-07-21 09:47:40 -04:00
John Ericson
cdb3f39b64
Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors
2020-07-21 13:43:20 +00:00
John Ericson
5055c595bd
Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into ca-derivation-data-types
2020-07-21 01:20:53 +00:00
John Ericson
5a37413796
Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into from-dump-stream
2020-07-21 01:20:14 +00:00
John Ericson
bf0dca7790
Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into misc-ca
2020-07-21 01:18:33 +00:00
Carlo Nucera
9aae179f34
Correct bug, thoroughly document addToStoreSlow
2020-07-20 20:18:12 -04:00
Carlo Nucera
0ca9744694
Use heuristics to decide when to show the response
...
Due to https://github.com/NixOS/nix/issues/3841 we don't know how print
different messages for different verbosity levels.
2020-07-20 15:57:58 -04:00
Carlo Nucera
6357b1b0fb
Add another Unimplemented case
2020-07-20 14:17:25 -04:00
Carlo Nucera
007bf6d17a
Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca
2020-07-20 14:14:39 -04:00
Carlo Nucera
362ae93851
Add UnimplementedError to ease grepping for these
2020-07-20 14:13:37 -04:00
John Ericson
bf61871271
parser.hh -> split.hh
2020-07-20 17:42:34 +00:00
John Ericson
eab945f591
Merge remote-tracking branch 'upstream/master' into from-dump-stream
2020-07-20 17:29:23 +00:00
Eelco Dolstra
1c5f8bbfb5
Merge pull request #3822 from obsidiansystems/dump-thrice-fixme
...
Optimize `addToStoreSlow` and remove `TeeParseSink`
2020-07-20 18:55:05 +02:00
John Ericson
ac2fc7ba1f
Apply suggestions from code review
...
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2020-07-20 11:29:46 -04:00
John Ericson
6756cecfcf
Add DerivationType::CAFloating
2020-07-17 19:55:41 +00:00
Carlo Nucera
fefd6c9e5f
Merge branch 'master' of github.com:NixOS/nix into allow-relative-paths-in-store-option
2020-07-17 14:34:20 -04:00
Carlo Nucera
bbc633c98c
Revert "Don't anticipate CA but not fixed outputs for now"
...
This reverts commit 3a9e4c3262
2020-07-17 13:10:32 -04:00
Carlo Nucera
205dcd140d
Revert "Don't anticipate multiple CA outputs for now"
...
This reverts commit 74b251b2f3
2020-07-17 12:43:46 -04:00
Carlo Nucera
1feb8981df
Revert "Don't anticipate hash algo without hash in derivation for now"
...
This reverts commit 3804e3df9b
2020-07-17 12:05:54 -04:00
Carlo Nucera
487c5751c6
Merge branch 'ca-drv' of github.com:Ericson2314/nix into misc-ca
2020-07-17 11:32:59 -04:00
Carlo Nucera
5cb840541b
Merge branch 'multi-output-hashDerivationModulo' of github.com:Ericson2314/nix into misc-ca
2020-07-17 10:28:33 -04:00
Carlo Nucera
745a03cef5
Merge branch 'optional-derivation-output-storepath' of github.com:obsidiansystems/nix into ca-derivation-data-types
2020-07-16 13:36:01 -04:00
Carlo Nucera
048e916f64
Merge branch 'master' of github.com:NixOS/nix into optional-derivation-output-storepath
2020-07-16 13:32:28 -04:00
John Ericson
cc0d77f8c9
Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors
2020-07-16 17:28:52 +00:00
Carlo Nucera
e3a3406db8
Merge branch 'master' of github.com:NixOS/nix into add-body-to-network-errors
2020-07-16 12:04:10 -04:00
John Ericson
5ea817dace
Merge remote-tracking branch 'upstream/master' into hash-always-has-type
2020-07-16 14:58:53 +00:00
John Ericson
3dcca18c30
Fix bug in TeeSource
...
We use this to simplify `LocalStore::addToStoreFromDump`.
Also, hope I fixed build error with old clang (used in Darwin CI).
2020-07-16 13:39:27 +00:00
Carlo Nucera
e3b394b6e8
Small namespace fix
2020-07-16 09:36:02 -04:00
Eelco Dolstra
5517eee17e
Generations API cleanup
2020-07-16 15:18:21 +02:00
Eelco Dolstra
3f264916db
Merge remote-tracking branch 'origin/flakes'
2020-07-16 14:07:32 +02:00
John Ericson
68dfb8c6ae
Optimize addToStoreSlow and remove TeeParseSink
2020-07-16 05:09:41 +00:00
John Ericson
5602637d9e
Revert "LocalStore::addToStoreFromDump copy in chunks"
...
This reverts commit 592851fb67
2020-07-15 23:37:49 +00:00
John Ericson
bc109648c4
Get rid of LocalStore::addToStoreCommon
...
I got it to just become `LocalStore::addToStoreFromDump`, cleanly taking
a store and then doing nothing too fancy with it.
`LocalStore::addToStore(...Path...)` is now just a simple wrapper with a
bare-bones sinkToSource of the right dump command.
2020-07-15 23:14:30 +00:00
Carlo Nucera
455bdee205
Merge branch 'master' of github.com:NixOS/nix into derivation-header-include-order
2020-07-15 17:58:30 -04:00
Carlo Nucera
d090562348
Merge branch 'master' of github.com:NixOS/nix into hash-always-has-type
2020-07-15 17:21:01 -04:00
John Ericson
aff69e229c
Merge remote-tracking branch 'obsidian/fix-add-to-star-narhash' into from-dump-stream
2020-07-15 21:05:29 +00:00
Carlo Nucera
2d2a10e79a
Merge branch 'master' of github.com:NixOS/nix into add-body-to-network-errors
2020-07-15 13:06:08 -04:00
Eelco Dolstra
cff2157185
Revert "LocalStore::addToStore(srcPath): Handle the flat case"
...
This reverts commit a2c27022e9
2020-07-15 12:49:03 +02:00
John Ericson
650c2c6558
Rename variable nar -> dump according to TODO
2020-07-14 21:28:50 +00:00
John Ericson
8173e7bfef
Fix localhost::addToStore(...Path...)
...
We were calculating the nar hash wrong when the file ingestion method
was flat. I don't think there's anything we can do in that case but dump
the file again, so that's what I do.
As an optomization, we again could reuse the original dump for just the
recursive and non-sha256 case, but I rather do that after this fix, and
after my other PRs which deduplicate this code.
2020-07-14 21:12:11 +00:00
John Ericson
592851fb67
LocalStore::addToStoreFromDump copy in chunks
...
Rather than copying byte-by-byte, we let the coroutine know how much
data we would like it to send back to us.
2020-07-14 13:56:36 +00:00
John Ericson
9de96ef7d4
Dedup LocalStore::addToStore*
...
The downsides is that the coroutine has byte-by-byte loop transfer. Will
fix that next.
2020-07-14 13:56:36 +00:00
John Ericson
c86fc3a965
Crudely make addToStoreFromDump take Source not string
...
I just as little beyond the type as possible, so the implementation
changes this enables can be reviewed separately.
2020-07-14 13:56:36 +00:00
John Ericson
9ec10046e0
Narrow scope of temporary value
2020-07-14 13:54:29 +00:00
Eelco Dolstra
832e111494
Merge remote-tracking branch 'origin/master' into flakes
2020-07-14 13:56:18 +02:00
Eelco Dolstra
926c3a6664
Doh
2020-07-14 11:55:54 +02:00
Eelco Dolstra
43b8e96d30
Fix 'nix verify --all' on a binary cache (cached case)
2020-07-13 20:17:00 +02:00
Eelco Dolstra
7c2fef0a81
Make 'nix copy' to s3:// binary caches run in constant memory
2020-07-13 20:07:19 +02:00
Eelco Dolstra
493961b689
Remove istringstream_nocopy
2020-07-13 18:31:19 +02:00
Eelco Dolstra
545bb2ed03
Remove 'accessor' from addToStore()
...
This is only used by hydra-queue-runner and it's better to implement
it there.
2020-07-13 18:31:19 +02:00
Eelco Dolstra
0a9da00a10
NarAccessor: Run in constant memory
2020-07-13 17:30:42 +02:00
Eelco Dolstra
fc84c358d9
Make 'nix copy' to file:// binary caches run in constant memory
2020-07-13 16:28:45 +02:00
Eelco Dolstra
400f1a9b59
Store::pathInfoToJSON(): Use consistent format for downloadHash
2020-07-13 16:26:09 +02:00
Eelco Dolstra
c0dd05131e
toStorePath(): Return a StorePath and the suffix
2020-07-13 16:25:48 +02:00
Eelco Dolstra
1d01ae816b
Fix 'nix verify --all' on a binary cache and add a test
2020-07-13 14:35:01 +02:00
John Ericson
c466cb2091
Merge branch 'hash-always-has-type' into better-ca-parse-errors
2020-07-13 03:02:09 +00:00
John Ericson
4415765385
Merge remote-tracking branch 'upstream/master' into hash-always-has-type
2020-07-13 03:01:44 +00:00
John Ericson
230c9b4329
Change types to prepare the way for CA derivations
...
We've added the variant to `DerivationOutput` to support them, but made
`DerivationOutput::path` partial to avoid actually implementing them.
With this chage, we can all collaborate on "just" removing
`DerivationOutput::path` calls to implement CA derivations.
2020-07-12 23:53:33 +00:00
John Ericson
fedfc913ad
Use more std::visit to prepare for new variant
...
N.B. not using `std::visit` for fetchurl because there is no attempt to
handle all the cases (e.g. no `else`) and lambda complicates early
return.
2020-07-12 23:51:40 +00:00
John Ericson
ffc18583b1
Move C++17 "pattern matching" boilerplat to utils.hh
2020-07-12 22:15:14 +00:00
John Ericson
503b425690
DerivationOutputExtensional -> DerivationOutputInputAddressed
...
Thanks @regnat for the great name.
2020-07-12 15:56:20 +00:00
John Ericson
a8d4707107
Undo erroneous indentation change
2020-07-12 15:54:12 +00:00
John Ericson
18152406ce
String .drv suffix to create derivation name
2020-07-12 15:40:14 +00:00
John Ericson
5d0b75e5b6
std::string_view for new derivation name parameters
2020-07-12 15:02:36 +00:00
John Ericson
abea26a968
BasicDerivation::findOutput cannot return reference anymore
2020-07-12 01:57:06 +00:00
John Ericson
442d43178b
Merge remote-tracking branch 'upstream/master' into optional-derivation-output-storepath
2020-07-11 02:27:45 +00:00
Matthew Bauer
acb74d4d94
Merge remote-tracking branch 'origin/master' into substitute-other-storedir
2020-07-10 18:13:45 -04:00
Eelco Dolstra
8efa23bb99
Avoid a redundant hash
2020-07-10 15:56:24 +02:00
Eelco Dolstra
5dff49f661
Factor out commonality between nix-prefetch-url and nix-store --add-fixed
2020-07-10 13:21:37 +02:00
John Ericson
323d51a7a0
Merge branch 'master' of github.com:NixOS/nix into better-ca-parse-errors
2020-07-09 19:07:06 +00:00
Matthew Bauer
a7884970c5
Fix DerivationOutputExtensional name
2020-07-09 11:37:18 -04:00
Eelco Dolstra
a2c27022e9
LocalStore::addToStore(srcPath): Handle the flat case
...
This helps nix-prefetch-url when using a local store.
2020-07-09 15:54:32 +02:00
Matthew Bauer
06a4e15478
Fix build.cc on linux
2020-07-08 19:27:51 -04:00
Matthew Bauer
8e0d0689be
Only store hash of fixed derivation output
...
we don’t need a full storepath for a fixedoutput derivation. So just
putting the ingestion method + the hash is sufficient.
2020-07-08 19:11:39 -04:00
Eelco Dolstra
b981e5aacf
Cleanup
2020-07-08 22:07:21 +02:00
Eelco Dolstra
34f25124ba
Make LocalStore::addToStore(srcPath) run in constant memory
...
This reduces memory consumption of
nix-instantiate \
-E 'with import <nixpkgs> {}; runCommand "foo" { src = ./blender; } "echo foo"' \
--option nar-buffer-size 10000
(where ./blender is a 1.1 GiB tree) from 1716 to 36 MiB, while still
ensuring that we don't do any write I/O for small source paths (up to
'nar-buffer-size' bytes). The downside is that large paths are now
always written to a temporary location in the store, even if they
produce an already valid store path. Thus, adding large paths might be
slower and run out of disk space. ¯\_(ツ)_/¯ Of course, you can always
restore the old behaviour by setting 'nar-buffer-size' to a very high
value.
2020-07-08 22:07:21 +02:00
Matthew Bauer
af95a7c16b
Add name to BasicDerivation
...
We always have a name for BasicDerivation, since we have a derivation
store path that has a name.
2020-07-08 15:38:01 -04:00
Eelco Dolstra
9223603908
Merge remote-tracking branch 'origin/master' into flakes
2020-07-08 15:55:19 +02:00
Eelco Dolstra
16ec7785ca
Fix 'got unknown message type 1 from Nix daemon'
...
Example:
$ nix-build -E 'with import <nixpkgs> {}; runCommand "foo" { x = runCommand "bar" {} "exit 1"; } "echo foo; exit 1"'
warning: unknown setting 'auto-allocate-uids'
these 2 derivations will be built:
/nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv
/nix/store/k4fsvrjl7cp2xpz7927iv7g0dqj1zyhs-foo.drv
warning: unknown setting 'auto-allocate-uids'
building '/nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv'...
error: --- Error ----------------------------------------------------------------------------------------------------------------------------------------------------------------- nix-daemon
builder for '/nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv' failed with exit code 1
error: --- Error ------------------------------------------------------------------------------------------------------------------------------------------------------------------ nix-build
got unknown message type 1 from Nix daemon
2020-07-08 15:53:14 +02:00
Eelco Dolstra
1ab9da9154
Merge remote-tracking branch 'origin/master' into flakes
2020-07-07 14:38:57 +02:00
Eelco Dolstra
7c9ece5dca
exportReferencesGraph: Fix support for non-top-level store paths
...
Fixes #3471 .
2020-07-07 14:25:43 +02:00
Carlo Nucera
0f05a36e20
Merge branch 'master' of github.com:NixOS/nix into better-ca-parse-errors
2020-07-06 16:57:09 -04:00
Eelco Dolstra
54712aaf8a
Merge remote-tracking branch 'origin/master' into flakes
2020-07-06 16:40:10 +02:00
John Ericson
a38ab99d57
Merge remote-tracking branch 'upstream/master' into derivation-header-include-order
2020-07-05 21:49:01 +00:00
John Ericson
465daa9396
Merge remote-tracking branch 'upstream/master' into add-body-to-network-errors
2020-07-03 17:08:39 +00:00
John Ericson
3134db1a83
Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors
2020-07-03 14:12:38 +00:00
John Ericson
dbffd309fe
Merge branch 'master' of github.com:NixOS/nix into hash-always-has-type
2020-07-03 14:11:38 +00:00
Ben Burdette
b29a4ea1dc
Merge branch 'master' into add-trace
2020-07-03 07:57:36 -06:00
regnat
223fbe644a
Shorten the path to the test root
...
Fix a socket length failure on the OSX builders
2020-07-03 09:20:01 +02:00
John Ericson
13796be78d
Have splitPrefix and splitPrefixTo parser helpers
2020-07-02 23:18:22 +00:00
John Ericson
2f93d9f2ba
Merge branch 'hash-always-has-type' into HEAD
2020-07-02 21:47:51 +00:00
John Ericson
1be279af26
Fix Narinfo corruption detection bug
...
The aim of this check was just to ensure each key occurs once.
2020-07-02 21:46:10 +00:00
Carlo Nucera
1fc835aa22
Tighten parsing for drv files and pathinfo
2020-07-02 11:57:21 -04:00
Matthew Bauer
fc2ab42e86
Merge remote-tracking branch 'origin/master' into substitute-other-storedir
2020-07-02 11:14:04 -04:00
Matthew Bauer
d2e8b9ff0e
Store subPath in SubstitutionGoal
2020-07-02 11:12:05 -04:00
Ben Burdette
bf2788e4c1
move showTrace to new loggerSettings
2020-07-02 09:04:31 -06:00
Matthew Bauer
1f9cb06db2
Try next when no ca exists and have different store dirs
2020-07-02 10:59:24 -04:00
Carlo Nucera
263ccdd489
Rename two hash constructors to proper functions
2020-07-01 18:34:18 -04:00
Eelco Dolstra
38ccf2e241
Cleanup
2020-07-01 15:31:34 +02:00
Eelco Dolstra
86a4aba6c4
Merge branch 'remote-query-outputs' of https://github.com/tweag/nix
2020-07-01 15:10:29 +02:00
Ben Burdette
9159dfe3d8
comments and cleanup
2020-06-30 16:31:55 -06:00
Matthew Bauer
09833126f3
Merge remote-tracking branch 'origin/master' into substitute-other-storedir
2020-06-30 16:35:08 -04:00
Ben Burdette
ddb81ca126
Merge branch 'master' into add-trace
2020-06-30 12:21:45 -06:00
Carlo Nucera
77b51f4598
Factor the prefix splitting in content-address
2020-06-30 11:57:46 -04:00
John Ericson
a83566e5bc
Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors
2020-06-29 18:40:34 +00:00
John Ericson
987a4a0be9
Merge remote-tracking branch 'upstream/master' into hash-always-has-type
2020-06-29 18:03:21 +00:00
Ben Burdette
8f81fae116
showTrace flag in loggers
2020-06-29 10:20:51 -06:00
Eelco Dolstra
58bc3b6578
Merge pull request #3729 from obsidiansystems/simpler-hased-mirror
...
hashed-mirrors: Use parsed derivation output rather than reconstructing it
2020-06-29 14:04:12 +02:00
Eelco Dolstra
64232f3ea6
Merge pull request #3749 from rodarima/master
...
Fall back to copyPath if link fails with EPERM
2020-06-29 13:31:24 +02:00
Carlo Nucera
278a679782
Merge branch 'master' of github.com:NixOS/nix into allow-relative-paths-in-store-option
2020-06-26 10:40:13 -04:00
Eelco Dolstra
adf2fbbdc2
Merge remote-tracking branch 'origin/master' into flakes
2020-06-26 08:46:46 +02:00
John Ericson
d92d4f85a5
Move ValidPathInfo to its own header
2020-06-25 17:48:49 +00:00
Eelco Dolstra
de2641ae99
Fix empty std::optional dereference in writeDerivation()
...
https://hydra.nixos.org/build/123017579
2020-06-25 15:50:30 +02:00
Rodrigo
3a642187c3
Fall back to copyPath if link fails with EPERM
...
BeeGFS doesn't allow hard-links and returns EPERM, so we fall back
to copyPath. See https://github.com/NixOS/nix/issues/3748
2020-06-25 12:03:26 +02:00
regnat
d38f860c3e
Add a way to get all the outputs of a derivation with their label
...
Generalize `queryDerivationOutputNames` and `queryDerivationOutputs` by
adding a `queryDerivationOutputMap` that returns the map
`outputName=>outputPath`
(not that this is not equivalent to merging the results of
`queryDerivationOutputs` and `queryDerivationOutputNames` as sets don't
preserve the order, so we would end up with an incorrect mapping).
squash! Add a way to get all the outputs of a derivation with their label
Rename StorePathMap to OutputPathMap
2020-06-24 20:38:40 +02:00
John Ericson
d9fb9e9f26
Merge remote-tracking branch 'upstream/master' into ca-drv
2020-06-24 15:41:29 +00:00
John Ericson
98e5d1af03
Merge remote-tracking branch 'upstream/master' into hash-always-has-type
2020-06-23 17:03:37 +00:00
Carlo Nucera
e197bc6229
Enable the --store option to take relative paths
...
In nix commands which accept --store options, we can now specify a
relative path, which will be canonicalized.
2020-06-23 11:13:18 -04:00
John Ericson
55d4bd6e0e
Improve content address parsing
...
- Ensure hash is in form <algo>-<prefix> and not SRI.
- Better errors if something goes wrong
- string_view for no coppying
2020-06-22 18:08:27 +00:00
Matthew Bauer
66a62b3189
Merge remote-tracking branch 'origin/master' into substitute-other-storedir
2020-06-22 13:08:11 -04:00
John Ericson
f4a5913125
hashed-mirrors: Use parsed derivation output rather than reconstructing it
...
Now the derivation outputs are parsed up front, we can avoid a reparse
by doing it. Also, this just feels a bit better as the `output*` env
vars are more of a `libnixexpr` interface than `libnixstore` interface:
ultimately, it's the derivation outputs that decide whether the
derivation is fixed-output.
Yes, hashed mirrors might go away with #3689 , but this bit of code would
be moved rather than deleted, so it's worth doing a cleanup anyways I
think.
2020-06-22 15:17:20 +00:00
John Ericson
93bbe6e8ab
Merge remote-tracking branch 'upstream/master' into ca-drv
2020-06-21 21:11:58 +00:00
John Ericson
3804e3df9b
Don't anticipate hash algo without hash in derivation for now
...
When we merge with master, the new lack of string types make this case
impossible (after parsing). Later, when we actually implemenent
CA-derivations, we'll change the types to allow that.
2020-06-21 21:05:37 +00:00
John Ericson
8313f0e939
Merge remote-tracking branch 'upstream/master' into derivation-header-include-order
2020-06-21 20:39:10 +00:00
John Ericson
bf9f040112
Tweak declaration
...
I think this is clearer
2020-06-21 16:51:39 +00:00
John Ericson
fdeabf7160
Merge remote-tracking branch 'upstream/master' into multi-output-hashDerivationModulo
2020-06-21 16:43:17 +00:00
Matthew Bauer
f2a6cee334
Update worker protocol to support sending storepath maps
...
We need to also send the ca to daemon in addition to the path.
2020-06-19 18:06:19 -04:00
John Ericson
507aa48739
WIP: Make Hash always store a valid hash type
2020-06-19 21:48:57 +00:00
Matthew Bauer
e8e1f5282f
Replace error message in getDerivationCA
2020-06-19 15:19:09 -04:00
Matthew Bauer
0c9c1b8826
Return map of StorePaths in copyPaths
...
This allows the caller to know what values were actually added to the
store.
2020-06-19 14:48:57 -04:00
Matthew Bauer
7e11cf3399
Merge remote-tracking branch 'origin/master' into substitute-other-storedir
2020-06-19 14:40:49 -04:00
Matthew Bauer
2796b01e86
Merge remote-tracking branch 'origin/master' into substitute-other-storedir
2020-06-19 14:22:06 -04:00
John Ericson
68294746ae
Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown
2020-06-19 17:53:34 +00:00
John Ericson
911fc88bcb
More designated initializers
2020-06-19 17:42:56 +00:00
John Ericson
2f0e395c99
Merge remote-tracking branch 'me/no-stringly-typed-derivation-output' into validPathInfo-ca-proper-datatype
2020-06-19 15:26:59 +00:00
John Ericson
fb39a5e00c
Remove unneeded constructor for DerivationOutputHash
2020-06-19 15:11:11 +00:00
John Ericson
01dc8b0bab
Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output
2020-06-19 14:59:05 +00:00
John Ericson
145d88cb2a
Use designated initializers for DerivationOutputHash
2020-06-19 14:58:30 +00:00
John Ericson
237d88c97e
FileSystemHash -> DerivationOutputHash
2020-06-19 14:47:10 +00:00
John Ericson
3fc58a9638
Remove some Base:: that crept in
2020-06-19 00:24:47 +00:00
John Ericson
3f8dcfe3fd
Merge branch 'validPathInfo-temp' into validPathInfo-ca-proper-datatype
2020-06-18 23:01:58 +00:00
John Ericson
669c3992e8
Merge branch 'no-hash-type-unknown' into validPathInfo-temp
2020-06-18 22:33:07 +00:00
John Ericson
15abb2aa2b
Revert the enum struct change
...
Not a regular git revert as there have been many merges and things.
2020-06-18 22:11:26 +00:00
John Ericson
bbbf3602a3
Merge branch 'enum-class' into no-hash-type-unknown
2020-06-18 22:11:19 +00:00
John Ericson
40526fbea5
Merge remote-tracking branch 'upstream/master' into enum-class
2020-06-18 21:38:15 +00:00
John Ericson
1b23fe4afb
Fix bugs
...
- Bad dynamic cast target ...classic
- std::shared_ptr need explicit deref
2020-06-18 19:03:10 +00:00
John Ericson
fb432b0bd2
Merge remote-tracking branch 'obsidian/fix-url-format' into add-body-to-network-errors
2020-06-18 18:08:02 +00:00
John Ericson
639e20dc3e
Prevent '%' in URL from causing crashes
...
We have a larger problem that passsing computed strings to the first
variable argument of many exception constructors is unsafe because that
first variable argument is interpreted not as a plain string, but format
string, and if it contains '%' boost::format will abort, since there are
no arguments to the format string.
In this particular instance '%' was used as part of an escape code in a
URL, which, when the download failed, caused Nix to abort displaying the
`FileTransferError`.
2020-06-18 17:54:16 +00:00
John Ericson
74b219ef6e
Adjust FileTransferError message to use opt response
2020-06-18 14:48:45 +00:00
regnat
4fef2ba7e4
Rename content-addressed-paths into ca-derivations
...
See <https://github.com/NixOS/nix/pull/3710#issuecomment-645480333 >
2020-06-18 09:25:55 +02:00
Carlo Nucera
004570a377
Add HTTP responses to FileTransferErrors
2020-06-17 22:34:31 +00:00
Carlo Nucera
a835c740ca
Replace TransferItem::status with a local variable
...
Everywhere seems to use `getHTTPStatus` now.
2020-06-17 22:16:24 +00:00
Carlo Nucera
fb05a6adcf
Eliminate old TeeSink abstraction
...
This was introduced in fa125b9b281cf4801108
2020-06-17 22:07:22 +00:00
Eelco Dolstra
2b8f33bf5f
Merge pull request #3713 from matthewbauer/cleanup-warnings
...
Cleanup class StorePath warning
2020-06-17 21:19:30 +02:00
Matthew Bauer
5e631e3304
Add StorePathCAMap for querySubstitutablePathInfos
...
I’m not 100% sure this is wanted since it kind of makes everything
have to know about ca even if they don’t really want to. But it also
make things easier in dealing with looking up ca.
2020-06-17 15:03:05 -04:00
Matthew Bauer
be50de1142
Make sure references are empty for store path replacing
...
also copy info2 instead of casting
2020-06-17 14:15:08 -04:00
Matthew Bauer
8974755d19
Add assert for replaced storePath
2020-06-17 14:04:46 -04:00
Eelco Dolstra
3078404e35
Merge pull request #3712 from obsidiansystems/make-http-successful-states-coherent
...
Make successful states coherent
2020-06-17 19:37:46 +02:00
Matthew Bauer
7bd88cc1dc
Merge remote-tracking branch 'origin/master' into substitute-other-storedir
2020-06-17 13:32:29 -04:00
Matthew Bauer
22d7d36703
Remove unused narInfoFile in binary-cache-store
2020-06-17 13:27:19 -04:00
Matthew Bauer
f767bedfac
Replace struct StorePath with class StorePath
...
also a similar case with struct Goal
2020-06-17 13:26:37 -04:00
Carlo Nucera
4930cb48a2
Include review comments
2020-06-17 12:58:59 -04:00
Matthew Bauer
6438ba1e99
Update strings from review comment
2020-06-17 12:31:01 -04:00
Eelco Dolstra
2f51cd8dc9
Merge pull request #3710 from tweag/reserve_ca_derivations
...
Reserve the `__contentAddressed` derivation parameter
2020-06-17 18:28:26 +02:00
Eelco Dolstra
4d5169bdd5
Merge pull request #3707 from p01arst0rm/outdated-function-fix
...
replaced uncaught_exception with uncaught_exceptions
2020-06-17 18:26:01 +02:00
regnat
480b54e1c6
fixup! Reserve the __contentAddressed derivation parameter
2020-06-17 17:37:04 +02:00
Carlo Nucera
079c6e87de
Make successful states coherent
...
The successful states used in these two places in the code were slightly
different. Should they be the same list?
2020-06-17 11:16:16 -04:00
regnat
56d75bf4fc
Reserve the __contentAddressed derivation parameter
...
Not implementing anything here, just throwing an error if a derivation
sets `__contentAddressed = true` without
`--experimental-features content-addressed-paths`
(and also with it as there's nothing implemented yet)
2020-06-17 15:41:17 +02:00
Eelco Dolstra
fdff09e57c
Fix coverage build
2020-06-17 15:18:10 +02:00
Eelco Dolstra
ccbea8255c
Merge pull request #3657 from obsidiansystems/sligthly-improve-store-path-documentation
...
Clarify the description of StorePath inputs
2020-06-17 14:54:37 +02:00
Eelco Dolstra
1524752c17
Merge remote-tracking branch 'origin/master' into flakes
2020-06-17 10:26:52 +02:00
John Ericson
517f5980e2
Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output
2020-06-17 04:58:43 +00:00
John Ericson
18493fd9c4
Move some Store functions from derivations.cc to store-api.cc
...
This further continues with the dependency inverstion. Also I just went
ahead and exposed `parseDerivation`: it seems like the more proper
building block, and not a bad thing to expose if we are trying to be
less wedded to drv files on disk anywas.
2020-06-17 04:30:16 +00:00
John Ericson
7130f0a3a6
Don't need abstract struct Derivation in local-store
2020-06-17 04:05:38 +00:00
John Ericson
21ef342172
Merge remote-tracking branch 'upstream/master' into derivation-header-include-order
2020-06-17 03:52:01 +00:00
John Ericson
02928f76fd
Merge remote-tracking branch 'upstream/master' into multi-output-hashDerivationModulo
2020-06-17 03:51:02 +00:00
John Ericson
49308ef039
Merge remote-tracking branch 'upstream/master' into ca-drv
2020-06-17 03:39:39 +00:00
p01arst0rm
c9d06558b6
replaced uncaught_exception with uncaught_exceptions
2020-06-17 03:15:47 +01:00
Eelco Dolstra
29542865ce
Remove StorePath::clone() and related functions
2020-06-16 22:20:18 +02:00
