max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2024-11-23 22:46:16 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
9243 commits 7 branches 0 tags 64 MiB
Commit graph

5759 commits

Author SHA1 Message Date
Eelco Dolstra
34f25124ba Make LocalStore::addToStore(srcPath) run in constant memory 
This reduces memory consumption of

  nix-instantiate \
    -E 'with import <nixpkgs> {}; runCommand "foo" { src = ./blender; } "echo foo"' \
    --option nar-buffer-size 10000

(where ./blender is a 1.1 GiB tree) from 1716 to 36 MiB, while still
ensuring that we don't do any write I/O for small source paths (up to
'nar-buffer-size' bytes). The downside is that large paths are now
always written to a temporary location in the store, even if they
produce an already valid store path. Thus, adding large paths might be
slower and run out of disk space. ¯\_(ツ)_/¯ Of course, you can always
restore the old behaviour by setting 'nar-buffer-size' to a very high
value.
 2020-07-08 22:07:21 +02:00
Matthew Bauer
af95a7c16b Add name to BasicDerivation 
We always have a name for BasicDerivation, since we have a derivation
store path that has a name.
 2020-07-08 15:38:01 -04:00
Eelco Dolstra
9223603908 Merge remote-tracking branch 'origin/master' into flakes 2020-07-08 15:55:19 +02:00
Eelco Dolstra
16ec7785ca Fix 'got unknown message type 1 from Nix daemon' 
Example:

  $ nix-build -E 'with import <nixpkgs> {}; runCommand "foo" { x = runCommand "bar" {} "exit 1"; } "echo foo; exit 1"'
  warning: unknown setting 'auto-allocate-uids'
  these 2 derivations will be built:
    /nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv
    /nix/store/k4fsvrjl7cp2xpz7927iv7g0dqj1zyhs-foo.drv
  warning: unknown setting 'auto-allocate-uids'
  building '/nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv'...
  error: --- Error ----------------------------------------------------------------------------------------------------------------------------------------------------------------- nix-daemon
  builder for '/nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv' failed with exit code 1
  error: --- Error ------------------------------------------------------------------------------------------------------------------------------------------------------------------ nix-build
  got unknown message type 1 from Nix daemon
 2020-07-08 15:53:14 +02:00
Eelco Dolstra
1ab9da9154 Merge remote-tracking branch 'origin/master' into flakes 2020-07-07 14:38:57 +02:00
Eelco Dolstra
4055cfee36 Fix coverage build 2020-07-07 14:37:47 +02:00
Eelco Dolstra
7c9ece5dca exportReferencesGraph: Fix support for non-top-level store paths 
Fixes #3471.
 2020-07-07 14:25:43 +02:00
Carlo Nucera
0f05a36e20 Merge branch 'master' of github.com:NixOS/nix into better-ca-parse-errors 2020-07-06 16:57:09 -04:00
Ben Burdette
efd6a8b230 bump 2020-07-06 11:54:53 -06:00
Ben Burdette
75bfcf8d15 revamp trace code and test 2020-07-06 10:51:48 -06:00
Eelco Dolstra
68f524d717 nix develop: Support derivations with multiple outputs 2020-07-06 18:34:58 +02:00
Eelco Dolstra
cd8eb8a7d1 nix develop: Fall back to "bash" if nixpkgs#bashInteractive is unavailable 2020-07-06 17:08:54 +02:00
Eelco Dolstra
54712aaf8a Merge remote-tracking branch 'origin/master' into flakes 2020-07-06 16:40:10 +02:00
John Ericson
a38ab99d57 Merge remote-tracking branch 'upstream/master' into derivation-header-include-order 2020-07-05 21:49:01 +00:00
Ben Burdette
a168224464 spacing 2020-07-04 18:30:49 -06:00
John Ericson
465daa9396 Merge remote-tracking branch 'upstream/master' into add-body-to-network-errors 2020-07-03 17:08:39 +00:00
John Ericson
3134db1a83 Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-07-03 14:12:38 +00:00
John Ericson
dbffd309fe Merge branch 'master' of github.com:NixOS/nix into hash-always-has-type 2020-07-03 14:11:38 +00:00
Ben Burdette
b29a4ea1dc Merge branch 'master' into add-trace 2020-07-03 07:57:36 -06:00
Eelco Dolstra
c3c7aedbb5 nix develop: Fix bad regex 
This was accepted by libstdc++ but not libc++.

https://hydra.nixos.org/build/123569154
 2020-07-03 14:58:58 +02:00
Eelco Dolstra
6f8fd3a3f2 Shut up a clang warning 2020-07-03 14:50:07 +02:00
regnat
223fbe644a Shorten the path to the test root 
Fix a socket length failure on the OSX builders
 2020-07-03 09:20:01 +02:00
John Ericson
13796be78d Have splitPrefix and splitPrefixTo parser helpers 2020-07-02 23:18:22 +00:00
John Ericson
a7cd7425d9 Move getParsedTypeAndSRI to a more suitable location 
Also mark it static
 2020-07-02 23:10:11 +00:00
John Ericson
2f93d9f2ba Merge branch 'hash-always-has-type' into HEAD 2020-07-02 21:47:51 +00:00
John Ericson
1be279af26 Fix Narinfo corruption detection bug 
The aim of this check was just to ensure each key occurs once.
 2020-07-02 21:46:10 +00:00
Eelco Dolstra
b5e4253697 Fix abort in 'nix develop' 2020-07-02 18:24:11 +02:00
Carlo Nucera
1fc835aa22 Tighten parsing for drv files and pathinfo 2020-07-02 11:57:21 -04:00
Ben Burdette
5818271c6e spacing 2020-07-02 09:41:54 -06:00
Carlo Nucera
b6b10b1d4c Write the implementation for parseNonSRIUnprefixed 2020-07-02 11:34:40 -04:00
Carlo Nucera
ea48e3a5b5 Abstract common parsing functionality 2020-07-02 11:29:33 -04:00
Carlo Nucera
36cbc74689 Inline and simplify in parseAnyPrefixed 2020-07-02 11:21:00 -04:00
Matthew Bauer
fc2ab42e86 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-07-02 11:14:04 -04:00
Matthew Bauer
d2e8b9ff0e Store subPath in SubstitutionGoal 2020-07-02 11:12:05 -04:00
Carlo Nucera
9462d8a50b Rename fromSRI to parseSRI for constistency 2020-07-02 11:11:18 -04:00
Carlo Nucera
f61bc45d19 Get rid of the std::pair 2020-07-02 11:09:04 -04:00
Ben Burdette
bf2788e4c1 move showTrace to new loggerSettings 2020-07-02 09:04:31 -06:00
Carlo Nucera
27c8029573 Inline newFunction 2020-07-02 11:01:10 -04:00
Matthew Bauer
1f9cb06db2 Try next when no ca exists and have different store dirs 2020-07-02 10:59:24 -04:00
Carlo Nucera
343d1569b1 Fix test suite 2020-07-02 10:48:47 -04:00
Ben Burdette
5ae498872a assert for invalid fileorigin 2020-07-02 07:14:40 -06:00
Carlo Nucera
263ccdd489 Rename two hash constructors to proper functions 2020-07-01 18:34:18 -04:00
Carlo Nucera
c8c4bcf90e Inline Hash::init() 2020-07-01 18:03:35 -04:00
Carlo Nucera
d63a5ded76 Remove unused import 2020-07-01 17:53:24 -04:00
Carlo Nucera
6faeec3b2a Keep the previous name, for diffing 2020-07-01 17:50:34 -04:00
Carlo Nucera
274a8136fb Correct FIXMEs in libfetchers 2020-07-01 17:47:15 -04:00
Carlo Nucera
c2e7f7a712 Fixed build, we still have test errors 2020-07-01 17:32:06 -04:00
Ben Burdette
8497891b99 spacing 2020-07-01 13:50:18 -06:00
Eelco Dolstra
6ff9aa8df7 Don't process an option if any of its arguments need completion 2020-07-01 20:31:39 +02:00
Eelco Dolstra
d746503e5c Add --inputs-from to use flake inputs as registry entries 
This allows you to refer to an input from another flake. For example,

  $ nix run --inputs-from /path/to/hydra nixpkgs#hello

runs 'hello' from the 'nixpkgs' inputs of the 'hydra' flake.

Fixes #3769.
 2020-07-01 20:25:13 +02:00
Ben Burdette
a295b2ea96 if no errLoc, no Loc. 2020-07-01 12:02:02 -06:00
Ben Burdette
3629b0585a don't include errpos for addErrorContext 2020-07-01 11:49:01 -06:00
Ben Burdette
2a39c083dc non-pos trace test 2020-07-01 10:37:31 -06:00
Eelco Dolstra
38ccf2e241 Cleanup 2020-07-01 15:31:34 +02:00
Eelco Dolstra
86a4aba6c4 Merge branch 'remote-query-outputs' of https://github.com/tweag/nix 2020-07-01 15:10:29 +02:00
Eelco Dolstra
7d554f295c Support building flakes from a shallow Git repo 
Fixes #3756.
 2020-07-01 14:57:59 +02:00
Ben Burdette
a7d5d26443 fix tests with the 'from string' change 2020-06-30 22:05:21 -06:00
Ben Burdette
9159dfe3d8 comments and cleanup 2020-06-30 16:31:55 -06:00
Ben Burdette
70bcb39d3f double addtrace for 'called from' 2020-06-30 15:44:19 -06:00
Matthew Bauer
09833126f3 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-30 16:35:08 -04:00
Ben Burdette
ddb81ca126 Merge branch 'master' into add-trace 2020-06-30 12:21:45 -06:00
Carlo Nucera
b798efb829 WIP initial design 2020-06-30 14:10:30 -04:00
Ben Burdette
a0705e0dd1 invalid pos check 2020-06-30 11:01:46 -06:00
Ben Burdette
e72a16a339 check for a null symbol 2020-06-30 11:00:51 -06:00
Carlo Nucera
a1f66d1d9e Factor the prefix splitting in hash 2020-06-30 12:49:00 -04:00
Carlo Nucera
77b51f4598 Factor the prefix splitting in content-address 2020-06-30 11:57:46 -04:00
Carlo Nucera
7ba0fae0dd Create the spitPrefix function in parser.hh 2020-06-30 11:57:09 -04:00
Ben Burdette
c484a67914 trace formatting 2020-06-29 15:46:21 -06:00
Eelco Dolstra
2b834d48aa NAR parser: Fix missing name field check 
Discovered by @Kloenk.
 2020-06-29 22:45:41 +02:00
John Ericson
a83566e5bc Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-06-29 18:40:34 +00:00
John Ericson
987a4a0be9 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-06-29 18:03:21 +00:00
Eelco Dolstra
26cf0c674f nix run: Use packages/legacyPackages as fallback if there is no app definition 
'nix run' will try to run $out/bin/<name>, where <name> is the
derivation name (excluding the version). This often works well:

  $ nix run nixpkgs#hello
  Hello, world!

  $ nix run nix -- --version
  nix (Nix) 2.4pre20200626_adf2fbb

  $ nix run patchelf -- --version
  patchelf 0.11.20200623.e61654b

  $ nix run nixpkgs#firefox -- --version
  Mozilla Firefox 77.0.1

  $ nix run nixpkgs#gimp -- --version
  GNU Image Manipulation Program version 2.10.14

though not always:

  $ nix run nixpkgs#git
  error: unable to execute '/nix/store/kp7wp760l4gryq9s36x481b2x4rfklcy-git-2.25.4/bin/git-minimal': No such file or directory
 2020-06-29 19:08:50 +02:00
Eelco Dolstra
50f13b06fb EvalCache: Store string contexts 2020-06-29 19:08:37 +02:00
Ben Burdette
8f81fae116 showTrace flag in loggers 2020-06-29 10:20:51 -06:00
Eelco Dolstra
b681408879 Factor out EvalCache::forceDerivation() 2020-06-29 16:39:41 +02:00
Eelco Dolstra
ca946860ce Fix bash completion 2020-06-29 14:37:22 +02:00
Eelco Dolstra
bc03c6f23d Move App 2020-06-29 14:14:23 +02:00
Eelco Dolstra
58bc3b6578
Merge pull request #3729 from obsidiansystems/simpler-hased-mirror 
hashed-mirrors: Use parsed derivation output rather than reconstructing it
 2020-06-29 14:04:12 +02:00
Eelco Dolstra
64232f3ea6
Merge pull request #3749 from rodarima/master 
Fall back to copyPath if link fails with EPERM
 2020-06-29 13:31:24 +02:00
Ben Burdette
ef24a0835d showtrace as function arg 2020-06-27 12:19:31 -06:00
Carlo Nucera
278a679782 Merge branch 'master' of github.com:NixOS/nix into allow-relative-paths-in-store-option 2020-06-26 10:40:13 -04:00
Eelco Dolstra
adf2fbbdc2 Merge remote-tracking branch 'origin/master' into flakes 2020-06-26 08:46:46 +02:00
John Ericson
d92d4f85a5 Move ValidPathInfo to its own header 2020-06-25 17:48:49 +00:00
Ben Burdette
bc9e87412c 'string' makes more sense in nix repl 2020-06-25 09:56:32 -06:00
Ben Burdette
9ab808c926 showTrace flag for ErrorInfo; showTrace test. 2020-06-25 09:23:12 -06:00
Eelco Dolstra
de2641ae99 Fix empty std::optional dereference in writeDerivation() 
https://hydra.nixos.org/build/123017579
 2020-06-25 15:50:30 +02:00
Rodrigo
3a642187c3 Fall back to copyPath if link fails with EPERM 
BeeGFS doesn't allow hard-links and returns EPERM, so we fall back
to copyPath. See https://github.com/NixOS/nix/issues/3748
 2020-06-25 12:03:26 +02:00
Ben Burdette
9c0e1fd4f1 add trace test; error formatting refinements 2020-06-24 18:31:28 -06:00
Ben Burdette
6359d71d6b re-enable --show-trace check 2020-06-24 18:28:20 -06:00
Ben Burdette
023912def3 convenience form of addTrace 2020-06-24 13:46:25 -06:00
Ben Burdette
93e9307329 repl indenting 2020-06-24 13:14:49 -06:00
Ben Burdette
b18ed02b76 repl indenting 2020-06-24 13:10:41 -06:00
regnat
d38f860c3e Add a way to get all the outputs of a derivation with their label 
Generalize `queryDerivationOutputNames` and `queryDerivationOutputs` by
adding a `queryDerivationOutputMap` that returns the map
`outputName=>outputPath`

(not that this is not equivalent to merging the results of
`queryDerivationOutputs` and `queryDerivationOutputNames` as sets don't
preserve the order, so we would end up with an incorrect mapping).

squash! Add a way to get all the outputs of a derivation with their label

Rename StorePathMap to OutputPathMap
 2020-06-24 20:38:40 +02:00
Ben Burdette
6fe660acf9 re-remove 2020-06-24 12:33:05 -06:00
Matthew Bauer
bcf8582091 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-24 12:52:09 -04:00
John Ericson
d9fb9e9f26 Merge remote-tracking branch 'upstream/master' into ca-drv 2020-06-24 15:41:29 +00:00
Ben Burdette
00fe653ea5 nixCode -> LinesOfCode 2020-06-24 08:33:53 -06:00
Ben Burdette
1d43a6e123 use plain errPos instead of nixCode; fix tests 2020-06-23 15:30:13 -06:00
Carlo Nucera
8d51d38e4c Fix test suite 2020-06-23 17:16:20 -04:00
John Ericson
ec3a857307 Fix and clean up hash parser 2020-06-23 18:19:40 +00:00
John Ericson
98e5d1af03 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-06-23 17:03:37 +00:00
Ben Burdette
d0e78fbb03 re-add Pos origin in tests 2020-06-23 10:51:58 -06:00
Ben Burdette
abe0552504 Merge remote-tracking branch 'upstream/master' into add-trace 2020-06-23 09:40:28 -06:00
Ben Burdette
13e87535ff traces to bottom 2020-06-23 09:36:58 -06:00
Carlo Nucera
e197bc6229 Enable the --store option to take relative paths 
In nix commands which accept --store options, we can now specify a
relative path, which will be canonicalized.
 2020-06-23 11:13:18 -04:00
Eelco Dolstra
09fc06daab nix flake init: Use git add --force 2020-06-23 16:25:32 +02:00
Eelco Dolstra
015e1c2131
Merge pull request #3724 from bburdette/hintfmt-percent 
Hintfmt percent test, and fix
 2020-06-23 12:26:00 +02:00
John Ericson
55d4bd6e0e Improve content address parsing 
- Ensure hash is in form <algo>-<prefix> and not SRI.

 - Better errors if something goes wrong

 - string_view for no coppying
 2020-06-22 18:08:27 +00:00
Ben Burdette
9d1cb0c5e6 with normaltxt, elide yellow color code instead of canceling it; use normaltxt on plain_string hintfmt 2020-06-22 11:32:20 -06:00
Matthew Bauer
66a62b3189 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-22 13:08:11 -04:00
Ben Burdette
28b079067f
Update src/libutil/fmt.hh 
Co-authored-by: John Ericson <git@JohnEricson.me>
 2020-06-22 10:00:37 -06:00
John Ericson
f4a5913125 hashed-mirrors: Use parsed derivation output rather than reconstructing it 
Now the derivation outputs are parsed up front, we can avoid a reparse
by doing it. Also, this just feels a bit better as the `output*` env
vars are more of a `libnixexpr` interface than `libnixstore` interface:
ultimately, it's the derivation outputs that decide whether the
derivation is fixed-output.

Yes, hashed mirrors might go away with #3689, but this bit of code would
be moved rather than deleted, so it's worth doing a cleanup anyways I
think.
 2020-06-22 15:17:20 +00:00
Eelco Dolstra
334e26bfc2 nix flake check: Don't build apps 
This was inconsistent since we're not building 'packages' or
'defaultPackage' either.

Closes #3726.
 2020-06-22 11:31:07 +02:00
John Ericson
93bbe6e8ab Merge remote-tracking branch 'upstream/master' into ca-drv 2020-06-21 21:11:58 +00:00
John Ericson
3804e3df9b Don't anticipate hash algo without hash in derivation for now 
When we merge with master, the new lack of string types make this case
impossible (after parsing). Later, when we actually implemenent
CA-derivations, we'll change the types to allow that.
 2020-06-21 21:05:37 +00:00
John Ericson
8313f0e939 Merge remote-tracking branch 'upstream/master' into derivation-header-include-order 2020-06-21 20:39:10 +00:00
John Ericson
bf9f040112 Tweak declaration 
I think this is clearer
 2020-06-21 16:51:39 +00:00
John Ericson
fdeabf7160 Merge remote-tracking branch 'upstream/master' into multi-output-hashDerivationModulo 2020-06-21 16:43:17 +00:00
Ben Burdette
be4f444175 tidying up 2020-06-19 16:58:12 -06:00
Ben Burdette
0309488a66 fmt -> hintfmt test 2020-06-19 16:46:49 -06:00
Matthew Bauer
f2a6cee334 Update worker protocol to support sending storepath maps 
We need to also send the ca to daemon in addition to the path.
 2020-06-19 18:06:19 -04:00
Ben Burdette
397dbe114e remove formathelper 2020-06-19 15:57:19 -06:00
John Ericson
507aa48739 WIP: Make Hash always store a valid hash type 2020-06-19 21:48:57 +00:00
Ben Burdette
b193aca4ae escape percents 2020-06-19 15:29:19 -06:00
Ben Burdette
db475f9e7e too few, too many args 2020-06-19 15:28:13 -06:00
Ben Burdette
cdddf24f25 add hintfmt test 2020-06-19 14:54:41 -06:00
Ben Burdette
54e8f550c9 addErrorTrace 2020-06-19 13:44:08 -06:00
Matthew Bauer
e8e1f5282f Replace error message in getDerivationCA 2020-06-19 15:19:09 -04:00
Matthew Bauer
0c9c1b8826 Return map of StorePaths in copyPaths 
This allows the caller to know what values were actually added to the
store.
 2020-06-19 14:48:57 -04:00
John Ericson
e288c0987a Merge remote-tracking branch 'upstream/master' into validPathInfo-ca-proper-datatype 2020-06-19 18:44:24 +00:00
Matthew Bauer
7e11cf3399 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-19 14:40:49 -04:00
Matthew Bauer
2796b01e86 Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-19 14:22:06 -04:00
John Ericson
68294746ae Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown 2020-06-19 17:53:34 +00:00
John Ericson
c98081d270 Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown 2020-06-19 17:50:05 +00:00
John Ericson
c1892a5316 tabs -> spaces 2020-06-19 17:49:57 +00:00
John Ericson
911fc88bcb More designated initializers 2020-06-19 17:42:56 +00:00
John Ericson
2f0e395c99 Merge remote-tracking branch 'me/no-stringly-typed-derivation-output' into validPathInfo-ca-proper-datatype 2020-06-19 15:26:59 +00:00
John Ericson
fb39a5e00c Remove unneeded constructor for DerivationOutputHash 2020-06-19 15:11:11 +00:00
John Ericson
b90cac3bad Remove uneeded = default for Hash 2020-06-19 15:00:38 +00:00
John Ericson
01dc8b0bab Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-19 14:59:05 +00:00
John Ericson
145d88cb2a Use designated initializers for DerivationOutputHash 2020-06-19 14:58:30 +00:00
John Ericson
237d88c97e FileSystemHash -> DerivationOutputHash 2020-06-19 14:47:10 +00:00
Eelco Dolstra
2886c92aef
Merge pull request #3669 from gilligan/add-compression-tests 
Add compression unit tests
 2020-06-19 13:59:04 +02:00
John Ericson
3fc58a9638 Remove some Base:: that crept in 2020-06-19 00:24:47 +00:00
John Ericson
3f8dcfe3fd Merge branch 'validPathInfo-temp' into validPathInfo-ca-proper-datatype 2020-06-18 23:01:58 +00:00
John Ericson
669c3992e8 Merge branch 'no-hash-type-unknown' into validPathInfo-temp 2020-06-18 22:33:07 +00:00
John Ericson
15abb2aa2b Revert the enum struct change 
Not a regular git revert as there have been many merges and things.
 2020-06-18 22:11:26 +00:00
John Ericson
bbbf3602a3 Merge branch 'enum-class' into no-hash-type-unknown 2020-06-18 22:11:19 +00:00
John Ericson
40526fbea5 Merge remote-tracking branch 'upstream/master' into enum-class 2020-06-18 21:38:15 +00:00
Ben Burdette
4d1a4f0217 addTrace 2020-06-18 15:25:26 -06:00
Ben Burdette
e6f93b94fc Merge branch 'master' into caveman-LOCs 2020-06-18 13:07:53 -06:00
John Ericson
1b23fe4afb Fix bugs 
- Bad dynamic cast target ...classic

 - std::shared_ptr need explicit deref
 2020-06-18 19:03:10 +00:00
John Ericson
fb432b0bd2 Merge remote-tracking branch 'obsidian/fix-url-format' into add-body-to-network-errors 2020-06-18 18:08:02 +00:00
John Ericson
639e20dc3e Prevent '%' in URL from causing crashes 
We have a larger problem that passsing computed strings to the first
variable argument of many exception constructors is unsafe because that
first variable argument is interpreted not as a plain string, but format
string, and if it contains '%' boost::format will abort, since there are
no arguments to the format string.

In this particular instance '%' was used as part of an escape code in a
URL, which, when the download failed, caused Nix to abort displaying the
`FileTransferError`.
 2020-06-18 17:54:16 +00:00
Eelco Dolstra
6c000eed80
Merge pull request #3709 from expipiplus1/master 
Mention number of derivations to be build/fetched in output
 2020-06-18 19:03:05 +02:00
John Ericson
74b219ef6e Adjust FileTransferError message to use opt response 2020-06-18 14:48:45 +00:00
Eelco Dolstra
5771c8bbf2 Don't provide 'getFlake' if the 'flakes' feature is not enabled 
(cherry picked from commit 0a1d3c1dd3)
 2020-06-18 14:03:00 +02:00
Eelco Dolstra
2a61bbf77f Some backports from the flakes branch 2020-06-18 14:03:00 +02:00
Eelco Dolstra
377345e26f Remove unneeded #include 2020-06-18 13:47:05 +02:00
Eelco Dolstra
7083d33efe Make constant primops lazy 
(cherry picked from commit aa0e2a2e70)
 2020-06-18 13:42:47 +02:00
Eelco Dolstra
3d492199bb github: Respect default branch 2020-06-18 13:25:08 +02:00
regnat
4fef2ba7e4 Rename content-addressed-paths into ca-derivations 
See <https://github.com/NixOS/nix/pull/3710#issuecomment-645480333>
 2020-06-18 09:25:55 +02:00
Carlo Nucera
004570a377 Add HTTP responses to FileTransferErrors 2020-06-17 22:34:31 +00:00
Carlo Nucera
a835c740ca Replace TransferItem::status with a local variable 
Everywhere seems to use `getHTTPStatus` now.
 2020-06-17 22:16:24 +00:00
Carlo Nucera
289b9b8dcf Create a new TeeSink abstraction 
This is a bit complex because we want to expose extra functionality the
wrapped class has. Perhaps there is some inheritancy trickery to do this
nicer, but I don't know it, and this is the first thing we tried after a
series of attempts that did build.

This design is kind of like that of Rust's Writer, Reader, or Iter
adapters, which impliment more traits based on what the inner type
implements.
 2020-06-17 22:09:32 +00:00
Carlo Nucera
fb05a6adcf Eliminate old TeeSink abstraction 
This was introduced in fa125b9b28, and
then "reverted" in 1cf4801108, except that
revert left the struct around doing nothing useful.

We're removing it all the way now because we want to make a new
`TeeSink` complementing the already-exiting `TeeSource`, that is
actually a completely different concept as far as the class hierarchy is
concerned.
 2020-06-17 22:07:22 +00:00
Eelco Dolstra
2b8f33bf5f
Merge pull request #3713 from matthewbauer/cleanup-warnings 
Cleanup class StorePath warning
 2020-06-17 21:19:30 +02:00
Matthew Bauer
5e631e3304 Add StorePathCAMap for querySubstitutablePathInfos 
I’m not 100% sure this is wanted since it kind of makes everything
have to know about ca even if they don’t really want to. But it also
make things easier in dealing with looking up ca.
 2020-06-17 15:03:05 -04:00
Matthew Bauer
be50de1142 Make sure references are empty for store path replacing 
also copy info2 instead of casting
 2020-06-17 14:15:08 -04:00
Matthew Bauer
8974755d19 Add assert for replaced storePath 2020-06-17 14:04:46 -04:00
Eelco Dolstra
3078404e35
Merge pull request #3712 from obsidiansystems/make-http-successful-states-coherent 
Make successful states coherent
 2020-06-17 19:37:46 +02:00
Matthew Bauer
7bd88cc1dc Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-17 13:32:29 -04:00
Matthew Bauer
22d7d36703 Remove unused narInfoFile in binary-cache-store 2020-06-17 13:27:19 -04:00
Matthew Bauer
f767bedfac Replace struct StorePath with class StorePath 
also a similar case with struct Goal
 2020-06-17 13:26:37 -04:00
Carlo Nucera
4930cb48a2 Include review comments 2020-06-17 12:58:59 -04:00
Matthew Bauer
6438ba1e99 Update strings from review comment 2020-06-17 12:31:01 -04:00
Eelco Dolstra
2f51cd8dc9
Merge pull request #3710 from tweag/reserve_ca_derivations 
Reserve the `__contentAddressed` derivation parameter
 2020-06-17 18:28:26 +02:00
Eelco Dolstra
4d5169bdd5
Merge pull request #3707 from p01arst0rm/outdated-function-fix 
replaced uncaught_exception with uncaught_exceptions
 2020-06-17 18:26:01 +02:00
Eelco Dolstra
2e4bd78211 nix eval: Add --apply flag for post-processing the result 2020-06-17 18:12:24 +02:00
regnat
480b54e1c6 fixup! Reserve the __contentAddressed derivation parameter 2020-06-17 17:37:04 +02:00
Carlo Nucera
079c6e87de Make successful states coherent 
The successful states used in these two places in the code were slightly
different. Should they be the same list?
 2020-06-17 11:16:16 -04:00
Eelco Dolstra
5332c439d0 InstallableFlake: Show all possible attribute names 
E.g.

  $ nix run nixpkgs#hello
  error: --- Error ---------- nix
  flake 'flake:nixpkgs' does not provide attribute 'apps.x86_64-linux.hello' or 'hello'

instead of

  $ nix run nixpkgs#hello
  error: --- Error ---------- nix
  flake 'flake:nixpkgs' does not provide attribute 'hello'
 2020-06-17 17:13:01 +02:00
John Ericson
6403508f5a Use ansicolor.hh in nix repl rather than duplicates 2020-06-17 15:13:00 +00:00
Eelco Dolstra
ad66fb0a37 getFlake -> builtins.getFlake 2020-06-17 17:05:08 +02:00
Eelco Dolstra
0a1d3c1dd3 Don't provide 'getFlake' if the 'flakes' feature is not enabled 2020-06-17 16:54:32 +02:00
regnat
56d75bf4fc Reserve the __contentAddressed derivation parameter 
Not implementing anything here, just throwing an error if a derivation
sets `__contentAddressed = true` without
`--experimental-features content-addressed-paths`
(and also with it as there's nothing implemented yet)
 2020-06-17 15:41:17 +02:00
Eelco Dolstra
fdff09e57c Fix coverage build 2020-06-17 15:18:10 +02:00
Eelco Dolstra
ccbea8255c
Merge pull request #3657 from obsidiansystems/sligthly-improve-store-path-documentation 
Clarify the description of StorePath inputs
 2020-06-17 14:54:37 +02:00
Joe Hermaszewski
da8aac6ce8 Mention number of derivations to be build/fetched in output 
Also correct grammar for the case of a single derivation.
 2020-06-17 20:27:27 +08:00
Eelco Dolstra
1524752c17 Merge remote-tracking branch 'origin/master' into flakes 2020-06-17 10:26:52 +02:00
John Ericson
517f5980e2 Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-17 04:58:43 +00:00
John Ericson
18493fd9c4 Move some Store functions from derivations.cc to store-api.cc 
This further continues with the dependency inverstion. Also I just went
ahead and exposed `parseDerivation`: it seems like the more proper
building block, and not a bad thing to expose if we are trying to be
less wedded to drv files on disk anywas.
 2020-06-17 04:30:16 +00:00
John Ericson
7130f0a3a6 Don't need abstract struct Derivation in local-store 2020-06-17 04:05:38 +00:00
John Ericson
21ef342172 Merge remote-tracking branch 'upstream/master' into derivation-header-include-order 2020-06-17 03:52:01 +00:00
John Ericson
02928f76fd Merge remote-tracking branch 'upstream/master' into multi-output-hashDerivationModulo 2020-06-17 03:51:02 +00:00
John Ericson
49308ef039 Merge remote-tracking branch 'upstream/master' into ca-drv 2020-06-17 03:39:39 +00:00
p01arst0rm
e9970a34e8 appended ' __attribute__((weak)); ' to 'extern char * * environ ' 2020-06-17 03:25:34 +01:00
p01arst0rm
c9d06558b6 replaced uncaught_exception with uncaught_exceptions 2020-06-17 03:15:47 +01:00
Eelco Dolstra
29542865ce Remove StorePath::clone() and related functions 2020-06-16 22:20:18 +02:00
Eelco Dolstra
df4da4f5da
Merge pull request #3702 from NixOS/store-path-cxx 
Rewrite StorePath class in C++
 2020-06-16 21:37:26 +02:00
John Ericson
fbf90bd693 Add another missing #include 2020-06-16 14:19:49 +00:00
Eelco Dolstra
759947bf72 StorePath: Rewrite in C++ 
On nix-env -qa -f '<nixpkgs>', this reduces maximum RSS by 20970 KiB
and runtime by 0.8%. This is mostly because we're not parsing the hash
part as a hash anymore (just validating that it consists of base-32
characters).

Also, replace storePathToHash() by StorePath::hashPart().
 2020-06-16 14:28:41 +02:00
Eelco Dolstra
72e17290d4 Fix FTP support 
Fixes #3618.
 2020-06-16 11:53:04 +02:00
Tobias Pflug
cd8214c398 Fix logging unit tests 2020-06-16 10:23:15 +02:00
John Ericson
7e7e3b71f3 Add mising #include for strerror 2020-06-15 23:35:07 +00:00
Eelco Dolstra
a588b6b19d Print only one error message if a build fails 
E.g. instead of

  error: --- BuildError ----------------------------------------------- nix
  builder for '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed with exit code 1
  error: --- Error ---------------------------------------------------- nix
  build of '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed

we now get

  error: --- Error ---------------------------------------------------- nix
  builder for '/nix/store/03nk0a3n8h2948k4lqfgnnmym7knkcma-foo.drv' failed with exit code 1
 2020-06-15 19:35:31 +02:00
Eelco Dolstra
24a3208247 Include only the base name of the program in error messages 2020-06-15 19:35:31 +02:00
Eelco Dolstra
8b099812ea Respect terminal width printing error messages 2020-06-15 19:35:31 +02:00
Eelco Dolstra
f20bb983ca Cleanup 2020-06-15 18:16:03 +02:00
Eelco Dolstra
4e995bc8a6 Always hide the progress bar on exit 2020-06-15 18:01:05 +02:00
Eelco Dolstra
31707735b6 Remove unnecessary amDone() overrides 2020-06-15 16:47:21 +02:00
Eelco Dolstra
ccfa6b3eee Give better error message about <...> in pure eval mode 2020-06-15 16:12:27 +02:00
Eelco Dolstra
5ed5d7acbd Improve "waiting for locks" messages 
These are now shown in the progress bar.

Closes #3577.
 2020-06-15 16:03:29 +02:00
Eelco Dolstra
e14e62fddd Remove trailing whitespace 2020-06-15 14:12:39 +02:00
Eelco Dolstra
1fb762d11f Get rid of explicit ErrorInfo constructors 2020-06-15 14:06:58 +02:00
Eelco Dolstra
fd64e4fb96 Disambiguate BaseError(Args) constructor 
This means that 'throw Error({ ... ErrorInfo ... })' now works.
 2020-06-15 13:50:33 +02:00
Eelco Dolstra
7a77762961 Merge branch 'errors-phase-2' of https://github.com/bburdette/nix 2020-06-15 11:46:31 +02:00
Matthew Bauer
c214cda940 Correctly substitute from different storeDir 
Originally, the test was only checking for different “real” storeDir.
That’s an easy case to handle, but the much harder one is if different
virtual store dirs are used. To do this, we need the SubstitutionGoal
to know about the ca, so it can recalculate the path to copy it over.
An important note here is that the store path passed to copyStorePath
needs to be one for srcStore - so that queryPathInfo works properly.

This also adds an error message when the store path from queryPathInfo
is different from the one we requested.
 2020-06-13 00:07:42 -05:00
Matthew Bauer
b2cb288cdd Add makeFixedOutputPathFromCA function 
This puts what we are already doing into a shared method. It just
needs a path name and a ca and produces a store path.
 2020-06-12 16:36:35 -05:00
John Ericson
f6f01416b7 Use std::string_view in a few more places 2020-06-12 21:32:30 +00:00
Matthew Bauer
88120442d2 Debug when storePath changes 
these rewrites should be transparent, but they are important to know
about when debugging
 2020-06-12 15:32:52 -05:00
Matthew Bauer
e3cb536f19 Fix add-to-store --flat to put in correct hash 2020-06-12 15:25:45 -05:00
Matthew Bauer
5b386b05f5 Recompute storePath based on isContentAddressed 2020-06-12 14:57:45 -05:00
Matthew Bauer
85d01e1f65 Don’t use makeStorePath 2020-06-12 14:53:30 -05:00
Matthew Bauer
9077b9e6b2 Separate dstStore path from srcStore path 2020-06-12 14:27:28 -05:00
Matthew Bauer
2f2ac850b5 Compute new store path correctly 2020-06-12 14:03:17 -05:00
Matthew Bauer
1c55f16a16 Add --flat to nix add-to-store 
This can be used to add flat hashes to the nix store.
 2020-06-12 12:42:54 -05:00
Eelco Dolstra
2853ba4ab2 Fix build 2020-06-12 19:00:48 +02:00
Eelco Dolstra
00fa7e2205
Merge pull request #3674 from matthewbauer/allow-empty-hash2 
Allow empty hash in derivations
 2020-06-12 18:18:12 +02:00
Matthew Bauer
ea0d29d99a Provide base argument to to_string 2020-06-12 10:18:27 -05:00
Matthew Bauer
11c97070f3 Fix storeDir != storeDir condition 
this needs to only continue if the path replacement fails.
 2020-06-12 10:14:03 -05:00
Matthew Bauer
b260c9ee03 Add newHashAllowEmpty helper function 
This replaces the copy&paste with a helper function in hash.hh.
 2020-06-12 10:11:16 -05:00
Matthew Bauer
3e3eaa90dd Remove hashed-mirrors 2020-06-12 09:51:44 -05:00
Matthew Bauer
79c169d1c6 Allow substituting from different storeDir 
Substituters can substitute from one store dir to another with a
little bit of help. The store api just needs to have a CA so it can
recompute the store path based on the new store dir. We can only do
this for fixed output derivations with no references, though.
 2020-06-12 09:49:09 -05:00
Eelco Dolstra
9f736dd89d Add Store::readDerivation() convenience function 2020-06-12 13:04:52 +02:00
Eelco Dolstra
045b07200c Remove Store::queryDerivationOutputNames() 
This function was used in only one place, where it could easily be
replaced by readDerivation() since it's not
performance-critical. (This function appears to have been modelled
after queryDerivationOutputs(), which exists only to make the garbage
collector faster.)
 2020-06-12 12:46:33 +02:00
Eelco Dolstra
4a4c063222
Merge pull request #3670 from gilligan/add-pool-tests 
Add tests for pool.hh
 2020-06-12 11:19:05 +02:00
Eelco Dolstra
7db879e65e Check 'follows' inputs 2020-06-12 00:52:56 +02:00
Ben Burdette
ef1b3f21b6 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-11 14:06:35 -06:00
Eelco Dolstra
d15c20efd5 diffLockFiles(): Show 'follows' changes 2020-06-11 22:00:58 +02:00
Eelco Dolstra
ac4d43a31b
Merge pull request #3073 from tweag/machine-logs 
Add an option to print the logs in a machine-readable format
 2020-06-11 15:45:18 +02:00
Eelco Dolstra
dd9bb11d0d Move names.{cc,hh} to libstore 2020-06-11 15:42:18 +02:00
Eelco Dolstra
95eb064062 Shut up warning 2020-06-11 15:39:30 +02:00
Eelco Dolstra
8bd892117a Style fixes 2020-06-11 15:39:08 +02:00
Eelco Dolstra
0c62b4ad0f Represent 'follows' inputs explicitly in the lock file 
This fixes an issue where lockfile generation was not idempotent:
after updating a lockfile, a "follows" node would end up pointing to a
new copy of the node, rather than to the original node.
 2020-06-11 14:40:21 +02:00
Tobias Pflug
4750d98bbd Add tests for pool.hh 2020-06-10 22:29:50 +02:00
Eelco Dolstra
195ed43b60 Preserve 'isFlake' when not updating a lock file entry 2020-06-10 16:24:05 +02:00
Eelco Dolstra
2226e97ec2 Combine lock file update messages 2020-06-10 15:22:12 +02:00
Eelco Dolstra
fc6c7af424 Add helper function printInputPath() 2020-06-10 15:20:00 +02:00
Eelco Dolstra
b9ae1bdd7a
Merge pull request #3655 from zimbatm/hash-encoding-prepare 
libutils/hash: remove default encoding
 2020-06-10 11:48:38 +02:00
Matthew Bauer
b2c8061b44
Disable extra-platforms = i686-linux on wsl1 (#3676) 
WSL1 doesn’t support i686-linux emulation, see https://github.com/microsoft/wsl/issues/2468
 2020-06-09 21:53:53 +00:00
Matthew Bauer
19aa892f20 Support empty hash in fetchers 
fetchTarball, fetchTree, and fetchGit all have *optional* hash attrs.
This means that we need to be careful with what we allow to avoid
accidentally making these defaults. When ‘hash = ""’ we assume the
empty hash is wanted.
 2020-06-09 11:10:54 -05:00
Eelco Dolstra
29e0748847 Show HTTP status message 
For example:

  warning: unable to download 'https://api.github.com/repos/edolstra/dwarffs/commits/master': HTTP error 403 ('rate limit exceeded'); using cached version
 2020-06-09 14:20:22 +02:00
Eelco Dolstra
447ea52b07 FileTransfer: Don't store status since curl already does that 2020-06-09 14:05:15 +02:00
Eelco Dolstra
6cfc2db494 Fix applyOverride() for github 2020-06-09 13:45:07 +02:00
Matthew Bauer
762273f1fd Allow empty hash in derivations 
follow up of https://github.com/NixOS/nix/pull/3544

This allows hash="" so that it can be used for debugging purposes. For
instance, this gives you an error message like:

  warning: found empty hash, assuming you wanted 'sha256:0000000000000000000000000000000000000000000000000000'
  hash mismatch in fixed-output derivation '/nix/store/asx6qw1r1xk6iak6y6jph4n58h4hdmbm-nix':
    wanted: sha256:0000000000000000000000000000000000000000000000000000
    got:    sha256:0fpfhipl9v1mfzw2ffmxiyyzqwlkvww22bh9wcy4qrfslb4jm429
 2020-06-09 01:23:37 -05:00
Ben Burdette
2f19650768 add file origin to Pos in stests 2020-06-08 11:21:17 -06:00
Ben Burdette
b1c53b034c Merge branch 'errors-phase-2' into caveman-LOCs 2020-06-08 11:10:13 -06:00
regnat
801112de1a Move progress-bar.cc to libmain 
Needed so that we can include it as a logger in loggers.cc without
adding a dependency on nix

This also requires moving names.hh to libutil to prevent a circular
dependency between libmain and libexpr
 2020-06-08 17:16:52 +02:00
Eelco Dolstra
e073f2c584 nix flake: Require 'flakes' feature 2020-06-08 16:23:54 +02:00
Eelco Dolstra
6470450ab4 Add completion for --update-input 2020-06-08 16:20:00 +02:00
Eelco Dolstra
c27f92698b Style fixes 2020-06-08 13:24:01 +02:00
Tobias Pflug
cd6dbf951a Add compression unit tests 2020-06-08 11:34:37 +02:00
regnat
4983401440 Unify the printing of the logs between bar-with-logs and raw 
Make the printing of the build logs systematically go through the
logger, and replicate the behavior of `no-build-output` by having two
different loggers (one that prints the build logs and one that doesn't)
 2020-06-08 09:31:15 +02:00
regnat
170e86dff5 Make the logger customisable 
Add a new `--log-format` cli argument to change the format of the logs.
The possible values are
- raw (the default one for old-style commands)
- bar (the default one for new-style commands)
- bar-with-logs (equivalent to `--print-build-logs`)
- internal-json (the internal machine-readable json format)
 2020-06-08 09:31:15 +02:00
Ben Burdette
94c347577e set verbosity levels 2020-06-07 07:24:49 -06:00
Tobias Pflug
e60747b5fb Remove error-demo/error-demo.cc 
The logging.hh superseeds the demo
 2020-06-06 10:23:12 +02:00
Tobias Pflug
952e72c804 Add tests for logging.hh 2020-06-06 10:22:32 +02:00
Eelco Dolstra
9ef6048d78 diffLockFiles(): Fix assertion failure 
There are some cases where this inequality didn't hold, in particular
due to the Input / TreeInfo merge, where we're not always showing
narHash.
 2020-06-05 20:52:23 +02:00
Eelco Dolstra
d558fb98f6
Merge pull request #3656 from obsidiansystems/handle-unknown-file-ingestion 
Add error message when FileIngestionMethod is out of bounds
 2020-06-05 17:18:12 +02:00
Eelco Dolstra
488ff83e6b Fix completion of --template 2020-06-05 14:09:12 +02:00
Eelco Dolstra
39e84c35d0 Fix log-prefix of nix build -L 
Alternative fix to #3661. The cause was that 'name' is a
std::string_view into a temporary which could get overwritten.
 2020-06-05 10:45:05 +02:00
Carlo Nucera
d614166cb6 Fix condition error and make test suite pass 2020-06-04 17:21:21 -04:00
John Ericson
e5cc1ebc5d Merge remote-tracking branch 'upstream/master' into no-stringly-typed-derivation-output 2020-06-04 21:04:35 +00:00
John Ericson
a7b82fd006 Remove file which shouldn't be committed 2020-06-04 21:04:20 +00:00
John Ericson
94ddea9e2f Use readString rather than >> temporary 
Fixed the rest of these before, but this one slipped through.
 2020-06-04 20:55:08 +00:00
John Ericson
744ce9ce16 Merge branch 'master' of github.com:NixOS/nix into validPathInfo-ca-proper-datatype 2020-06-04 20:46:58 +00:00
John Ericson
2041499b5e Flip boolean 
Thanks Matt!
 2020-06-04 20:42:25 +00:00
John Ericson
ed86acf02a Use some std::optional::has_value for clarity 2020-06-04 20:42:02 +00:00
John Ericson
574d5460f0 Make sure info.ca tag bit is set in nix add-to-store 2020-06-04 20:33:28 +00:00
Eelco Dolstra
ab54031e04 getDefaultUrl() -> value_or() 2020-06-04 20:24:28 +02:00
Eelco Dolstra
810b2c6a48 nix flake init: Add a '--template' flag 
The initial contents of the flake is specified by the
'templates.<name>' or 'defaultTemplate' output of another flake. E.g.

  outputs = { self }: {

    templates = {

      nixos-container = {
        path = ./nixos-container;
        description = "An example of a NixOS container";
      };

    };

  };

allows

  $ nix flake init -t templates#nixos-container

Also add a command 'nix flake new', which is identical to 'nix flake
init' except that it initializes a specified directory rather than the
current directory.
 2020-06-04 20:22:25 +02:00
Ben Burdette
94427ffee3 add some comments 2020-06-04 11:53:19 -06:00
Matthew Bauer
2299ef705c Add error message when FileIngestionMethod is out of bounds 
bool coerces anything >0 to true, but in the future we may have other
file ingestion methods. This shows a better error message when the
“recursive” byte isn’t 1.
 2020-06-04 11:32:39 -05:00
Eelco Dolstra
dc305500c3
Merge pull request #3660 from Kloenk/selfhost-gitlab 
add support for selfhosted gitlab/github
 2020-06-04 14:59:33 +02:00
Eelco Dolstra
d746ef4a81 Disable eval cache with --impure 
Fixes

  $ nix build nixpkgs#zoom-us
  error: Package ‘zoom-us-5.0.399860.0429’ in /nix/store/m79v7h75b69fkk8d2qcwm555l3wq6fmv-source/pkgs/applications/networking/instant-messengers/zoom-us/default.nix:126 has an unfree license (‘unfree’), refusing to evaluate.

  $ nix build nixpkgs#zoom-us --impure
  error: cached failure of attribute 'legacyPackages.x86_64-linux.zoom-us.drvPath'
 2020-06-04 14:53:51 +02:00
Finn Behrens
108debef6f
add support for selfhosted gitlab/github 2020-06-04 14:42:39 +02:00
Eelco Dolstra
f85606c431 Merge remote-tracking branch 'origin/master' into flakes 2020-06-04 13:16:28 +02:00
Eelco Dolstra
0f44b60e6d Make 'nix dev-shell' a deprecated alias for 'nix develop' 2020-06-04 11:14:19 +02:00
Eelco Dolstra
61e3d598b6 Rename 'nix dev-shell' to 'nix develop' 
Fixes #3648.
 2020-06-04 10:57:40 +02:00
John Ericson
53bc8ff152 No C++ designated initializers yet with Clang 7 2020-06-03 20:45:14 -04:00
Ben Burdette
4335ba999b Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-03 17:00:00 -06:00
Ben Burdette
f97576c5d9 newline-as-prefix; no final newline in output. 2020-06-03 14:47:00 -06:00
Carlo Nucera
132d6f2c24 Clarify the description of StorePath construction 2020-06-03 16:08:32 -04:00
John Ericson
74b251b2f3 Don't anticipate multiple CA outputs for now 2020-06-03 18:53:04 +00:00
John Ericson
3a9e4c3262 Don't anticipate CA but not fixed outputs for now 2020-06-03 18:50:45 +00:00
John Ericson
2500403059 Use enum and predicates rather than bitfile for derivation type 2020-06-03 17:38:54 +00:00
John Ericson
6b7f4ec4ab Merge remote-tracking branch 'upstream/master' into ca-drv 2020-06-03 16:36:06 +00:00
Eelco Dolstra
c20591ddc3 Merge remote-tracking branch 'origin/master' into flakes 2020-06-03 16:15:22 +02:00
zimbatm
6ee03b8444
libutils/hash: remove default encoding 
This will make it easier to reason about the hash encoding and switch to
SRI everywhere where possible.
 2020-06-03 13:49:51 +02:00
John Ericson
01572c2198
Missing #include <cassert> in lru-cache.hh (#3654) 
This was a latent bug that just appeared because of the tests that were
added. Remember to wait for CI! :)
 2020-06-03 10:15:22 +00:00
John Ericson
3c78ac348c Merge remote-tracking branch 'obsidian/no-hash-type-unknown' into validPathInfo-ca-proper-datatype 2020-06-03 04:44:24 +00:00
John Ericson
fecff16a6e Merge remote-tracking branch 'obsidian/missing-include-0' into validPathInfo-ca-proper-datatype 2020-06-02 23:23:30 +00:00
John Ericson
39ba87be9b Missing #include <cassert> in lru-cache.hh 
This was a latent bug that just appeared because of the tests that were
added. Remember to wait for CI! :)
 2020-06-02 21:36:53 +00:00
John Ericson
406dbb7fce outputHashAlgo can be blank so parse accordingly 
It is blank for SRI hashes.
 2020-06-02 21:09:15 +00:00
John Ericson
1fcd3afc38 Fix hashes 2020-06-02 20:35:17 +00:00
Carlo Nucera
75d2581390 Typo 2020-06-02 16:21:18 -04:00
Carlo Nucera
78f137e931 Validate text version instead, throw Errors 2020-06-02 16:20:22 -04:00
Carlo Nucera
a5cdf1867e Add assertions for SHA256 in fixed case 2020-06-02 16:13:08 -04:00
Carlo Nucera
fd2eb41e64 Move file-hash to content-address 2020-06-02 15:44:58 -04:00
Carlo Nucera
343c20a404 WIP Completed implementation 2020-06-02 15:23:21 -04:00
John Ericson
c664e68b87 Fix to-base --type handler to correctly set std::optional flag 
Now that we have a separate flag function, also describe why it is
optional.
 2020-06-02 18:25:32 +00:00
Carlo Nucera
390bf64858 WIP 2020-06-02 14:15:58 -04:00
John Ericson
c502119fd3 to-base supports parsing SRI hashes, so make type flag optional 2020-06-02 18:05:26 +00:00
John Ericson
a33270ce1d Clean up ValidPathInfo::isContentAddressed with std::visit 2020-06-02 17:04:21 +00:00
John Ericson
25e61812f3
Apply suggestions from code review 
Co-authored-by: Matthew Bauer <mjbauer95@gmail.com>
 2020-06-02 12:47:18 -04:00
John Ericson
d73dbc8e4c Remove hashingWithUnknownAlgoExits 
A valid hash type must be provided now. The hash itself can still be
invalid, but that doesn't cause an `abort()`.
 2020-06-02 16:28:54 +00:00
John Ericson
64cffb804a Merge remote-tracking branch 'upstream/master' into no-hash-type-unknown 2020-06-02 16:07:25 +00:00
John Ericson
450dcf2c1b Remove HashType::Unknown 
Instead, `Hash` uses `std::optional<HashType>`. In the future, we may
also make `Hash` itself require a known hash type, encoraging people to
use `std::optional<Hash>` instead.
 2020-06-02 15:52:13 +00:00
Carlo Nucera
a5d820a0a3 Change parseCa(Opt) to parseContentAddress(Opt) 2020-06-02 11:00:10 -04:00
Ben Burdette
156d4f8bc8 remove extra space in SysErrors 2020-06-02 08:45:37 -06:00
John Ericson
1b6461f671 Merge remote-tracking branch 'upstream/master' into validPathInfo-ca-proper-datatype 2020-06-02 14:31:18 +00:00
Ben Burdette
d82d230b40 elide the 'ErrorInfo' in logError and logWarning calls 2020-06-02 08:22:24 -06:00
Eelco Dolstra
bfa1acd85c
Merge pull request #3639 from obsidiansystems/do-fixme-store-removes 
Remove `addToStore` variant as requested by `FIXME`
 2020-06-02 15:39:07 +02:00
Eelco Dolstra
c16fdda3a6 Merge branch 'lru-tests' of https://github.com/gilligan/nix 2020-06-02 12:07:48 +02:00
Eelco Dolstra
e9fee8e6a7
src/libutil/tests/lru-cache.cc: Check erase() 
Co-authored-by: James Lee <jbit@jbit.net>
 2020-06-02 12:06:59 +02:00
John Ericson
efcd30da89 WIP 2020-06-02 00:37:43 +00:00
John Ericson
754c910953 WIP more progress 2020-06-01 19:26:40 -04:00
Carlo Nucera
da39092a39 WIP 2020-06-01 18:53:31 -04:00
Carlo Nucera
0e9438b6d3 Create new file-hash files 2020-06-01 17:32:40 -04:00
Carlo Nucera
0cb67ecbd3 Merge branch 'derivation-header-include-order' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype 2020-06-01 17:13:11 -04:00
Carlo Nucera
f4b89e11a4 Merge branch 'no-stringly-typed-derivation-output' of github.com:Ericson2314/nix into validPathInfo-ca-proper-datatype 2020-06-01 17:12:50 -04:00
Matthew Kenigsberg
c254254a80 use Tree ctor 2020-06-01 12:32:17 -06:00
Matthew Kenigsberg
7680993506 Tree ctors 2020-06-01 09:01:37 -06:00
Matthew Kenigsberg
ff1320b850 fetchOrSubstituteTree improvements 
Caches tree in addition to lockedRef, and explicitly writes out the logic for different combinations of cached/uncached flakes and indirect/resolved/locked flakes. This eliminates uneccessary calls to lookupInFlakeCache, fetchTree, maybeLookupFlake, and flakeCache.push_back
 2020-06-01 02:57:22 -06:00
Tobias Pflug
eca1ff7a9f Add tests for lru-cache.hh 2020-05-31 01:05:05 +02:00
Nikola Knezevic
fb38459d6e Ensure we restrict refspec interpretation while fetching 
As `git fetch` may chose to interpret refspec to it's liking, ensure that we
only pass refs that begin with `refs/` as is, otherwise, prepend them with
`refs/heads`. Otherwise, branches named `heads/foo` (I know it's bad, but it's
allowed), would be fetched as `foo`, instead of `heads/foo`.
 2020-05-30 12:33:38 +02:00
Nikola Knezevic
77007d4eab Improve ref validity checking in fetchGit 
The previous regex was too strict and did not match what git was allowing. It
could lead to `fetchGit` not accepting valid branch names, even though they
exist in a repository (for example, branch names containing `/`, which are
pretty standard, like `release/1.0` branches).

The new regex defines what a branch name should **NOT** contain. It takes the
definitions from `refs.c` in https://github.com/git/git and `git help
check-ref-format` pages.

This change also introduces a test for ref name validity checking, which
compares the result from Nix with the result of `git check-ref-format --branch`.
 2020-05-30 12:29:35 +02:00
Eelco Dolstra
89e0b3e2d6 Move substitution into Input::fetch() 
Closes #3520.
 2020-05-30 01:16:53 +02:00
Eelco Dolstra
0e7f77a59a Check revCount / lastModified input attributes if specified 2020-05-30 00:59:13 +02:00
Eelco Dolstra
950b46821f Remove TreeInfo 
The attributes previously stored in TreeInfo (narHash, revCount,
lastModified) are now stored in Input. This makes it less arbitrary
what attributes are stored where.

As a result, the lock file format has changed. An entry like

    "info": {
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },
    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github"
    },

is now stored as

    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github",
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },

The 'Input' class is now a dumb set of attributes. All the fetcher
implementations subclass InputScheme, not Input. This simplifies the
API.

Also, fix substitution of flake inputs. This was broken since lazy
flake fetching started using fetchTree internally.
 2020-05-30 00:44:11 +02:00
John Ericson
fac0c2d54a Remove addToStore variant as requested by FIXME 
The idea is it's always more flexible to consumer a `Source` than a
plain string, and it might even reduce memory consumption.

I also looked at `addToStoreFromDump` with its `// FIXME: remove?`, but
the worked needed for that is far more up for interpretation, so I
punted for now.
 2020-05-29 17:02:32 -04:00
Ben Burdette
734283d636 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-05-29 09:51:37 -06:00
Eelco Dolstra
5633c0975b Factor out GitHub / GitLab commonality 2020-05-29 14:23:32 +02:00
Finn Behrens
5256bc77ca
add gitlab libfetcher 2020-05-28 23:00:08 +02:00
Carlo Nucera
6dd471ebf6 Fixing the result of merge 2020-05-28 12:14:36 -04:00
Carlo Nucera
4f597fb901 Merge branch 'master' of github.com:NixOS/nix into enum-class 2020-05-28 10:58:22 -04:00
John Ericson
5b4cd84bc2 Merge remote-tracking branch 'me/more-rust-ffi' into no-stringly-typed-derivation-output 2020-05-28 10:35:53 -04:00
John Ericson
ef71caba29 Merge remote-tracking branch 'upstream/master' into more-rust-ffi 2020-05-28 10:31:46 -04:00
Eelco Dolstra
f60ce4fa20
Merge pull request #3631 from andir/libutil-config-tests 
Add unit tests for config.cc
 2020-05-28 13:51:37 +02:00
Eelco Dolstra
de141fcb79
Merge pull request #3455 from Ericson2314/enum-FileIngestionMethod 
Replace some `bool recursive` with a new `FileIngestionMethod` enum
 2020-05-28 13:50:06 +02:00
Eelco Dolstra
17ca997fc6 Merge remote-tracking branch 'origin/master' into flakes 2020-05-28 12:55:24 +02:00
Eelco Dolstra
c3eff22f46 Merge branch 'store-visited' of https://github.com/mkenigs/nix into flakes 2020-05-28 12:15:39 +02:00
Eelco Dolstra
6286272371 nixpkgsFlakeRef(): Use locked nixpkgs 2020-05-28 12:13:13 +02:00
John Ericson
0f96f45061 Use FileIngestionMethod for nix hash 
There was an enum there that matched in perfectly.
 2020-05-27 23:50:11 -04:00
Matthew Bauer
c66441a646 Rename some variables named “recursive” to “method” 
This is much less confusing since recursive is no longer a boolean.
 2020-05-27 13:21:26 -05:00
Matthew Bauer
7873fd175d Don’t use FileIngestionMethod for StorePathsCommand 
This is a different recursive than used in makeFixedOutputPath.
 2020-05-27 13:21:11 -05:00
Andreas Rammhold
fc137d2f00
config.hh: Add documentation 
Provides some general overview on the mechanics of Config/Setting and
comments for the public methods of Config.
 2020-05-27 17:47:18 +02:00
Andreas Rammhold
9df3d8ccd7
tests/config.cc: add tests for Config::applyConfig 2020-05-27 17:47:18 +02:00
Andreas Rammhold
e1b8c64c04
config.cc: extract parts of applyConfigFile into applyConfig 
This moves the actual parsing of configuration contents into applyConfig
which applyConfigFile is then going to call. By changing this we can now
test the configuration file parsing without actually create a file on
disk.
 2020-05-27 17:47:18 +02:00
Andreas Rammhold
93129cf1dd
Add unit tests for config.cc 2020-05-27 17:47:17 +02:00
Eelco Dolstra
66d3ac94c9 Merge pull request #3621 from gilligan/add-json-tests 
Add unit tests for "json.hh"
 2020-05-27 11:08:12 +02:00
Eelco Dolstra
dae6a267a8
Merge pull request #3625 from gilligan/xml-writer-tests 
Add unit tests for xml-writer
 2020-05-27 11:07:53 +02:00
Carlo Nucera
f3f520c14c Change syntax for CI 2020-05-26 12:51:28 -04:00
Carlo Nucera
89a5ac9d3b Merge remote-tracking branch 'john-ericson/more-rust-ffi' into no-stringly-typed-derivation-output 2020-05-26 12:31:26 -04:00
Carlo Nucera
d49e65ba9d Merge remote-tracking branch 'john-ericson/enum-FileIngestionMethod' into no-stringly-typed-derivation-output 2020-05-26 12:30:48 -04:00
Carlo Nucera
0f3f901071 Merge remote-tracking branch 'origin/master' into more-rust-ffi 2020-05-26 11:46:42 -04:00
Carlo Nucera
c2f33edd1f
Update src/libutil/rust-ffi.hh 
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
 2020-05-26 11:43:18 -04:00
Carlo Nucera
b90241ceb1 Change remaining bools with FileIngestionMethod 2020-05-26 11:32:41 -04:00
Carlo Nucera
6d73c10041 Merge remote-tracking branch 'origin/master' into enum-FileIngestionMethod 2020-05-26 11:14:08 -04:00
Tobias Pflug
4b388e8431 Add unit tests for xml-writer 2020-05-25 18:34:55 +02:00
Tobias Pflug
c284700867 Add unit tests for "json.hh" 2020-05-25 11:57:45 +02:00
Tobias Pflug
ecc5c90dfc Add unit tests for hashing functions 2020-05-25 11:50:41 +02:00
Matthew Kenigsberg
8d67794da1 handle circular flake dependencies in list-inputs 2020-05-21 17:06:11 -06:00
Ben Burdette
b7057fa627 remove error-demo from make; clean up comment 2020-05-21 16:04:18 -06:00
Ben Burdette
0e49de6a2b position for stdin, string; (string) for trace; fix tests 2020-05-21 14:28:45 -06:00
Ben Burdette
6a420d672c print LOC for stdin, string args 2020-05-20 22:18:26 -06:00
Ben Burdette
85ce455b85 get code lines from the nix file 2020-05-20 17:25:02 -06:00
Tobias Pflug
a73a820a5d Add unit testes for url.cc 
This adds tests for

- parseURL
- percentDecode
- decodeQuery
 2020-05-20 16:37:35 +02:00
Eelco Dolstra
5ef64f05e6 Cleanup 2020-05-18 15:50:29 +02:00
Eelco Dolstra
0ed946aa61 Merge branch 'wait-for-builders' of https://github.com/serokell/nix 2020-05-18 13:48:45 +02:00
Matthew Kenigsberg
c4beded32e rm includes 2020-05-16 11:19:41 -06:00
Matthew Kenigsberg
ba7d7ed2e3 Create bashInteractive InstallableFlake 2020-05-16 11:03:06 -06:00
Matthew Kenigsberg
0858793604 Call lockFlake once and store in _lockedFlake 2020-05-16 11:03:06 -06:00
Matthew Kenigsberg
04821bc171 use flake's nixpkgs to find bashInteractive 2020-05-16 11:03:06 -06:00
Matthew Kenigsberg
8fbc8540d3 use nixpkgs#bashInteractive for dev-shell 2020-05-16 11:03:06 -06:00
Ben Burdette
92123c6c79 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-05-15 07:00:36 -06:00
Eelco Dolstra
5f64655ff4 Move registry-related commands from 'nix flake' to 'nix registry' 
This makes 'nix flake' less cluttered and more consistent (it's only
subcommands that operator on a flake). Also, the registry is not
inherently flake-related (e.g. fetchTree could also use it to remap
inputs).
 2020-05-15 14:38:10 +02:00
Ben Burdette
19694aa213 fix compile errors 2020-05-14 12:28:18 -06:00
Ben Burdette
4daccb279c formatting 2020-05-14 10:28:17 -06:00
Alexander Bantyev
183dd28266
Don't lock a user while doing remote builds 2020-05-14 17:00:54 +03:00
Ben Burdette
ef9dd9f9bc formatting and a few minor changes 2020-05-13 15:56:39 -06:00
Ben Burdette
c79d4addab consistent capitalization 2020-05-13 10:02:18 -06:00
Ben Burdette
bfca5fc395 change status messages to info level 2020-05-13 09:52:36 -06:00
Eelco Dolstra
849d3968db
Update src/libfetchers/git.cc 
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
 2020-05-13 10:41:21 +02:00
Ben Burdette
ecbb8e9c0a no blank line if no LOC 2020-05-12 14:41:30 -06:00
Ben Burdette
960d4362ed hint only 2020-05-12 13:54:18 -06:00
Ben Burdette
72ecccee57 convert to logWarning format 2020-05-12 12:19:34 -06:00
Ben Burdette
19cffc29c9 remove unused extra json fields 2020-05-12 12:09:12 -06:00
Ben Burdette
2a19bf8619 move pos to the first arg, to indicate its not used in a fmt template 2020-05-12 11:27:37 -06:00