max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2024-11-23 22:46:16 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
7966 commits 7 branches 0 tags 64 MiB
Commit graph

4738 commits

Author SHA1 Message Date
John Ericson
bc109648c4 Get rid of LocalStore::addToStoreCommon 
I got it to just become `LocalStore::addToStoreFromDump`, cleanly taking
a store and then doing nothing too fancy with it.

`LocalStore::addToStore(...Path...)` is now just a simple wrapper with a
bare-bones sinkToSource of the right dump command.
 2020-07-15 23:14:30 +00:00
John Ericson
64b7421741 Merge branch 'handle-flat-case' into from-dump-stream 2020-07-15 21:22:55 +00:00
John Ericson
d087cf4855 Revert "Revert "LocalStore::addToStore(srcPath): Handle the flat case"" 
This reverts commit cff2157185.
 2020-07-15 21:10:33 +00:00
John Ericson
aff69e229c Merge remote-tracking branch 'obsidian/fix-add-to-star-narhash' into from-dump-stream 2020-07-15 21:05:29 +00:00
Eelco Dolstra
36a1242603 nix why-depends: Fix shortest path calculation 
This was completely broken since d8972317fc.
 2020-07-15 21:11:07 +02:00
Eelco Dolstra
cff2157185 Revert "LocalStore::addToStore(srcPath): Handle the flat case" 
This reverts commit a2c27022e9. See
addToStoreSlow(), we don't need to handle this case efficiently
anymore. In fact, we can almost remove the method/hashAlgo arguments
since the non-recursive and/or non-SHA256 are almost not used anymore.
 2020-07-15 12:49:03 +02:00
John Ericson
650c2c6558 Rename variable nar -> dump according to TODO 2020-07-14 21:28:50 +00:00
John Ericson
8173e7bfef Fix localhost::addToStore(...Path...) 
We were calculating the nar hash wrong when the file ingestion method
was flat. I don't think there's anything we can do in that case but dump
the file again, so that's what I do.

As an optomization, we again could reuse the original dump for just the
recursive and non-sha256 case, but I rather do that after this fix, and
after my other PRs which deduplicate this code.
 2020-07-14 21:12:11 +00:00
John Ericson
592851fb67 LocalStore::addToStoreFromDump copy in chunks 
Rather than copying byte-by-byte, we let the coroutine know how much
data we would like it to send back to us.
 2020-07-14 13:56:36 +00:00
John Ericson
9de96ef7d4 Dedup LocalStore::addToStore* 
The downsides is that the coroutine has byte-by-byte loop transfer. Will
fix that next.
 2020-07-14 13:56:36 +00:00
John Ericson
c86fc3a965 Crudely make addToStoreFromDump take Source not string 
I just as little beyond the type as possible, so the implementation
changes this enables can be reviewed separately.
 2020-07-14 13:56:36 +00:00
John Ericson
9ec10046e0 Narrow scope of temporary value 2020-07-14 13:54:29 +00:00
Eelco Dolstra
926c3a6664 Doh 2020-07-14 11:55:54 +02:00
Eelco Dolstra
43b8e96d30 Fix 'nix verify --all' on a binary cache (cached case) 2020-07-13 20:17:00 +02:00
Eelco Dolstra
9502c0e2eb nix verify: Show correct path when using --all on a binary cache 2020-07-13 20:12:44 +02:00
Eelco Dolstra
7c2fef0a81 Make 'nix copy' to s3:// binary caches run in constant memory 2020-07-13 20:07:19 +02:00
Eelco Dolstra
493961b689 Remove istringstream_nocopy 2020-07-13 18:31:19 +02:00
Eelco Dolstra
545bb2ed03 Remove 'accessor' from addToStore() 
This is only used by hydra-queue-runner and it's better to implement
it there.
 2020-07-13 18:31:19 +02:00
Eelco Dolstra
0a9da00a10 NarAccessor: Run in constant memory 2020-07-13 17:30:42 +02:00
Eelco Dolstra
fc84c358d9 Make 'nix copy' to file:// binary caches run in constant memory 2020-07-13 16:28:45 +02:00
Eelco Dolstra
400f1a9b59 Store::pathInfoToJSON(): Use consistent format for downloadHash 2020-07-13 16:26:09 +02:00
Eelco Dolstra
c0dd05131e toStorePath(): Return a StorePath and the suffix 2020-07-13 16:25:48 +02:00
Eelco Dolstra
1d01ae816b Fix 'nix verify --all' on a binary cache and add a test 2020-07-13 14:35:01 +02:00
Maximilian Bosch
64f03635d7
Fix ANSI color constants 
The `m` acts as termination-symbol when declaring graphics. Because
of this, the `;1m` doesn't have any effect and is directly printed to
the console:

```
$ nix repl
> builtins.fetchGit { /* ... */ }
{ outPath = "/nix/store/s0f0iz4a41cxx2h055lmh6p2d5k5bc6r-source"; rev = "e73e45b723a9a6eecb98bd5f3df395d9ab3633b6"; revCount = ;1m428; shortRev = "e73e45b"; submodules = ;1mfalse; }
```

Introduced by 6403508f5a.
 2020-07-12 16:52:20 +02:00
Eelco Dolstra
8efa23bb99 Avoid a redundant hash 2020-07-10 15:56:24 +02:00
Eelco Dolstra
5dff49f661 Factor out commonality between nix-prefetch-url and nix-store --add-fixed 2020-07-10 13:21:37 +02:00
Eelco Dolstra
7f1a86d57c nix-store --add-fixed: Run in constant memory 2020-07-10 12:51:56 +02:00
Eelco Dolstra
06e3dd9005 nix-prefetch-url: Run in constant memory when using RemoteStore 
Fixes #3684.
 2020-07-10 11:22:48 +02:00
Eelco Dolstra
a2c27022e9 LocalStore::addToStore(srcPath): Handle the flat case 
This helps nix-prefetch-url when using a local store.
 2020-07-09 15:54:32 +02:00
Eelco Dolstra
b981e5aacf Cleanup 2020-07-08 22:07:21 +02:00
Eelco Dolstra
34f25124ba Make LocalStore::addToStore(srcPath) run in constant memory 
This reduces memory consumption of

  nix-instantiate \
    -E 'with import <nixpkgs> {}; runCommand "foo" { src = ./blender; } "echo foo"' \
    --option nar-buffer-size 10000

(where ./blender is a 1.1 GiB tree) from 1716 to 36 MiB, while still
ensuring that we don't do any write I/O for small source paths (up to
'nar-buffer-size' bytes). The downside is that large paths are now
always written to a temporary location in the store, even if they
produce an already valid store path. Thus, adding large paths might be
slower and run out of disk space. ¯\_(ツ)_/¯ Of course, you can always
restore the old behaviour by setting 'nar-buffer-size' to a very high
value.
 2020-07-08 22:07:21 +02:00
Eelco Dolstra
16ec7785ca Fix 'got unknown message type 1 from Nix daemon' 
Example:

  $ nix-build -E 'with import <nixpkgs> {}; runCommand "foo" { x = runCommand "bar" {} "exit 1"; } "echo foo; exit 1"'
  warning: unknown setting 'auto-allocate-uids'
  these 2 derivations will be built:
    /nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv
    /nix/store/k4fsvrjl7cp2xpz7927iv7g0dqj1zyhs-foo.drv
  warning: unknown setting 'auto-allocate-uids'
  building '/nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv'...
  error: --- Error ----------------------------------------------------------------------------------------------------------------------------------------------------------------- nix-daemon
  builder for '/nix/store/v4fbdbhcdi949929a67g8farwf72zgam-bar.drv' failed with exit code 1
  error: --- Error ------------------------------------------------------------------------------------------------------------------------------------------------------------------ nix-build
  got unknown message type 1 from Nix daemon
 2020-07-08 15:53:14 +02:00
Eelco Dolstra
4055cfee36 Fix coverage build 2020-07-07 14:37:47 +02:00
Eelco Dolstra
7c9ece5dca exportReferencesGraph: Fix support for non-top-level store paths 
Fixes #3471.
 2020-07-07 14:25:43 +02:00
Ben Burdette
efd6a8b230 bump 2020-07-06 11:54:53 -06:00
Ben Burdette
75bfcf8d15 revamp trace code and test 2020-07-06 10:51:48 -06:00
Ben Burdette
a168224464 spacing 2020-07-04 18:30:49 -06:00
Ben Burdette
b29a4ea1dc Merge branch 'master' into add-trace 2020-07-03 07:57:36 -06:00
Eelco Dolstra
c3c7aedbb5 nix develop: Fix bad regex 
This was accepted by libstdc++ but not libc++.

https://hydra.nixos.org/build/123569154
 2020-07-03 14:58:58 +02:00
Eelco Dolstra
6f8fd3a3f2 Shut up a clang warning 2020-07-03 14:50:07 +02:00
regnat
223fbe644a Shorten the path to the test root 
Fix a socket length failure on the OSX builders
 2020-07-03 09:20:01 +02:00
Eelco Dolstra
b5e4253697 Fix abort in 'nix develop' 2020-07-02 18:24:11 +02:00
Ben Burdette
5818271c6e spacing 2020-07-02 09:41:54 -06:00
Ben Burdette
bf2788e4c1 move showTrace to new loggerSettings 2020-07-02 09:04:31 -06:00
Ben Burdette
5ae498872a assert for invalid fileorigin 2020-07-02 07:14:40 -06:00
Ben Burdette
8497891b99 spacing 2020-07-01 13:50:18 -06:00
Ben Burdette
a295b2ea96 if no errLoc, no Loc. 2020-07-01 12:02:02 -06:00
Ben Burdette
3629b0585a don't include errpos for addErrorContext 2020-07-01 11:49:01 -06:00
Ben Burdette
2a39c083dc non-pos trace test 2020-07-01 10:37:31 -06:00
Eelco Dolstra
38ccf2e241 Cleanup 2020-07-01 15:31:34 +02:00