max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2024-11-11 08:46:16 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
6354 commits 7 branches 0 tags 64 MiB
Commit graph

1748 commits

Author SHA1 Message Date
Eelco Dolstra
848a9375c3
Support escaping in store URIs 2018-08-03 21:20:38 +02:00
Eelco Dolstra
4e7d5f660c
SSHMaster: Bypass SSH when connecting to localhost 
This is primarily useful for testing since it removes the need to have
SSH working.
 2018-08-03 21:20:38 +02:00
Eelco Dolstra
e268bbc054
LegacySSHStore: Add remote-store option 
This is primarily useful for testing, e.g.

  $ nix copy --to 'ssh://localhost?remote-store=/tmp/nix' ...
 2018-08-03 21:20:38 +02:00
Graham Christensen
49a53c1d3f s3 binary cache: support specifying an endpoint 
Works for uploading and not downloading.
 2018-08-02 08:19:50 -04:00
Eelco Dolstra
4b4adbc93d
Fix race downloading into a sink 
This fixes 'error 10 while decompressing xz file'.

https://hydra.nixos.org/build/78308551
 2018-07-31 17:26:04 +02:00
Linus Heckemann
9ac1a79882 copyPathsToStore: honour keep-going 2018-07-24 17:05:06 +02:00
Eelco Dolstra
b920b90857
Detect redirect loops 
Fixes #2225.
 2018-06-18 10:36:19 +02:00
Eelco Dolstra
2e244fb68f
Merge pull request #2234 from symphorien/esrch2 
libstore/gc.cc: ignore ESRCH when reading symlinks in /proc
 2018-06-18 09:01:55 +02:00
Symphorien Gibol
ef8122064b libstore/gc.cc: ignore ESRCH when reading symlinks in /proc 
readlink is also affected by the problem fixed for regular files in
c567afe355df
 2018-06-16 18:04:09 +02:00
Matthew Justin Bauer
6d09e4400c
Fix optimise-store.cc to skip more paths on macOS 
I hate to make this such a large check but the lack of documentation means we really have no idea what's allowed. All of them reported so far have been within ".app/Contents" directories. That appears to be a safe starting point. However, I would not be surprised to also find more paths that are disallowed for instance in .framework or .bundle directories.

Fixes #2031
Fixes #2229
 2018-06-14 12:34:55 -04:00
Eelco Dolstra
7902fccf6f
hashDerivationModulo(): Handle chroot stores 
This makes 'nix copy --to /mnt /nix/store/bla.drv' work.
 2018-06-13 18:00:54 +02:00
Eelco Dolstra
3c5d9f478d
std::random_shuffle -> std::shuffle 
The former is removed in C++17.
 2018-06-13 16:56:19 +02:00
Yorick van Pelt
72a78beb34
Fix #2162: use getaddrinfo instead of curl to preload NSS 2018-06-12 13:05:14 +02:00
Symphorien Gibol
8c567afe35 libstore/gc.cc: ignore ESRCH when reading /proc 
If a process disappears between the time /proc/[pid]/maps is opened and
the time it is read, the read() syscall will return ESRCH. This should be ignored.
 2018-06-11 16:29:47 +02:00
AmineChikhaoui
170165ee5a
use debug instead of printTalkative to avoid spamming hydra queue 
runner logs.
 2018-06-08 11:43:32 +01:00
Eelco Dolstra
4ac4f675df
Don't require --fallback to recover from disappeared binary cache NARs 2018-06-05 16:04:41 +02:00
Eelco Dolstra
691b7582c7
Handle HTTP 410 Gone errors 2018-06-05 16:03:32 +02:00
Eelco Dolstra
ca06a9cea7
Improve binary cache fallback test 2018-06-05 15:26:12 +02:00
Eelco Dolstra
a2ec7a3bfd
Further improve upload messages 2018-06-05 14:37:26 +02:00
Will Dietz
54b1c59643 download.cc: fix readCallback return type (int -> size_t) 
I'm not sure if curl ever asks for enough data at once
for truncation to occur but better safe than sorry.
 2018-06-01 10:59:26 -05:00
Eelco Dolstra
73357500ac
Improve binary cache upload messages 
Don't say "download" when we mean "upload".
 2018-06-01 14:14:22 +02:00
Peter Simons
93aa3bea2e
Merge pull request #767 from mogorman/garbage_collect_keep_last_few 
Implement --delete-generations + flag for keeping last N number of gens
 2018-05-31 10:00:21 +02:00
Eelco Dolstra
3c2de9830d
RemoteStore::addToStore(): Ensure capacity is decreased on exceptions 2018-05-30 13:48:31 +02:00
Eelco Dolstra
a4c1618876
Fix deadlock in RemoteStore::queryPathInfoUncached() 2018-05-30 13:46:14 +02:00
Eelco Dolstra
7d21863bb3
Make <nix/fetchurl.nix> run in constant memory 
E.g.

  nix-build --store ~/my-nix/ -E 'import <nix/fetchurl.nix> { url = https://cache.nixos.org/nar/0nwi996rgq4b914qyx0mv2wq4k80hjac7xilikavagw7kxmn2iiv.nar.xz; sha256 = "0nwi996rgq4b914qyx0mv2wq4k80hjac7xilikavagw7kxmn2iiv"; }'

now runs in 17 MiB (was 70 MiB), while

  nix-build --store ~/my-nix/ -E 'import <nix/fetchurl.nix> { url = https://cache.nixos.org/nar/0nwi996rgq4b914qyx0mv2wq4k80hjac7xilikavagw7kxmn2iiv.nar.xz; sha256 = "0d2fxljdih3nc5dqx41hjzic3141ajil94m8kdbpryq569dpsbvb"; unpack = true; }'

runs in 17 MiB (was 346 MiB).
 2018-05-30 13:42:51 +02:00
Eelco Dolstra
5a654fd7dd
getDownloader(): Simplify 2018-05-30 13:42:30 +02:00
Eelco Dolstra
e87e4a60d6
Make HttpBinaryCacheStore::narFromPath() run in constant memory 
This reduces memory consumption of

  nix copy --from https://cache.nixos.org --to ~/my-nix /nix/store/95cwv4q54dc6giaqv6q6p4r02ia2km35-blender-2.79

from 176 MiB to 82 MiB. (The remaining memory is probably due to xz
decompression overhead.)

Issue https://github.com/NixOS/nix/issues/1681.
Issue https://github.com/NixOS/nix/issues/1969.
 2018-05-30 13:42:29 +02:00
Eelco Dolstra
08ec757726
Make LocalBinaryCacheStore::narFromPath() run in constant memory 
This reduces memory consumption of

  nix copy --from file://... --to ~/my-nix /nix/store/95cwv4q54dc6giaqv6q6p4r02ia2km35-blender-2.79

from 514 MiB to 18 MiB for an uncompressed binary cache, and from 192
MiB to 53 MiB for a bzipped binary cache. It may also be faster
because fetching can happen concurrently with decompression/writing.

Continuation of 48662d151b.

Issue https://github.com/NixOS/nix/issues/1681.
 2018-05-30 13:34:57 +02:00
Eelco Dolstra
81ea8bd5ce
Simplify the callback mechanism 2018-05-30 13:34:37 +02:00
Eelco Dolstra
1672bcd230
Move evaluator-specific settings out of libstore 2018-05-30 13:29:50 +02:00
Eelco Dolstra
c1d445ecec
Move some Download-specific settings to download.cc 2018-05-30 13:29:50 +02:00
Eelco Dolstra
737ed88f35
Modularize config settings 
Allow global config settings to be defined in multiple Config
classes. For example, this means that libutil can have settings and
evaluator settings can be moved out of libstore. The Config classes
are registered in a new GlobalConfig class to which config files
etc. are applied.

Relevant to https://github.com/NixOS/nix/issues/2009 in that it
removes the need for ad hoc handling of useCaseHack, which was the
underlying cause of that issue.
 2018-05-30 13:28:01 +02:00
Eelco Dolstra
e606cd412f
Fix assertion failure in storePathToHash() 
Fixes https://github.com/NixOS/nix/issues/2015.
 2018-05-30 13:27:19 +02:00
Eelco Dolstra
6185d25e52
Make 'nix copy --to daemon' run in constant memory (daemon side) 
Continuation of 97002b684c. This makes
the daemon use constant memory. For example, it reduces the daemon's
maximum RSS on

  $ nix copy --from ~/my-nix --to daemon /nix/store/1n7x0yv8vq6zi90hfmian84vdhd04bgp-blender-2.79a

from 264 MiB to 7 MiB.

We now use a TunnelSource to prevent the connection from ending up in
an undefined state if an exception is thrown while the NAR is being
sent.

Issue https://github.com/NixOS/nix/issues/1681.
 2018-05-30 13:26:48 +02:00
Eelco Dolstra
23d6bb583a
LocalStore::addToStore: Throw an exception if no hash is given 2018-05-30 13:25:40 +02:00
Eelco Dolstra
3cab639e21
Use extended SQLite error codes 2018-05-30 13:25:03 +02:00
Will Dietz
ca9c6cb95d libstore/download: remove unused function "getTime()" 2018-05-25 19:49:45 -05:00
Eelco Dolstra
966407bcf1
Merge pull request #2145 from serokell/uncached-channel 
make sure not to use cached channels for nix-channel --update
 2018-05-15 14:57:18 +02:00
Eelco Dolstra
2b3c1b3a88
Remove mention of default since we already know the default 2018-05-10 12:06:28 +02:00
Eelco Dolstra
38def17627
Merge branch 's3-multipart-uploads' of https://github.com/AmineChikhaoui/nix 2018-05-10 12:06:02 +02:00
Yorick van Pelt
b9289e4855
make sure not to use cached channels for nix-channel --update 
fixes #1964
 2018-05-09 16:18:20 +02:00
AmineChikhaoui
854c0860f4
share the executor between multiple copy threads. 2018-05-07 15:07:00 +01:00
AmineChikhaoui
591e75cd01
add a FIXME note to find a way to abort the multipart uploads in 
case the nix command is interrupted.
 2018-05-07 14:27:53 +01:00
AmineChikhaoui
129394fb95
Support multi-part uploads for large NARs that exceed the size of 
5Gb.
 2018-05-07 14:23:51 +01:00
Eelco Dolstra
c717d8e3bd
Don't set a CA assertion for paths with references 
Really fixes #2133.
 2018-05-03 14:49:56 +02:00
Eelco Dolstra
80a7b16593
LocalStore::addValidPath(): Throw error when the CA assertion is wrong 
Closes #2133.
 2018-05-03 11:02:40 +02:00
Eelco Dolstra
53ec5ac69f
Fix some random -Wconversion warnings 2018-05-02 13:56:34 +02:00
Will Dietz
8e7d77d494 globals.hh: don't use '==' to compare string literals 
Saw this in logs, also reported here:

8e6108ff71 (r28707288)
 2018-04-23 20:57:05 -05:00
Shea Levy
8e6108ff71
Merge branch 'aarch64-armv7' of git://github.com/lheckemann/nix 
Support extra compatible architectures (#1916)
 2018-04-23 08:48:22 -04:00
Shea Levy
58f9ae0dcd
Merge branch 'hash-mismatch' of git://github.com/LnL7/nix 2018-04-23 08:40:34 -04:00