max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2024-11-11 00:36:20 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
6091 commits 7 branches 0 tags 64 MiB
Commit graph

102 commits

Author SHA1 Message Date
Will Dietz
74da813912 download: improve error for hash mismatch ("store mismatch") 
Fixes #1905
 2018-03-22 15:22:06 -05:00
Will Dietz
59b32403f2 download.cc: fix path for cached files 2018-03-22 15:05:46 -05:00
Eelco Dolstra
eb75bc5afb
Don't retry on CURLE_SSL_CACERT_BADFILE 
The certificates won't get any better if we retry.
 2018-03-16 12:03:18 +01:00
Shea Levy
cfdbfa6b2c
Catch more possible instances of passing NULL to memcpy. 
Actually fixes #1976.
 2018-03-14 23:53:43 -04:00
Eelco Dolstra
64e486ab63
Fix downloadCached() with a chroot store 
E.g.

  nix run --store ~/my-nix -f channel:nixos-17.03 hello -c hello

This problem was mentioned in #1897.
 2018-02-23 13:00:42 +01:00
Eelco Dolstra
88f4f0231b Fix building without aws-sdk-cpp 2018-01-31 22:34:51 +01:00
Eelco Dolstra
6270b2e50f
Merge branch 'http-binary-cache-put-upsert' of https://github.com/adelbertc/nix 2018-01-31 15:10:12 +01:00
Eelco Dolstra
478e3e4649
Indent properly 2018-01-31 13:46:38 +01:00
Shea Levy
1d5d277ac7
HttpBinaryCacheStore: Support upsertFile with PUT. 
Some servers, such as Artifactory, allow uploading with PUT and BASIC
auth. This allows nix copy to work to upload binaries to those
servers.

Worked on together with @adelbertc
 2018-01-26 11:12:30 -08:00
Eelco Dolstra
2896bb6826
Don't retry CURLE_URL_MALFORMAT 2018-01-19 14:05:08 +01:00
Benjamin Hipple
d76c4fe770 Fix for builds with system libcurl < 7.30 
CentOS 7.4 and RHEL 7.4 ship with libcurl-devel-7.29.0-42.el7.x86_64; this flag
was added in 7.30.0
https://curl.haxx.se/libcurl/c/CURLMOPT_MAX_TOTAL_CONNECTIONS.html
 2017-12-09 20:02:21 -05:00
Eelco Dolstra
8956ae1987
Add a "profile" option to S3BinaryCacheStore 
This allows specifying the AWS configuration profile to use. E.g.

  nix copy --from s3://my-cache?profile=aws-dev-account /nix/store/cf3isrlqavvd5w7rpky1fa8j9lcnlggm-...
 2017-11-15 14:18:20 +01:00
Eelco Dolstra
1969f357b7
Add fetchMercurial primop 
E.g.

  $ nix eval '(fetchMercurial https://www.mercurial-scm.org/repo/hello)'
  { branch = "default"; outPath = "/nix/store/alvb9y1kfz42bjishqmyy3pphnrh1pfa-source"; rev = "82e55d328c8ca4ee16520036c0aaace03a5beb65"; revCount = 1; shortRev = "82e55d328c8c"; }

  $ nix eval '(fetchMercurial { url = https://www.mercurial-scm.org/repo/hello; rev = "0a04b987be5ae354b710cefeba0e2d9de7ad41a9"; })'
  { branch = "default"; outPath = "/nix/store/alvb9y1kfz42bjishqmyy3pphnrh1pfa-source"; rev = "0a04b987be5ae354b710cefeba0e2d9de7ad41a9"; revCount = 0; shortRev = "0a04b987be5a"; }

  $ nix eval '(fetchMercurial /tmp/unclean-hg-tree)'
  { branch = "default"; outPath = "/nix/store/cm750cdw1x8wfpm3jq7mz09r30l9r024-source"; rev = "0000000000000000000000000000000000000000"; revCount = 0; shortRev = "000000000000"; }
 2017-11-01 17:45:32 +01:00
Eelco Dolstra
66ddbef754
fetchurl/fetchTarball: Respect name changes 
The computation of urlHash didn't take the name into account, so
subsequent fetchurl calls with the same URL but a different name would
resolve to the same cached store path.
 2017-10-30 10:22:58 +01:00
Dan Peebles
476493dbf5 Reverse retry logic to retry in all but a few cases 
It was getting too much like whac-a-mole listing all the retriable error
conditions, so we now retry by default and list the cases where retrying
is almost certainly hopeless.
 2017-10-02 23:22:02 -04:00
Eelco Dolstra
aca4f7dff0
Don't remove Content-Encoding in fetchurl / nix-prefetch-url 
Fixes #1568.
 2017-09-18 11:07:28 +02:00
Eelco Dolstra
8215b75d36
Abort curl downloads if there is no progress for 5 minutes 
Maybe this will fix the curl hangs on macOS. (We could also use
CURLOPT_TIMEOUT but that seems more of a sledgehammer.)
 2017-09-01 16:51:26 +02:00
Eelco Dolstra
fd73c1e20a
Add an activity for binary cache queries 2017-08-31 15:25:58 +02:00
Eelco Dolstra
2cc345b95f
Give activities a verbosity level again 
And print them (separately from the progress bar) given sufficient -v
flags.
 2017-08-28 19:13:24 +02:00
Eelco Dolstra
c137c0a5eb
Allow activities to be nested 
In particular, this allows more relevant activities ("substituting X")
to supersede inferior ones ("downloading X").
 2017-08-25 17:49:40 +02:00
Eelco Dolstra
40bffe0a43
Progress indicator: Cleanup 2017-08-16 20:56:03 +02:00
Eelco Dolstra
c36467ad2e
Improve substitution progress indicator 
E.g.

  $ nix build --store local?root=/tmp/nix nixpkgs.firefox-unwrapped
  [0/1 built, 1/97/98 fetched, 65.8/92.8 MiB DL, 203.2/309.2 MiB copied] downloading 'https://cache.nixos.org/nar/1czm9fk0svacy4h6a3fzkpafi4f7a9gml36kk8cq1igaghbspg3k.nar.xz'
 2017-08-16 20:56:02 +02:00
Jörg Thalheim
2fd8f8bb99 Replace Unicode quotes in user-facing strings by ASCII 
Relevant RFC: NixOS/rfcs#4

$ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
 2017-07-30 12:32:45 +01:00
Eelco Dolstra
8e8caf7f3e fetchTarball: Prevent concurrent downloads of the same file 
Fixes #849.
 2017-07-11 23:21:24 +02:00
Eelco Dolstra
c0015e87af
Support base-64 hashes 
Also simplify the Hash API.

Fixes #1437.
 2017-07-04 15:07:41 +02:00
Eelco Dolstra
fcca702a96
Replace a few bool flags with enums 
Functions like copyClosure() had 3 bool arguments, which creates a
severe risk of mixing up arguments.

Also, implement copyClosure() using copyPaths().
 2017-07-03 11:38:08 +02:00
Eelco Dolstra
847f19a5f7
Provide a builtin default for $NIX_SSL_CERT_FILE 
This is mostly to ensure that when Nix is started on macOS via a
launchd service or sshd (for a remote build), it gets a certificate
bundle.
 2017-06-12 16:44:43 +02:00
Eelco Dolstra
b01d62285c
Improve progress indicator 2017-05-16 16:09:57 +02:00
Eelco Dolstra
510bc1735b
Add an option for extending the user agent header 
This is useful e.g. for distinguishing traffic to a binary cache
(e.g. certain machines can use a different tag in the user agent).
 2017-05-11 15:10:22 +02:00
Eelco Dolstra
ea65ae0f9c
Tweak error message 2017-05-11 13:59:47 +02:00
Eelco Dolstra
45d7b1a9e9
LocalStore::addToStore(): Check info.narSize 
It allowed the client to specify bogus narSize values. In particular,
Downloader::downloadCached wasn't setting narSize at all.
 2017-05-11 13:26:03 +02:00
Eelco Dolstra
9cc8047f44
Reimplement connect-timeout 
Fixes #1339.
 2017-04-19 14:54:52 +02:00
Eelco Dolstra
f8a2e8a552
Shut up some warnings 2017-04-14 14:42:20 +02:00
Eelco Dolstra
ba9ad29fdb
Convert Settings to the new config system 
This makes all config options self-documenting.

Unknown or unparseable config settings and --option flags now cause a
warning.
 2017-04-13 20:53:23 +02:00
Eelco Dolstra
1860070548
Merge branch 'rework-options' of https://github.com/copumpkin/nix 2017-04-13 16:15:51 +02:00
Dan Peebles
d1fdade755 Add CURLE_WRITE_ERROR as a transient error condition 
We've observed it failing downloads in the wild and retrying the same URL
a few moments later seemed to fix it.
 2017-04-10 09:28:44 -04:00
Dan Peebles
98283915f5 Retry downloads on transient SSL errors too 2017-04-06 18:18:43 +00:00
Eelco Dolstra
e1509adbbb
Retry curl error 16 2017-03-31 15:50:11 +02:00
Eelco Dolstra
25dff2b7db
runProgram(): Distinguish between empty input and no input 
For example, if we call brotli with an empty input, it shouldn't read
from the caller's stdin.
 2017-03-15 16:50:19 +01:00
Eelco Dolstra
8b1d65bebe
S3BinaryCacheStore: Support compression of narinfo and log files 
You can now set the store parameter "text-compression=br" to compress
textual files in the binary cache (i.e. narinfo and logs) using
Brotli. This sets the Content-Encoding header; the extension of
compressed files is unchanged.

You can separately specify the compression of log files using
"log-compression=br". This is useful when you don't want to compress
narinfo files for backward compatibility.
 2017-03-15 16:49:28 +01:00
Eelco Dolstra
e8186085e0
Add support for brotli compression 
Build logs on cache.nixos.org are compressed using Brotli (since this
allows them to be decompressed automatically by Chrome and Firefox),
so it's handy if "nix log" can decompress them.
 2017-03-15 16:49:06 +01:00
Shea Levy
96443e94a1 Merge branch 'configurable-aws-region' 2017-03-12 10:01:11 -04:00
Shea Levy
3cc18d3753 Properly set the caFile for aws-sdk-cpp s3 2017-03-06 14:30:35 -05:00
Shea Levy
5789eaa3f4 Add aws-region param to S3 store URLs 2017-03-03 16:12:17 -05:00
Eelco Dolstra
0780805246
Fix building against older curl versions 
http://hydra.nixos.org/build/49490928
 2017-02-28 14:03:43 +01:00
Eelco Dolstra
3cb0387d3f
Retry downloads on HTTP/2 stream errors 
Issue #1254.
 2017-02-27 14:36:09 +01:00
Dan Peebles
e7cb2847ab Explicitly model all settings and fail on unrecognized ones 
Previously, the Settings class allowed other code to query for string
properties, which led to a proliferation of code all over the place making
up new options without any sort of central registry of valid options. This
commit pulls all those options back into the central Settings class and
removes the public get() methods, to discourage future abuses like that.

Furthermore, because we know the full set of options ahead of time, we
now fail loudly if someone enters an unrecognized option, thus preventing
subtle typos. With some template fun, we could probably also dump the full
set of options (with documentation, defaults, etc.) to the command line,
but I'm not doing that yet here.
 2017-02-22 20:19:25 -05:00
Eelco Dolstra
f023f64f40
RemoteStore::addToStore(): Pass content-addressability assertion 
... and use this in Downloader::downloadCached(). This fixes

  $ nix-build https://nixos.org/channels/nixos-16.09-small/nixexprs.tar.xz -A hello
  error: cannot import path ‘/nix/store/csfbp1s60dkgmk9f8g0zk0mwb7hzgabd-nixexprs.tar.xz’ because it lacks a valid signature
 2017-02-22 16:58:00 +01:00
Eelco Dolstra
8b1b5f9a12
Handle CURLE_RECV_ERROR as a transient error 
This fixes

  unable to download ‘https://cache.nixos.org/nar/077h8ji74y9b0qx7rjk71xd80vjqp6q5gy137r553jlvdlxdcdlk.nar.xz’: HTTP error 200 (curl error: Failure when receiving data from the peer)
 2017-02-21 16:04:47 +01:00
Eelco Dolstra
99bbddedb1
Fix building without S3 support 
http://hydra.nixos.org/build/49031196/nixlog/2/raw
 2017-02-21 13:15:07 +01:00