Commit graph

13211 commits

Author SHA1 Message Date
Shea Levy
3cc18d3753 Properly set the caFile for aws-sdk-cpp s3 2017-03-06 14:30:35 -05:00
Shea Levy
4fc30922cf istringstream_nocopy: Implement in a standards-compliant way.
Fixes the problem mentioned in e6a61b8da7

See #1135
2017-03-06 13:03:02 -05:00
Eelco Dolstra
1cf4801108
Revert fa125b9b28
This causes quadratic performance.
2017-03-06 13:56:20 +01:00
Shea Levy
689b825627 nix-daemon.plist: Set XDG_CACHE_HOME 2017-03-05 18:15:44 -05:00
Shea Levy
b667abc699 Add signing and s3 support on darwin 2017-03-05 07:39:10 -05:00
Shea Levy
5789eaa3f4 Add aws-region param to S3 store URLs 2017-03-03 16:12:17 -05:00
Eelco Dolstra
d1158bb816
Cache connection failures 2017-03-03 19:36:01 +01:00
Eelco Dolstra
8490ee37a6
SSHMaster: Make thread-safe 2017-03-03 19:28:27 +01:00
Eelco Dolstra
d3eb1cf3bb
build-remote: Don't use a SSH master
This is unnecessary because we make only one connection.
2017-03-03 19:23:20 +01:00
Eelco Dolstra
d140c75530
Pool: Don't hang if creating a connection fails 2017-03-03 19:21:43 +01:00
Eelco Dolstra
577ebeaefb
Improve SSH handling
* Unify SSH code in SSHStore and LegacySSHStore.

* Fix a race starting the SSH master. We now wait synchronously for
  the SSH master to finish starting. This prevents the SSH clients
  from starting their own connections.

* Don't use a master if max-connections == 1.

* Add a "max-connections" store parameter.

* Add a "compress" store parameter.
2017-03-03 19:05:50 +01:00
Eelco Dolstra
7f62be1bcd
build-remote: Fix passing SSH key 2017-03-03 16:33:18 +01:00
Eelco Dolstra
5a1fb03b8f
build-remote: Misc cleanup 2017-03-03 16:18:49 +01:00
Eelco Dolstra
2f992692e2
Fix fatal "broken pipe" error when $NIX_BUILD_HOOK is missing 2017-03-03 15:40:06 +01:00
Eelco Dolstra
ffcf9d24a6
builtins.fetchgit: Support paths 2017-03-02 11:46:28 +01:00
Eelco Dolstra
fbf17f1ad7
builtins.fetchgit: Fix bad format string 2017-03-02 11:40:11 +01:00
Eelco Dolstra
fa125b9b28
TeeSink: Pre-reserve string space
When receiving a very large file, this can prevent the string from
having tobe copied, which temporarily doubles memory consumption.
2017-03-01 16:16:04 +01:00
Eelco Dolstra
f61f67ddee
RemoteStore::addToStore(): Send NAR rather than string containing NAR
This allows the NAR to be streamed in the future (though we're not
doing that yet).
2017-03-01 16:07:15 +01:00
Eelco Dolstra
374908726b
readString(): Read directly into std::string
When reading a huge string, this halves memory consumption.

(Strictly speaking, this appears only valid in C++17, but who cares...)
2017-03-01 14:54:11 +01:00
Eelco Dolstra
07a0b8ca67
Tweak message 2017-03-01 14:52:16 +01:00
Eelco Dolstra
56e19d970d
nix-store --import: Fix importing unsigned paths 2017-03-01 14:47:52 +01:00
Eelco Dolstra
e321551d54
Fix assertion failure in nix-store --export
Fixes #1173.
2017-03-01 14:41:54 +01:00
Eelco Dolstra
c4a40949d9
Handle importing NARs containing files greater than 4 GiB
Also templatize readInt() to work for various integer types.
2017-03-01 13:52:54 +01:00
Eelco Dolstra
0780805246
Fix building against older curl versions
http://hydra.nixos.org/build/49490928
2017-02-28 14:03:43 +01:00
Eelco Dolstra
fd86dd93dd
Improve SQLite busy handling 2017-02-28 13:59:11 +01:00
Eelco Dolstra
34b12bad59
NarInfoDiskCache: Handle SQLite busy errors 2017-02-28 13:44:11 +01:00
Eelco Dolstra
80027144ae
In SQLite errors, include the database path
This is necessary because we have multiple SQLite databases (e.g. the
binary cache cache).
2017-02-28 13:20:12 +01:00
Eelco Dolstra
7251d048fa
Support auto-configuration of build-max-jobs
"build-max-jobs" and the "-j" option can now be set to "auto" to use
the number of CPUs in the system. (Unlike build-cores, it doesn't use
0 to imply auto-configuration, because a) magic values are a bad idea
in general; b) 0 is a legitimate value used to disable local
building.)

Fixes #1198.
2017-02-28 12:54:50 +01:00
Eelco Dolstra
3fab1f04a7
_SC_NPROCESSORS_ONLN -> std:🧵:hardware_concurrency() 2017-02-27 16:01:54 +01:00
Eelco Dolstra
4f3fb34844
Fix 1.8 release note about build-max-jobs
We set build-cores automatically, not build-max-jobs. (The commit
message for de4cdd0d47 also got this
wrong.)
2017-02-27 15:58:24 +01:00
Eelco Dolstra
3cb0387d3f
Retry downloads on HTTP/2 stream errors
Issue #1254.
2017-02-27 14:36:09 +01:00
Domen Kožar
f0bbd153c6 Merge pull request #1252 from bjornfor/channel-rename-fix
unpack-channel.nix: fix 'mv' corner case
2017-02-27 13:12:35 +01:00
Bjørn Forsman
f72206b736 unpack-channel.nix: fix 'mv' corner case
unpack-channel.nix fails if the tarball contains a directory named the
same as the channel:

  mv: cannot move 'nixpkgs' to a subdirectory of itself, '.../nixpkgs'

This commit fixes that by not moving the directory if it already has the
correct name.
2017-02-27 10:12:58 +01:00
Eelco Dolstra
89ffe1eff9
Fix nix-shell tests
The nix-shell fix in 668fef2e4f revealed
that we had some --pure tests that incorrectly depended on PATH from
config.nix's mkDerivation being overwritten by the caller's PATH.

http://hydra.nixos.org/build/49242478
2017-02-24 17:29:02 +01:00
Eelco Dolstra
d3e1aad421
nix-shell: Better error message when the shell can't be started 2017-02-24 17:25:00 +01:00
Eelco Dolstra
e76df9bd52
Register content-addressability assertion for fixed outputs 2017-02-24 16:39:53 +01:00
Eelco Dolstra
1c718f80d3
Verify content-addressability assertions at registration time 2017-02-24 16:38:14 +01:00
Eelco Dolstra
668fef2e4f
nix-shell: Overwrite environment variables
Need to remember that std::map::insert() and emplace() don't overwrite
existing entries...

This fixes a regression relative to 1.11 that in particular triggers
in nested nix-shells.

Before:

  $ nativeBuildInputs=/foo nix-shell -p hello --run 'hello'
  build input /foo does not exist

After:

  $ nativeBuildInputs=/foo nix-shell -p hello --run 'hello'
  Hello, world!
2017-02-24 13:31:46 +01:00
Dan Peebles
e7cb2847ab Explicitly model all settings and fail on unrecognized ones
Previously, the Settings class allowed other code to query for string
properties, which led to a proliferation of code all over the place making
up new options without any sort of central registry of valid options. This
commit pulls all those options back into the central Settings class and
removes the public get() methods, to discourage future abuses like that.

Furthermore, because we know the full set of options ahead of time, we
now fail loudly if someone enters an unrecognized option, thus preventing
subtle typos. With some template fun, we could probably also dump the full
set of options (with documentation, defaults, etc.) to the command line,
but I'm not doing that yet here.
2017-02-22 20:19:25 -05:00
Eelco Dolstra
f023f64f40
RemoteStore::addToStore(): Pass content-addressability assertion
... and use this in Downloader::downloadCached(). This fixes

  $ nix-build https://nixos.org/channels/nixos-16.09-small/nixexprs.tar.xz -A hello
  error: cannot import path ‘/nix/store/csfbp1s60dkgmk9f8g0zk0mwb7hzgabd-nixexprs.tar.xz’ because it lacks a valid signature
2017-02-22 16:58:00 +01:00
Eelco Dolstra
fe2db1dae5
Doh 2017-02-22 15:39:17 +01:00
Eelco Dolstra
b8ce649a35
Fix 32-bit RPM/Deb builds
http://hydra.nixos.org/build/49130529
2017-02-22 13:54:11 +01:00
Eelco Dolstra
8b1b5f9a12
Handle CURLE_RECV_ERROR as a transient error
This fixes

  unable to download ‘https://cache.nixos.org/nar/077h8ji74y9b0qx7rjk71xd80vjqp6q5gy137r553jlvdlxdcdlk.nar.xz’: HTTP error 200 (curl error: Failure when receiving data from the peer)
2017-02-21 16:04:47 +01:00
Eelco Dolstra
8d7c6644c5
useChroot -> useSandbox 2017-02-21 15:21:56 +01:00
Eelco Dolstra
1a57f499b0
Drop some Ubuntu releases 2017-02-21 15:20:40 +01:00
Eelco Dolstra
c0a133876e
Revert "configure.ac: We require C++14 now"
This reverts commit 81c53fe8e5. This
check appears to be stricter than we need (it broke a bunch of
platforms that previously did build:
http://hydra.nixos.org/eval/1331921#tabs-now-fail).
2017-02-21 15:03:32 +01:00
Eelco Dolstra
b95ce3194d
Debian build: Use parallel make and add Ubuntu 16.10 2017-02-21 15:03:23 +01:00
Eelco Dolstra
e4dd7dadf4
RPM build: Use parallel make 2017-02-21 14:52:36 +01:00
Eelco Dolstra
bb6656b8a2
Build RPMs for Fedora 25
Disabled hardened build because it makes the linker fail with messages like

  relocation R_X86_64_PC32 against undefined symbol `BZ2_bzWriteOpen' can not be used when making a shared object; recompile with -fPIC

See https://fedoraproject.org/wiki/Changes/Harden_All_Packages.
2017-02-21 14:26:23 +01:00
Eelco Dolstra
99bbddedb1
Fix building without S3 support
http://hydra.nixos.org/build/49031196/nixlog/2/raw
2017-02-21 13:15:07 +01:00