Commit graph

7179 commits

Author SHA1 Message Date
Eelco Dolstra
96051dd057
More progress indicator improvements
Fixes #1599.
2017-10-24 14:47:23 +02:00
Eelco Dolstra
be220702a7
Progress indicator: Show on what machine we're building
E.g.

  $ nix build nixpkgs.hello --builders 'root@wendy'
  [1/0/1 built] building hello-2.10 on ssh://root@wendy: checking for minix/config.h... no
2017-10-24 14:24:57 +02:00
Eelco Dolstra
f8624762ac
build-remote: Work properly on a chrooted store 2017-10-24 14:14:28 +02:00
Eelco Dolstra
fe9d2f974d
Handle log messages from build-remote
This makes the progress indicator show statuses like "connecting to
'root@machine'".
2017-10-24 14:05:22 +02:00
Eelco Dolstra
0d59f1ca49
nix: Respect -I, --arg, --argstr
Also, random cleanup to argument handling.
2017-10-24 12:58:34 +02:00
Eelco Dolstra
25f32625e2
Remove the remote-builds option
This is superfluous since you can now just set "builders" to empty,
e.g. "--builders ''".
2017-10-24 11:00:16 +02:00
Eelco Dolstra
af241ae7d3
Remove the builder-files option
You can now include files via the "builders" option, using the syntax
"@<filename>". Having only one option makes it easier to override
builders completely.

For backward compatibility, the default is "@/etc/nix/machines", or
"@<filename>" for each file name in NIX_REMOTE_SYSTEMS.
2017-10-24 10:54:43 +02:00
Eelco Dolstra
d4609bb3af
NIX_BUILD_HOOK variable is gone 2017-10-23 20:50:28 +02:00
Eelco Dolstra
37fbfffd8e
Pass all settings to build-remote
This ensures that command line flags such as --builders get passed
correctly.
2017-10-23 20:50:28 +02:00
Eelco Dolstra
f32cdc4fab
Turn $NIX_REMOTE into a configuration option 2017-10-23 20:50:28 +02:00
Eelco Dolstra
38dcd0c4b8
Release note updates 2017-10-23 20:50:28 +02:00
Eelco Dolstra
863a45f1f3
build-remote: Put current load under the store state directory
Fixes the error

  error: opening lock file '/nix/var/nix/current-load/main-lock': Permission denied

when using a chroot store.
2017-10-23 20:50:24 +02:00
Eelco Dolstra
f31c66d392 Merge pull request #1616 from copumpkin/separate-darwin-sandbox
Shift Darwin sandbox to separate installed files
2017-10-20 12:00:27 +02:00
Eelco Dolstra
311b59001a Merge pull request #1622 from kini/c++14-prerequisite
Mention C++14 dependency in the manual.
2017-10-20 11:35:14 +02:00
Keshav Kini
32940702fc Mention C++14 dependency in the manual.
A couple makefiles in the sources have -std=c++14 in the CFLAGS.
2017-10-19 18:03:26 -07:00
Shea Levy
c7af84ce84
nix-shell: Fix file name resolution in shebangs 2017-10-19 15:02:40 -04:00
Eelco Dolstra
ef1d0142a0
Revert "Let's not populate the NAR cache from hydra-queue-runner for now"
This reverts commit 908590dc6c. Since
hydra-server can have a different store URI from hydra-queue-runner
now, we don't really need this.
2017-10-19 12:11:07 +02:00
Eelco Dolstra
908590dc6c
Let's not populate the NAR cache from hydra-queue-runner for now 2017-10-18 15:45:39 +02:00
Eelco Dolstra
75cd75b1ae
Errors writing to the NAR cache should not be fatal 2017-10-18 15:41:14 +02:00
Eelco Dolstra
d8306148e0
Suppress "copying 0 paths" message 2017-10-18 15:02:58 +02:00
Dan Peebles
6e5165b773 Shift Darwin sandbox to separate installed files
This makes it slightly more manageable to see at a glance what in a
build's sandbox profile is unique to the build and what is standard. Also
a first step to factoring more of our Darwin logic into scheme functions
that will allow us a bit more flexibility. And of course less of that
nasty codegen in C++! 😀
2017-10-17 19:15:51 -04:00
Eelco Dolstra
b24b8ef77c BinaryCacheStore::addToStore(): Add NARs to the local cache 2017-10-17 21:39:48 +02:00
Eelco Dolstra
ca580bec35 BinaryCacheStore: Support local caching of NARs
This speeds up commands like "nix cat-store". For example:

  $ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nar-cache /nix/store/i60yncmq6w9dyv37zd2k454g0fkl3arl-systemd-234/etc/udev/udev.conf
  real    0m4.336s

  $ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nar-cache /nix/store/i60yncmq6w9dyv37zd2k454g0fkl3arl-systemd-234/etc/udev/udev.conf
  real    0m0.045s

The primary motivation is to allow hydra-server to serve files from S3
binary caches. Previously Hydra had a hack to do "nix-store -r
<path>", but that fetches the entire closure so is prohibitively
expensive.

There is no garbage collection of the NAR cache yet. Also, the entire
NAR is read when accessing a single member file. We could generate the
NAR listing to provide random access.

Note: the NAR cache is indexed by the store path hash, not the content
hash, so NAR caches should not be shared between binary caches, unless
you're sure that all your builds are binary-reproducible.
2017-10-17 21:15:33 +02:00
Eelco Dolstra
11ba4302e3 Remove BinaryCacheStoreAccessor
Probably as a result of a bad merge in
4b8f1b0ec0, we had both a
BinaryCacheStoreAccessor and a
RemoteFSAccessor. BinaryCacheStore::getFSAccessor() returned the
latter, but BinaryCacheStore::addToStore() checked for the
former. This probably caused hydra-queue-runner to download paths that
it just uploaded.
2017-10-17 20:51:42 +02:00
Shea Levy
cbc216911d
fetchgit: Remove incomplete/unneeded ref check.
This check fails for tags and branches, and is made redundant by the
checks git itself will do when fetching the repo.
2017-10-17 12:30:54 -04:00
Shea Levy
8522db1641
Merge branch 'nix-profile-daemon' 2017-10-16 18:17:51 -04:00
Eelco Dolstra
be59f07799 Merge pull request #1601 from shlevy/fetchgit-no-uri-check
fetchgit: Remove incomplete/unneeded isURI check.
2017-10-16 21:22:13 +02:00
Shea Levy
6a037a738a
Pull nix-profile-daemon from 1.11 2017-10-16 14:51:39 -04:00
Shea Levy
4e58294ae6
fetchgit: Remove incomplete/unneeded isURI check.
This check spuriously fails for e.g. git@github.com:NixOS/nixpkgs.git,
and even for ssh://git@github.com/NixOS/nixpkgs.git, and is made
redundant by the checks git itself will do when fetching the repo. We
instead pass a -- before passing the URI to git to avoid injection.
2017-10-16 14:35:28 -04:00
Eelco Dolstra
1dd29d7aeb
Add option to disable the seccomp filter
I needed this to test ACL/xattr removal in
canonicalisePathMetaData(). Might also be useful if you need to build
old Nixpkgs that doesn't have the required patches to remove
setuid/setgid creation.
2017-10-12 18:21:55 +02:00
Eelco Dolstra
97307811ee Merge pull request #1571 from teto/improve_priority_msg
Improve error message for conflicting priorities
2017-10-12 13:10:44 +02:00
Eelco Dolstra
73252aef18 Merge pull request #1591 from shlevy/darwin-installer-no-sudo-i
darwin installer: Fix on systems where sudo -i is disabled.
2017-10-12 13:08:15 +02:00
Eelco Dolstra
29205e0218 Merge pull request #1586 from acowley/patch-2
install-darwin-multi-user: relax assumption check
2017-10-12 13:06:25 +02:00
Eelco Dolstra
177aee07c6 Merge pull request #1580 from copumpkin/retry-after-timeout
Retry in all error cases but a few
2017-10-12 13:05:45 +02:00
Eelco Dolstra
3d3c7fadf7 Merge pull request #1595 from peti/master
docker: ensure that the installation works for users other than 'root'
2017-10-12 13:02:19 +02:00
Eelco Dolstra
fda7b95cb0
Fix a hang in ThreadPool
The worker threads could exit prematurely if they finished processing
all items while the main thread was still adding items. In particular,
this caused hanging nix-store --serve processes in the build farm.

Also, process items from the main thread.
2017-10-09 15:41:09 +02:00
Eelco Dolstra
838509d1a0
Whitespace 2017-10-09 15:41:09 +02:00
Eelco Dolstra
1893f78f8e Merge pull request #1594 from dtzWill/fix/busybox-builtins
release-common: Fix busybox builtins (busybox >= 1.27)
2017-10-09 12:09:33 +02:00
Peter Simons
37164ae389 docker: ensure that the installation works for users other than 'root', too
The path /root/.nix-profile is a sym-link to /nix/var/nix/profiles/default. The
latter path, however, works for everyone while the former path works only for
root, so we prefer the public path whenever possible.
2017-10-07 17:28:34 +02:00
Will Dietz
2f1a1c5a49 release-common: Fix busybox builtins (busybox >= 1.27)
See https://github.com/NixOS/nixpkgs/pull/28261
2017-10-07 07:43:55 -05:00
Eelco Dolstra
b2e00fd5b7 Merge pull request #1593 from Mic92/bashism
fixing bashisms in test code
2017-10-06 14:58:13 +02:00
Jörg Thalheim
e94fc238cf fixing bashisms in test code
This fixed the build on ubuntu/debian, where dash is the sh.
2017-10-06 06:12:33 -05:00
Shea Levy
fb98e29067 darwin installer: Fix on systems where sudo -i is disabled. 2017-10-05 09:07:55 -07:00
Eelco Dolstra
c6676ea253
Merge branch 'elegant-tests' of https://github.com/copumpkin/nix 2017-10-03 11:16:34 +02:00
Eelco Dolstra
fce24b7d6c Merge pull request #1587 from copumpkin/fix-tests
Fix tests
2017-10-03 11:06:03 +02:00
Dan Peebles
60ecbd7934 More elegant test output
I got sick of trying to find the failures in the sea of debug output, so
we now:
- Hide test output unless it fails
- Sprinkle in some simple color
- Pad results for a more tabular look

If Nix is getting a more friendly user interface, we might as well get
a friendlier developer interface, right? :)
2017-10-03 00:59:32 -04:00
Dan Peebles
3139cad9cd Fix tests
`nix copy` no longer accepts a `--recursive` argument
2017-10-02 23:45:27 -04:00
Dan Peebles
476493dbf5 Reverse retry logic to retry in all but a few cases
It was getting too much like whac-a-mole listing all the retriable error
conditions, so we now retry by default and list the cases where retrying
is almost certainly hopeless.
2017-10-02 23:22:02 -04:00
Anthony Cowley
92f9d18aa0 install-darwin-multi-user: relax assumption check
The installer will error out if a user's shell configuration includes any mention of ~nix-profile~, even if this is in a comment. This change is designed to do the bare minimum to ignore lines beginning with a `#`.
2017-10-02 20:07:56 -04:00
Eelco Dolstra
f3e0d46821
nix run: Restore signals
Otherwise Ctrl-C doesn't work.
2017-09-28 17:58:59 +02:00