Commit graph

3997 commits

Author SHA1 Message Date
Eelco Dolstra
01ddf53ee9 tests/remote-builds.nix: Test failing build 2014-07-24 12:10:06 +02:00
Eelco Dolstra
d7a92ed4a9 nix-store --serve: Only monitor stdin during builds
Other operations cannot hang indefinitely (except when we're reading
from stdin, in which case we'll notice a client disconnect). But
monitoring works badly during compressed imports, since there the
client can close the connection before we've sent an ack.

http://hydra.nixos.org/build/12711638
2014-07-24 11:59:35 +02:00
Eelco Dolstra
a6ab3cf35f tests/remote-builds.nix: Don't try cache.nixos.org 2014-07-24 11:50:05 +02:00
Eelco Dolstra
0fae20c362 Use pthread_cancel instead of a signal
Signal handlers are process-wide, so sending SIGINT to the monitor
thread will cause the normal SIGINT handler to run. This sets the
isInterrupted flag, which is not what we want. So use pthread_cancel
instead.
2014-07-24 11:47:51 +02:00
Eelco Dolstra
aa1560ca07 Fix bogus pass by reference
http://hydra.nixos.org/build/12711659
2014-07-24 09:58:07 +02:00
Eelco Dolstra
24c6d992c6 More debugging 2014-07-24 01:21:56 +02:00
Eelco Dolstra
1eb0af7ed5 Add some assertions 2014-07-24 00:16:06 +02:00
Eelco Dolstra
d3c61d83be Remove some dead code 2014-07-24 00:00:53 +02:00
Eelco Dolstra
62ad3dfc43 Remove some obsolete files 2014-07-23 23:56:58 +02:00
Eelco Dolstra
bd91453bb1 Pass -pthread only for programs that need it 2014-07-23 19:48:18 +02:00
Eelco Dolstra
ece531d105 nix-daemon: Less verbosity 2014-07-23 19:43:46 +02:00
Eelco Dolstra
d0fad1779f nix-daemon: Simplify stderr handling 2014-07-23 19:37:40 +02:00
Eelco Dolstra
78224cd871 nix-store --serve: Monitor for client disconnects
This is necessary because build-remote.pl now builds via ‘nix-store
--serve’. So if a build hangs without writing to stdout/stderr, and
the client disconnects, then we need to detect that.
2014-07-23 19:26:34 +02:00
Eelco Dolstra
49fe9592a4 nix-daemon: Use a thread instead of SIGPOLL to catch client disconnects
The thread calls poll() to wait until a HUP (or other error event)
happens on the client connection. If so, it sends SIGINT to the main
thread, which is then cleaned up normally. This is much nicer than
messing around with SIGPOLL.
2014-07-23 19:21:00 +02:00
Eelco Dolstra
fdee1ced43 startProcess: Make writing error messages from the child more robust 2014-07-23 19:11:26 +02:00
Eelco Dolstra
5989966ed3 Remove dead code 2014-07-23 14:46:28 +02:00
Eelco Dolstra
ee3c5d7916 Revert old useBuildHook behaviour 2014-07-19 02:25:47 +02:00
Eelco Dolstra
2e77bd70fa Better fix for strcasecmp on Darwin 2014-07-18 12:54:30 +02:00
Eelco Dolstra
f609eec71a Bump 2014-07-18 00:01:06 +02:00
Eelco Dolstra
8ddffe7aac Ugly hack to fix building on old Darwin
http://hydra.nixos.org/build/12580878
2014-07-17 23:57:17 +02:00
Eelco Dolstra
049c0eb49c nix-daemon: Add trusted-users and allowed-users options
‘trusted-users’ is a list of users and groups that have elevated
rights, such as the ability to specify binary caches. It defaults to
‘root’. A typical value would be ‘@wheel’ to specify all users in the
wheel group.

‘allowed-users’ is a list of users and groups that are allowed to
connect to the daemon. It defaults to ‘*’. A typical value would be
‘@users’ to specify the ‘users’ group.
2014-07-17 16:57:07 +02:00
Eelco Dolstra
0c730887c4 nix-daemon: Show name of connecting user 2014-07-17 15:49:33 +02:00
Eelco Dolstra
77c972c898 nix-daemon: Only print connection info if we have SO_PEERCRED 2014-07-17 15:44:51 +02:00
Eelco Dolstra
8f72e702a1 nix-daemon: Fix compat with older clients 2014-07-17 15:23:31 +02:00
Eelco Dolstra
2304a7dd21 Get rid of a compiler warning 2014-07-16 16:32:26 +02:00
Eelco Dolstra
985f1595fe Be more strict about file names in NARs 2014-07-16 16:30:50 +02:00
Eelco Dolstra
276a40b31f Handle case collisions on case-insensitive systems
When running NixOps under Mac OS X, we need to be able to import store
paths built on Linux into the local Nix store. However, HFS+ is
usually case-insensitive, so if there are directories with file names
that differ only in case, then importing will fail.

The solution is to add a suffix ("~nix~case~hack~<integer>") to
colliding files. For instance, if we have a directory containing
xt_CONNMARK.h and xt_connmark.h, then the latter will be renamed to
"xt_connmark.h~nix~case~hack~1". If a store path is dumped as a NAR,
the suffixes are removed. Thus, importing and exporting via a
case-insensitive Nix store is round-tripping. So when NixOps calls
nix-copy-closure to copy the path to a Linux machine, you get the
original file names back.

Closes #119.
2014-07-16 16:02:05 +02:00
Eelco Dolstra
bb65460feb Make dev-shell script work on Darwin 2014-07-16 11:53:47 +02:00
Eelco Dolstra
de8be7c3e0 Install systemd and Upstart stuff only on Linux 2014-07-16 11:53:47 +02:00
Shea Levy
048be62484 Pass *_proxy vars to bootstrap fetchurl 2014-07-16 11:21:45 +02:00
Eelco Dolstra
a2c85b2ef8 Manual: Typo 2014-07-16 11:21:44 +02:00
Eelco Dolstra
5bcb982711 Remove cruft 2014-07-14 12:39:53 +02:00
Eelco Dolstra
fa13d3f4f3 build-remote.pl: Fix building multiple output derivations
We were importing paths without sorting them topologically, leading to
"path is not valid" errors.

See e.g. http://hydra.nixos.org/build/12451761
2014-07-14 12:19:27 +02:00
Eelco Dolstra
b2e0293f02 build-remote.pl: Don't keep a shell process around 2014-07-12 00:43:28 +02:00
Eelco Dolstra
a00a98548e build-remote.pl: Fix build log 2014-07-12 00:09:43 +02:00
Eelco Dolstra
838138c5c4 Fix test 2014-07-11 16:22:24 +02:00
Eelco Dolstra
a5c6347ff0 build-remote.pl: Use ‘nix-store --serve’ on the remote side
This makes things more efficient (we don't need to use an SSH master
connection, and we only start a single remote process) and gets rid of
locking issues (the remote nix-store process will keep inputs and
outputs locked as long as they're needed).

It also makes it more or less secure to connect directly to the root
account on the build machine, using a forced command
(e.g. ‘command="nix-store --serve --write"’). This bypasses the Nix
daemon and is therefore more efficient.

Also, don't call nix-store to import the output paths.
2014-07-11 16:22:24 +02:00
Eelco Dolstra
b8f24f2535 Fix closure size display 2014-07-11 14:27:17 +02:00
Eelco Dolstra
e196eecbe6 Allow $NIX_BUILD_HOOK to be relative to Nix libexec directory 2014-07-11 13:55:06 +02:00
Eelco Dolstra
d0eb970fb4 Fix broken Pid constructor 2014-07-10 21:48:21 +02:00
Eelco Dolstra
edbfe2232e Replace message "importing path <...>" with "exporting path <...>"
This causes nix-copy-closure to show what it's doing before rather
than after.
2014-07-10 21:30:22 +02:00
Eelco Dolstra
42d91b079c Fix use of sysread 2014-07-10 21:17:21 +02:00
Eelco Dolstra
7bb632b024 nix-copy-closure -s: Do substitutions via ‘nix-store --serve’
This means we no longer need an SSH master connection, since we only
execute a single command on the remote host.
2014-07-10 20:43:04 +02:00
Eelco Dolstra
7c3a5090bf nix-copy-closure: Fix --dry-run 2014-07-10 17:44:18 +02:00
Eelco Dolstra
43b64f5038 Remove tabs 2014-07-10 17:32:21 +02:00
Eelco Dolstra
8e9140cfde Refactoring: Move all fork handling into a higher-order function
C++11 lambdas ftw.
2014-07-10 16:58:09 +02:00
Eelco Dolstra
1114c7bd57 nix-copy-closure: Restore compression and the progress viewer 2014-07-10 14:15:12 +02:00
Eelco Dolstra
7911e4c27a Remove maybeVfork 2014-07-10 13:35:44 +02:00
Eelco Dolstra
04170d06bf nix-copy-closure: Fix race condition
There is a long-standing race condition when copying a closure to a
remote machine, particularly affecting build-remote.pl: the client
first asks the remote machine which paths it already has, then copies
over the missing paths. If the garbage collector kicks in on the
remote machine between the first and second step, the already-present
paths may be deleted. The missing paths may then refer to deleted
paths, causing nix-copy-closure to fail. The client now performs both
steps using a single remote Nix call (using ‘nix-store --serve’),
locking all paths in the closure while querying.

I changed the --serve protocol a bit (getting rid of QueryCommand), so
this breaks the SSH substituter from older versions. But it was marked
experimental anyway.

Fixes #141.
2014-07-10 11:58:59 +02:00
Eelco Dolstra
2c3a8f787b Fix security hole in ‘nix-store --serve’
Since it didn't check that the path received from the client is a
store path, the client could dump any path in the file system.
2014-07-10 11:46:01 +02:00