nix-super/doc/manual/source/advanced-topics/distributed-builds.md

109 lines
3.2 KiB
Markdown
Raw Permalink Normal View History

2020-07-23 00:17:48 +03:00
# Remote Builds
2024-10-07 10:15:02 +03:00
A local Nix installation can forward Nix builds to other machines,
this allows multiple builds to be performed in parallel.
Remote builds also allow Nix to perform multi-platform builds in a
semi-transparent way. For example, if you perform a build for a
2020-07-23 00:17:48 +03:00
`x86_64-darwin` on an `i686-linux` machine, Nix can automatically
2024-10-07 10:15:02 +03:00
forward the build to a `x86_64-darwin` machine, if one is available.
## Requirements
For a local machine to forward a build to a remote machine, the remote machine must:
2020-07-23 00:17:48 +03:00
2024-10-07 10:15:02 +03:00
- Have Nix installed
- Be running an SSH server, e.g. `sshd`
- Be accessible via SSH from the local machine over the network
- Have the local machine's public SSH key in `/etc/ssh/authorized_keys.d/<username>`
- Have the username of the SSH user in the `trusted-users` setting in `nix.conf`
## Testing
To test connecting to a remote Nix instance (in this case `mac`), run:
2020-07-23 00:17:48 +03:00
2020-07-31 16:43:25 +03:00
```console
2024-10-07 10:15:02 +03:00
nix store info --store ssh://username@mac
2020-07-31 16:43:25 +03:00
```
2020-07-23 00:17:48 +03:00
2024-10-07 10:15:02 +03:00
To specify an SSH identity file as part of the remote store URI add a
query paramater, e.g.
2020-07-23 00:17:48 +03:00
2020-07-31 16:43:25 +03:00
```console
2024-10-07 10:15:02 +03:00
nix store info --store ssh://username@mac?ssh-key=/home/alice/my-key
2020-07-31 16:43:25 +03:00
```
2020-07-23 00:17:48 +03:00
Since builds should be non-interactive, the key should not have a
passphrase. Alternatively, you can load identities ahead of time into
`ssh-agent` or `gpg-agent`.
2024-10-07 10:15:02 +03:00
In a multi-user installation (default), builds are executed by the Nix
Daemon. The Nix Daemon cannot prompt for a passphrase via the terminal
or `ssh-agent`, so the SSH key must not have a passphrase.
In addition, the Nix Daemon's user (typically root) needs to have SSH
access to the remote builder.
Access can be verified by running `sudo su`, and then validating SSH
access, e.g. by running `ssh mac`. SSH identity files for root users
are usually stored in `/root/.ssh/` (Linux) or `/var/root/.ssh` (MacOS).
2020-07-23 00:17:48 +03:00
If you get the error
2020-07-31 16:43:25 +03:00
```console
2024-10-07 10:15:02 +03:00
bash: nix: command not found
2020-07-31 16:43:25 +03:00
error: cannot connect to 'mac'
```
2020-07-23 00:17:48 +03:00
2020-07-23 11:44:54 +03:00
then you need to ensure that the `PATH` of non-interactive login shells
2020-07-23 00:17:48 +03:00
contains Nix.
The [list of remote build machines](@docroot@/command-ref/conf-file.md#conf-builders) can be specified on the command line or in the Nix configuration file.
For example, the following command allows you to build a derivation for `x86_64-darwin` on a Linux machine:
2020-07-23 00:17:48 +03:00
2020-07-31 16:43:25 +03:00
```console
2024-10-07 10:15:02 +03:00
uname
```
```console
2020-07-31 16:43:25 +03:00
Linux
2024-10-07 10:15:02 +03:00
```
```console
nix build --impure \
--expr '(with import <nixpkgs> { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \
--builders 'ssh://mac x86_64-darwin'
```
2024-10-07 10:15:02 +03:00
```console
2020-07-31 16:43:25 +03:00
[1/0/1 built, 0.0 MiB DL] building foo on ssh://mac
2024-10-07 10:15:02 +03:00
```
2020-07-31 16:43:25 +03:00
2024-10-07 10:15:02 +03:00
```console
cat ./result
```
```console
2020-07-31 16:43:25 +03:00
Darwin
```
2020-07-23 00:17:48 +03:00
It is possible to specify multiple build machines separated by a semicolon or a newline, e.g.
2020-07-23 00:17:48 +03:00
2020-07-31 16:43:25 +03:00
```console
2020-07-23 00:17:48 +03:00
--builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
```
Remote build machines can also be configured in [`nix.conf`](@docroot@/command-ref/conf-file.md), e.g.
2020-07-23 00:17:48 +03:00
builders = ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd
2024-10-07 10:15:02 +03:00
After making changes to `nix.conf`, restart the Nix daemon for changes to take effect.
Finally, remote build machines can be configured in a separate configuration
file included in `builders` via the syntax `@/path/to/file`. For example,
2020-07-23 00:17:48 +03:00
builders = @/etc/nix/machines
causes the list of machines in `/etc/nix/machines` to be included.
(This is the default.)