2020-07-23 14:10:59 +02:00
|
|
|
# Serving a Nix store via S3
|
2020-07-22 23:17:48 +02:00
|
|
|
|
2023-03-22 14:23:36 +01:00
|
|
|
Nix has [built-in support](@docroot@/command-ref/new-cli/nix3-help-stores.md#s3-binary-cache-store)
|
|
|
|
for storing and fetching store paths from
|
2020-07-24 15:46:16 +02:00
|
|
|
Amazon S3 and S3-compatible services. This uses the same *binary*
|
|
|
|
cache mechanism that Nix usually uses to fetch prebuilt binaries from
|
|
|
|
[cache.nixos.org](https://cache.nixos.org/).
|
2020-07-22 23:17:48 +02:00
|
|
|
|
|
|
|
In this example we will use the bucket named `example-nix-cache`.
|
|
|
|
|
|
|
|
## Anonymous Reads to your S3-compatible binary cache
|
|
|
|
|
|
|
|
If your binary cache is publicly accessible and does not require
|
|
|
|
authentication, the simplest and easiest way to use Nix with your S3
|
|
|
|
compatible binary cache is to use the HTTP URL for that cache.
|
|
|
|
|
|
|
|
For AWS S3 the binary cache URL for example bucket will be exactly
|
|
|
|
<https://example-nix-cache.s3.amazonaws.com> or
|
|
|
|
<s3://example-nix-cache>. For S3 compatible binary caches, consult that
|
|
|
|
cache's documentation.
|
|
|
|
|
|
|
|
Your bucket will need the following bucket policy:
|
|
|
|
|
2020-07-31 15:43:25 +02:00
|
|
|
```json
|
|
|
|
{
|
|
|
|
"Id": "DirectReads",
|
|
|
|
"Version": "2012-10-17",
|
|
|
|
"Statement": [
|
|
|
|
{
|
|
|
|
"Sid": "AllowDirectReads",
|
|
|
|
"Action": [
|
|
|
|
"s3:GetObject",
|
|
|
|
"s3:GetBucketLocation"
|
|
|
|
],
|
|
|
|
"Effect": "Allow",
|
|
|
|
"Resource": [
|
|
|
|
"arn:aws:s3:::example-nix-cache",
|
|
|
|
"arn:aws:s3:::example-nix-cache/*"
|
|
|
|
],
|
|
|
|
"Principal": "*"
|
|
|
|
}
|
|
|
|
]
|
|
|
|
}
|
|
|
|
```
|
2020-07-22 23:17:48 +02:00
|
|
|
|
|
|
|
## Authenticated Reads to your S3 binary cache
|
|
|
|
|
|
|
|
For AWS S3 the binary cache URL for example bucket will be exactly
|
|
|
|
<s3://example-nix-cache>.
|
|
|
|
|
|
|
|
Nix will use the [default credential provider
|
|
|
|
chain](https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/credentials.html)
|
|
|
|
for authenticating requests to Amazon S3.
|
|
|
|
|
|
|
|
Nix supports authenticated reads from Amazon S3 and S3 compatible binary
|
|
|
|
caches.
|
|
|
|
|
|
|
|
Your bucket will need a bucket policy allowing the desired users to
|
|
|
|
perform the `s3:GetObject` and `s3:GetBucketLocation` action on all
|
2020-07-24 15:46:16 +02:00
|
|
|
objects in the bucket. The [anonymous policy given
|
|
|
|
above](#anonymous-reads-to-your-s3-compatible-binary-cache) can be
|
2020-07-22 23:17:48 +02:00
|
|
|
updated to have a restricted `Principal` to support this.
|
|
|
|
|
|
|
|
## Authenticated Writes to your S3-compatible binary cache
|
|
|
|
|
|
|
|
Nix support fully supports writing to Amazon S3 and S3 compatible
|
|
|
|
buckets. The binary cache URL for our example bucket will be
|
|
|
|
<s3://example-nix-cache>.
|
|
|
|
|
|
|
|
Nix will use the [default credential provider
|
|
|
|
chain](https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/credentials.html)
|
|
|
|
for authenticating requests to Amazon S3.
|
|
|
|
|
|
|
|
Your account will need the following IAM policy to upload to the cache:
|
|
|
|
|
2020-07-31 15:43:25 +02:00
|
|
|
```json
|
|
|
|
{
|
|
|
|
"Version": "2012-10-17",
|
|
|
|
"Statement": [
|
2020-07-22 23:17:48 +02:00
|
|
|
{
|
2020-07-31 15:43:25 +02:00
|
|
|
"Sid": "UploadToCache",
|
|
|
|
"Effect": "Allow",
|
|
|
|
"Action": [
|
|
|
|
"s3:AbortMultipartUpload",
|
|
|
|
"s3:GetBucketLocation",
|
|
|
|
"s3:GetObject",
|
|
|
|
"s3:ListBucket",
|
|
|
|
"s3:ListBucketMultipartUploads",
|
|
|
|
"s3:ListMultipartUploadParts",
|
|
|
|
"s3:PutObject"
|
|
|
|
],
|
|
|
|
"Resource": [
|
|
|
|
"arn:aws:s3:::example-nix-cache",
|
|
|
|
"arn:aws:s3:::example-nix-cache/*"
|
2020-07-22 23:17:48 +02:00
|
|
|
]
|
|
|
|
}
|
2020-07-31 15:43:25 +02:00
|
|
|
]
|
|
|
|
}
|
|
|
|
```
|
2020-07-22 23:17:48 +02:00
|
|
|
|
2020-07-23 14:10:59 +02:00
|
|
|
## Examples
|
2020-07-22 23:17:48 +02:00
|
|
|
|
2020-07-23 14:10:59 +02:00
|
|
|
To upload with a specific credential profile for Amazon S3:
|
|
|
|
|
2020-07-31 15:43:25 +02:00
|
|
|
```console
|
|
|
|
$ nix copy nixpkgs.hello \
|
|
|
|
--to 's3://example-nix-cache?profile=cache-upload®ion=eu-west-2'
|
|
|
|
```
|
2020-07-23 14:10:59 +02:00
|
|
|
|
|
|
|
To upload to an S3-compatible binary cache:
|
|
|
|
|
2020-07-31 15:43:25 +02:00
|
|
|
```console
|
|
|
|
$ nix copy nixpkgs.hello --to \
|
|
|
|
's3://example-nix-cache?profile=cache-upload&scheme=https&endpoint=minio.example.com'
|
|
|
|
```
|