Commit graph

7149 commits

Author SHA1 Message Date
Eelco Dolstra
0b59882fe3
Merge pull request #1717 from grahamc/upload-release-aarch64
upload-release.pl: support aarch64-linux
2017-12-08 14:00:06 +01:00
Eelco Dolstra
d73e16df05
Merge pull request #1718 from markus1189/nix-hash-doc
nix-hash: Add sentence and example for nix-prefetch-url hash
2017-12-08 13:59:41 +01:00
Eelco Dolstra
82e6275a7b
Merge pull request #1719 from grahamc/fixup-darwin-hs-master
Fixup darwin hs master
2017-12-08 13:58:54 +01:00
Graham Christensen
1db034364a
replace lolcat with nix-info 2017-12-08 07:19:32 -05:00
Graham Christensen
be79d1f189
darwin installer: fix on High Sierra 2017-12-08 07:19:26 -05:00
Markus Hauck
0af668426d nix-hash: Add sentence and example for nix-prefetch-url hash 2017-12-08 10:27:34 +01:00
Graham Christensen
fbab9ed01e
upload-release.pl: support aarch64-linux 2017-12-07 19:02:35 -05:00
Eelco Dolstra
e3013543d3 Fix test 2017-12-07 01:07:07 +01:00
Eelco Dolstra
2df9cbeb47 Provide random access to cached NARs
E.g.

  $ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nars \
    /nix/store/b0w2hafndl09h64fhb86kw6bmhbmnpm1-blender-2.79/share/icons/hicolor/scalable/apps/blender.svg > /dev/null
  real    0m4.139s

  $ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nars \
    /nix/store/b0w2hafndl09h64fhb86kw6bmhbmnpm1-blender-2.79/share/icons/hicolor/scalable/apps/blender.svg > /dev/null
  real    0m0.024s

(Before, the second call took ~0.220s.)

This will use a NAR listing in
/tmp/nars/b0w2hafndl09h64fhb86kw6bmhbmnpm1.ls containing all metadata,
including the offsets of regular files inside the NAR. Thus, we don't
need to read the entire NAR. (We do read the entire listing, but
that's generally pretty small. We could use a SQLite DB by borrowing
some more code from nixos-channel-scripts/file-cache.hh.)

This is primarily useful when Hydra is serving files from an S3 binary
cache, in particular when you have giant NARs. E.g. we had some 12 GiB
NARs, so accessing individuals files was pretty slow.
2017-12-07 01:07:07 +01:00
Eelco Dolstra
338f29dbd4 nix ls-{nar,store}: Return offset of files in the NAR if known
E.g.

  $ nix ls-store --json --recursive --store https://cache.nixos.org /nix/store/b0w2hafndl09h64fhb86kw6bmhbmnpm1-blender-2.79 \
    | jq .entries.bin.entries.blender.narOffset
  400
2017-12-07 01:07:07 +01:00
Eelco Dolstra
ba334bd6f7
Merge pull request #1714 from LnL7/fix-ssh-store-nar
ssh-store: fix length when writing nar
2017-12-06 13:11:36 +01:00
Daiderd Jordan
2cb8aaa610
ssh-store: fix length when writing nar
This fixes nix copy and other things that use copyStorePath.
2017-12-06 11:41:08 +01:00
Shea Levy
11a7f8ce14
Merge branch 'fetchGit-fast-revision-update' 2017-12-05 11:16:39 -05:00
Eelco Dolstra
7f2c324ed1
Simplify build by including nlohmann/json.hpp 2017-12-04 17:11:36 +01:00
Eelco Dolstra
f1efb97075
Fix #1635. 2017-12-04 13:19:16 +01:00
Will Dietz
79f5c296c0 ignore "interrupted" exception in progress callback
Context/discusson:

https://github.com/NixOS/nix/issues/1692#issuecomment-348282301
2017-11-30 12:45:54 -06:00
Eelco Dolstra
cd74a55afc
Merge pull request #1704 from lheckemann/buildenv-eol-fix
Fix "Unexpected EOF reading a line" error
2017-11-29 15:27:57 +01:00
Linus Heckemann
f986a44980 Fix "Unexpected EOF reading a line" error
propagated-user-env-packages files in nixpkgs aren't all terminated by
newlines, as buildenv expected. Now it does not require a terminating
newline; note that this introduces a behaviour change: propagated user
env packages may now be spread across multiple lines. However, nix
1.11.x still expects them to be on a single line so this shouldn't be
used in nixpkgs for now.
2017-11-29 11:56:15 +00:00
Eelco Dolstra
b52846ab5b
Show log tail when a remote build fails 2017-11-28 15:36:28 +01:00
Will Dietz
e50d7335bf doc: linenoise -> linenoise-ng 2017-11-27 18:36:20 -06:00
Will Dietz
964349e44d no "linenoiseFree" in linenoise-ng 2017-11-27 18:30:05 -06:00
Will Dietz
21f515724c replace vendored linenoise with linenoise-ng 2017-11-27 18:27:28 -06:00
Eelco Dolstra
3c470c97a1
Merge pull request #1699 from aszlig/fix-remote-build-log
build-remote: Fix missing log output
2017-11-27 13:59:51 +01:00
aszlig
6567ab95a0
build-remote: Fix missing log output
The storeUri variable in the build-remote hook is declared very much to
the start of the main function and a bunch of lines later, the same
variable gets checked via hasPrefix() but it gets assigned *after* that
check when the most suitable machine for the build was choosen.

So I guess this was just a typo in d16fd24973
and what we really want is to either checkd the prefix *after* assigning
storeUri or use bestMachine->storeUri directly.

I choose the latter, because the former could introduce even more
regressions if the try block where the variable gets assigned terminates
early.

Nevertheless, the reason why the log output didn't work is because
hasPrefix() checked for "ssh://" in front of storeUri, but if the
storeUri isn't set correctly (or at all), we don't get the log file
descriptor set up properly, leading to no log output.

I've adjusted the remote-builds test to include a regression test for
this, so that we can make sure we get a build output when using remote
builds.

In addition to that I've tested this with two of my build farms and the
build logs are emitted correctly again.

Signed-off-by: aszlig <aszlig@nix.build>
2017-11-25 01:34:18 +01:00
Eelco Dolstra
549c3706a5
nix path-info: Show URL of NARs in binary caches 2017-11-24 18:08:50 +01:00
Eelco Dolstra
90948a4e3a
nix-shell/nix-build: Support .drv files again
Fixes #1663.

Also handle '!<output-name>' (#1694).
2017-11-24 18:08:35 +01:00
Shea Levy
eedbc4e06c
fetchGit: Ignore tarballTtl if rev is set and not in the repo.
Fixes #1697.
2017-11-24 06:09:24 -05:00
Eelco Dolstra
0fc3e581e0
tests/run.sh: Fix /build conflict when building in a sandbox
https://hydra.nixos.org/build/64519355
2017-11-21 19:43:12 +01:00
Eelco Dolstra
d7da6c9ea9
fetchGit/fetchMercurial: Fix directory inclusion check
E.g. the existence of .gitignore would cause .git to be included.
2017-11-21 19:34:46 +01:00
Eelco Dolstra
6cdaa858d0
Propagate flags like --sandbox to the daemon properly 2017-11-21 18:50:56 +01:00
Eelco Dolstra
7536fe31dd
Add a warning about the 'trusted-users' option 2017-11-21 18:49:52 +01:00
Eelco Dolstra
4fcf44825f
Add tests for verifying/copying content-addressed paths
These don't require signatures.
2017-11-20 19:11:02 +01:00
Eelco Dolstra
0c9718aabc
Add tests for signature checking when copying between local stores 2017-11-20 19:02:57 +01:00
Eelco Dolstra
8df60b4ea8
Document secret-key-files 2017-11-20 18:51:04 +01:00
Eelco Dolstra
4cde04f476
Add tests for "nix run" 2017-11-20 18:36:36 +01:00
Eelco Dolstra
f5f29dc2b7
nix eval --raw: Use coerceToString()
Thus,

  $ nix eval --raw '("foo")'
  foo

  $ nix eval --raw nixpkgs.hello
  /nix/store/1y6ckg6khrdsvll54s5spcmf3w6ka9k4-hello-2.10

  $ nix eval --raw '(/etc/resolv.conf)'
  /nix/store/vml92ama92i8mz013nny461mlvg8mvap-resolv.conf
2017-11-20 18:27:29 +01:00
Eelco Dolstra
1ff01187e2
nix run: Fix "flag '--command' requires 2 argument(s)" 2017-11-20 18:07:58 +01:00
Eelco Dolstra
4eb9e20028
nix run: Fix accidental removal of /nix/store existence check
Parenthetical to #1686, we don't need to create a new root if we can
just bind-mount on top of the existing /nix/store.
2017-11-20 17:59:32 +01:00
Eelco Dolstra
a3aa850f0f
nix run: Ignore non-directories while setting up the chroot
Fixes #1686.
2017-11-20 17:58:47 +01:00
Eelco Dolstra
193330d321
Test: Replace --option with the corresponding flag 2017-11-20 17:50:49 +01:00
Eelco Dolstra
91a1987607
signed-binary-caches -> require-sigs
Unlike signed-binary-caches (which could only be '*' or ''),
require-sigs is a proper Boolean option. The default is true.
2017-11-20 17:44:07 +01:00
Eelco Dolstra
d0b88db441
Cleanup 2017-11-20 17:32:58 +01:00
Eelco Dolstra
7a2b64e55c
binary-cache-public-keys -> trusted-public-keys
The name had become a misnomer since it's not only for substitution
from binary caches, but when adding/copying any
(non-content-addressed) path to a store.
2017-11-20 17:32:34 +01:00
Eelco Dolstra
ea94a87493
install-darwin-multi-user.sh: Remove superfluous nix.conf settings 2017-11-20 17:32:34 +01:00
Eelco Dolstra
72804dc0bd
Merge pull request #1645 from twhitehead/stat-race
Fix (highly unlikely) race condition in readLink
2017-11-20 17:32:27 +01:00
Eelco Dolstra
7474ac871b
nix copy: Abbreviate "daemon" 2017-11-20 15:17:11 +01:00
Eelco Dolstra
8956ae1987
Add a "profile" option to S3BinaryCacheStore
This allows specifying the AWS configuration profile to use. E.g.

  nix copy --from s3://my-cache?profile=aws-dev-account /nix/store/cf3isrlqavvd5w7rpky1fa8j9lcnlggm-...
2017-11-15 14:18:20 +01:00
Eelco Dolstra
897ca33a1c
Merge pull request #1678 from dtzWill/feature/anchor-exit-exception
anchor nix::Exit exception
2017-11-15 12:31:47 +01:00
Eelco Dolstra
27131866c9
Doh 2017-11-15 12:23:31 +01:00
Will Dietz
9d7ce0bf45 anchor nix::Exit exception 2017-11-14 12:39:30 -06:00