Commit graph

1102 commits

Author SHA1 Message Date
Eelco Dolstra
7f9759b18d
Merge pull request #4781 from NixOS/locally_cache_the_remote_realisations
Add a realisations disk cache
2021-05-10 20:37:57 +02:00
Eelco Dolstra
8768398d5f
Merge pull request #4787 from Ma27/builtins-ceil-floor
Implement `builtins.floor` and `builtins.ceil` using the C library functions internally
2021-05-10 20:37:27 +02:00
Maximilian Bosch
7f7f99f350
Implement builtins.floor and builtins.ceil using the C library functions internally
Closes #4782

Note: even though the type is internally called `NixFloat`, it's
actually a `double`.
2021-05-10 12:19:32 +02:00
regnat
b66234134f Add a realisations disk cache
Similar to the nar-info disk cache (and using the same db).
This makes rebuilds muuch faster.

- This works regardless of the ca-derivations experimental feature.
  I could modify the logic to not touch the db if the flag isn’t there,
  but given that this is a trash-able local cache, it doesn’t seem to be
  really worth it.
- We could unify the `NARs` and `Realisation` tables to only have one
  generic kv table. This is left as an exercise to the reader.
- I didn’t update the cache db version number as the new schema just
  adds a new table to the previous one, so the db will be transparently
  migrated and is backwards-compatible.

Fix #4746
2021-05-06 17:38:20 +02:00
regnat
bf485dcf46 Properly normalize the content-addressed paths
Make sure that their timestamp are always normalized.
Otherwise, strange − and non-deterministic − things might happen, like
https://github.com/NixOS/nixpkgs/issues/121813

Fix #4775
2021-05-05 21:00:08 +02:00
regnat
6ea9c65aec fixup! Add a regression test for #4725 2021-04-23 09:34:16 +02:00
regnat
b1711071d1 Add a regression test for #4725 2021-04-22 20:07:02 +02:00
Eelco Dolstra
76980a1f3d Merge branch 'build-with-strictDeps' of https://github.com/hercules-ci/nix 2021-04-15 14:15:21 +02:00
Eelco Dolstra
6fb7582413 Merge branch 'libarchive-decompress' of https://github.com/serokell/nix 2021-04-15 13:39:04 +02:00
Domen Kožar
c2dfda007e
add tests for zstd compression 2021-04-09 23:13:08 +02:00
Robert Hensing
c3090bc6fd tests/*: show when tests are skipped 2021-03-31 08:30:12 +02:00
Eelco Dolstra
e7810665a7
Merge pull request #4676 from bjornfor/fix-fetchgit-ref-head
fetchGit: don't prefix "refs/heads/" on ref = "HEAD"
2021-03-30 12:07:05 +02:00
Bjørn Forsman
f2a799b16d tests: check that builtins.fetchGit { ..., ref = "HEAD"; } works 2021-03-30 11:39:37 +02:00
Eelco Dolstra
3ab5e8a391
Merge pull request #4239 from tweag/test-against-old-daemon
Add a CI check to ensure compatibility with an old daemon
2021-03-29 16:15:36 +02:00
Eelco Dolstra
dd77f71afe LocalBinaryCacheStore::upsertFile(): Fix race
When multiple threads try to upsert the same file, this could fail.

Fixes #4667.
2021-03-26 17:10:15 +01:00
Samuel Dionne-Riel
bf07581497 tests: Test .config stays clean with XDG_CONFIG_HOME set 2021-03-19 15:20:47 -04:00
Samuel Dionne-Riel
1765711b68 tests/config: Fix config test configuration
First, "XDG_CONFIG_HOME" shouldn't be named "home", as it may be
confusing compared with `$HOME`, which an upcoming test will be using.

Then, using a fixed location for the test is problematic. Use
`$TEST_ROOT` instead.
2021-03-19 15:20:47 -04:00
Eelco Dolstra
66fa1c7375 Merge 'nix flake {info,list-inputs}' into 'nix flake metadata'
Fixes #4613.
2021-03-16 17:19:04 +01:00
regnat
be60c9ef50 Fix the db-migration test 2021-03-16 14:21:41 +01:00
regnat
81df1b5c68 Remove the remote-store-old-daemon test
Doesn't make sense anymore with the new setup
2021-03-16 14:21:40 +01:00
regnat
a0866c8ea4 Make the tests (optionnally) run in another derivation
That way we can run them without rebuilding Nix
2021-03-16 14:21:39 +01:00
regnat
eab9cdbd75 Add a test for the migration of the db between versions 2021-03-16 14:20:41 +01:00
regnat
5716345adf Add a test ensuring compatibility with an old daemon
This requires adding `nix` to its own closure which is a bit unfortunate,
but as it is optional (the test will be disabled if `OUTER_NIX` is unset) it
shouldn't be too much of an issue.

(Ideally this should go in another derivation so that we can build Nix and run
the test independently, but as the tests are running in the same derivation
as the build it's a bit complicated to do so).
2021-03-16 14:20:41 +01:00
sternenseemann
5869b3025d tests/local.mk: fix missing newline escape
Fixes syntax error introduced in 54ced9072b.
2021-03-16 08:25:04 +01:00
Eelco Dolstra
c0073f6268
Merge pull request #4580 from obsidiansystems/restore-test-build-remote-ca-fixed
Restore now-working build-remote-content-addressed-fixed test
2021-03-15 16:48:28 +01:00
regnat
54ced9072b Check the signatures when copying store paths around
Broken atm
2021-03-15 16:35:14 +01:00
Eelco Dolstra
306c154632
Merge pull request #4592 from NixOS/ca/remote-cache
Substitute content-addressed derivations
2021-03-15 16:22:42 +01:00
Kjetil Orbekk
92a234322f Add test for git+file with bare repository 2021-03-01 09:03:25 -05:00
regnat
93b5a59b67 Add a test for the remote caching of CA derivations 2021-03-01 14:00:17 +01:00
regnat
259d6778ef Move the CA tests to a sub-directory
Requires a slight update to the test infra to work properly, but
having the possibility to group tests that way makes the whole thing
quite cleaner imho
2021-03-01 11:08:01 +01:00
John Ericson
5b42e5b177 Restore now-working build-remote-content-addressed-fixed test
This was

 - Added in dbf96e10ec.

 - Commented out in 07975979aa, which I
   believe only reached master by mistake.

 - Deleted in c32168c9bc, when
   `tests/build-hook-ca.nix` was reused for a new test.

But the test works, and we ought to have it.
2021-02-26 16:32:52 +00:00
Eelco Dolstra
94637cd7e5
Merge pull request #4477 from NixOS/ca/build-remote
Build ca derivations remotely
2021-02-26 16:54:44 +01:00
Eelco Dolstra
453c3a603f nix flake update: Recreate the lock file
This is probably what most people expect it to do. Fixes #3781.

There is a new command 'nix flake lock' that has the old behaviour of
'nix flake update', i.e. it just adds missing lock file entries unless
overriden using --update-input.
2021-02-26 14:55:54 +01:00
regnat
aead35531a Add a test for the copy of CA paths 2021-02-25 17:19:18 +01:00
Shea Levy
98d1b64400
Initialize plugins after handling initial command line flags
This is technically a breaking change, since attempting to set plugin
files after the first non-flag argument will now throw an error. This
is acceptable given the relative lack of stability in a plugin
interface and the need to tie the knot somewhere once plugins can
actually define new subcommands.
2021-02-24 08:22:17 -05:00
Eelco Dolstra
a878c448d8
Merge pull request #4551 from danieldk/system-features-compute-level
Add x86_64 compute levels as system features
2021-02-23 14:30:10 +01:00
regnat
c32168c9bc Test the remote building of ca derivations 2021-02-23 08:04:03 +01:00
Daniël de Kok
2de232d2b3 Add x86_64 compute levels as additional system types
When performing distributed builds of machine learning packages, it
would be nice if builders without the required SIMD instructions can
be excluded as build nodes.

Since x86_64 has accumulated a large number of different instruction
set extensions, listing all possible extensions would be unwieldy.
AMD, Intel, Red Hat, and SUSE have recently defined four different
microarchitecture levels that are now part of the x86-64 psABI
supplement and will be used in glibc 2.33:

https://gitlab.com/x86-psABIs/x86-64-ABI
https://lwn.net/Articles/844831/

This change uses libcpuid to detect CPU features and then uses them to
add the supported x86_64 levels to the additional system types. For
example on a Ryzen 3700X:

$ ~/aps/bin/nix -vv --version | grep "Additional system"
Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux
2021-02-22 09:11:15 +01:00
regnat
be1b5c4e59 Test the garbage collection of CA derivations
Simple test to ensure that `nix-build && nix-collect-garbage &&
nix-build -j0` works as it should
2021-02-19 15:48:31 +01:00
Graham Christensen
f2245091d0
Revert "narinfo: Change NAR URLs to be addressed on the NAR hash instead of the compressed hash" 2021-02-09 12:26:41 -05:00
Eelco Dolstra
ee3846b587
Merge pull request #4464 from tweag/nar-narhash-addressed
narinfo: Change NAR URLs to be addressed on the NAR hash instead of the compressed hash
2021-02-09 14:47:39 +01:00
Eelco Dolstra
0187838e2e Add a trace to readLine() failures
Hopefully this helps to diagnose 'error: unexpected EOF reading a
line' on macOS.
2021-02-05 12:18:11 +01:00
Eelco Dolstra
488a826842
Merge pull request #4467 from edolstra/error-formatting
Improve error formatting
2021-01-25 12:50:57 +01:00
Eelco Dolstra
680d8a5b86
Merge pull request #4387 from obsidiansystems/non-local-store-build
Make `nix-build --store whatever` work
2021-01-25 12:24:23 +01:00
John Ericson
8c07ed1dda Improve documentation and test and requested 2021-01-22 15:58:58 +00:00
Eelco Dolstra
55849e153e Change error position formatting
It's now

  at /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix:7:7:

instead of

  at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix

The new format is more standard and clickable.
2021-01-21 11:02:09 +01:00
adisbladis
144cad9069
narinfo: Change NAR URLs to be addressed on the NAR hash instead of the compressed hash
This change is to simplify [Trustix](https://github.com/tweag/trustix) indexing and makes it possible to reconstruct this URL regardless of the compression used.

In particular this means that 7c2e9ca597/contrib/nix/nar/nar.go (L61-L71) can be removed and only the bits that are required to establish trust needs to be published in the Trustix build logs.
2021-01-21 10:32:56 +01:00
Eelco Dolstra
d04d8463fa
Merge pull request #4281 from lilyball/shebang
Escape filename given to nix-shell in shebang mode
2021-01-20 10:00:10 +01:00
John Ericson
0027b05a15 Merge remote-tracking branch 'upstream/master' into non-local-store-build 2021-01-15 02:01:24 +00:00
Eelco Dolstra
7a472a76d4 Add 'nix daemon' command 2021-01-14 00:05:04 +01:00