Commit graph

1173 commits

Author SHA1 Message Date
Matthew Bauer
7bd88cc1dc Merge remote-tracking branch 'origin/master' into substitute-other-storedir 2020-06-17 13:32:29 -04:00
Eelco Dolstra
1524752c17 Merge remote-tracking branch 'origin/master' into flakes 2020-06-17 10:26:52 +02:00
Matthew Bauer
9ac0d98a59 Remove -vvvvv from tests/fetchurl.sh nix-build call 2020-06-13 00:14:30 -05:00
Matthew Bauer
c214cda940 Correctly substitute from different storeDir
Originally, the test was only checking for different “real” storeDir.
That’s an easy case to handle, but the much harder one is if different
virtual store dirs are used. To do this, we need the SubstitutionGoal
to know about the ca, so it can recalculate the path to copy it over.
An important note here is that the store path passed to copyStorePath
needs to be one for srcStore - so that queryPathInfo works properly.

This also adds an error message when the store path from queryPathInfo
is different from the one we requested.
2020-06-13 00:07:42 -05:00
Matthew Bauer
006f1252d2 Fix SRI test
We can’t use custom name here because different names will have
different store paths. This is a limitation of the Store API’s
reliance on store paths.

We might be able to get around the above in the future by using a
dummy name for certain fixed output paths.
2020-06-12 15:33:24 -05:00
Matthew Bauer
88120442d2 Debug when storePath changes
these rewrites should be transparent, but they are important to know
about when debugging
2020-06-12 15:32:52 -05:00
Matthew Bauer
1c148a80fe Replace --hashed-mirrors with substituters test 2020-06-12 14:39:44 -05:00
Ben Burdette
ef1b3f21b6 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-11 14:06:35 -06:00
Eelco Dolstra
0c62b4ad0f Represent 'follows' inputs explicitly in the lock file
This fixes an issue where lockfile generation was not idempotent:
after updating a lockfile, a "follows" node would end up pointing to a
new copy of the node, rather than to the original node.
2020-06-11 14:40:21 +02:00
Ben Burdette
b1c53b034c Merge branch 'errors-phase-2' into caveman-LOCs 2020-06-08 11:10:13 -06:00
regnat
f6ac888d3e Actually test nix-env with a remote store
The `remote-store` test loads the `user-env` one to test nix-env when
using the daemon, but actually does it incorrectly because every test
starts (in `common.sh`) by resetting the value of `NIX_REMOTE`, meaning
that the `user-env` test will never use the daemon.

Fix this by setting `NIX_REMOTE_` before sourcing `user-env.sh` in the
`remote-store` test, so that `NIX_REMOTE` is correctly set inside the
test
2020-06-08 10:01:14 +02:00
Eelco Dolstra
810b2c6a48 nix flake init: Add a '--template' flag
The initial contents of the flake is specified by the
'templates.<name>' or 'defaultTemplate' output of another flake. E.g.

  outputs = { self }: {

    templates = {

      nixos-container = {
        path = ./nixos-container;
        description = "An example of a NixOS container";
      };

    };

  };

allows

  $ nix flake init -t templates#nixos-container

Also add a command 'nix flake new', which is identical to 'nix flake
init' except that it initializes a specified directory rather than the
current directory.
2020-06-04 20:22:25 +02:00
Ben Burdette
721943e1d4 update error grep 2020-06-03 17:32:57 -06:00
Ben Burdette
4335ba999b Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-06-03 17:00:00 -06:00
Eelco Dolstra
81cafda306 Fix GitHub test 2020-06-03 16:29:04 +02:00
Eelco Dolstra
c20591ddc3 Merge remote-tracking branch 'origin/master' into flakes 2020-06-03 16:15:22 +02:00
Nikola Knezevic
77007d4eab Improve ref validity checking in fetchGit
The previous regex was too strict and did not match what git was allowing. It
could lead to `fetchGit` not accepting valid branch names, even though they
exist in a repository (for example, branch names containing `/`, which are
pretty standard, like `release/1.0` branches).

The new regex defines what a branch name should **NOT** contain. It takes the
definitions from `refs.c` in https://github.com/git/git and `git help
check-ref-format` pages.

This change also introduces a test for ref name validity checking, which
compares the result from Nix with the result of `git check-ref-format --branch`.
2020-05-30 12:29:35 +02:00
Eelco Dolstra
17ca997fc6 Merge remote-tracking branch 'origin/master' into flakes 2020-05-28 12:55:24 +02:00
Eelco Dolstra
c3eff22f46 Merge branch 'store-visited' of https://github.com/mkenigs/nix into flakes 2020-05-28 12:15:39 +02:00
Matthew Kenigsberg
934cc802f3 circular test 2020-05-21 17:06:19 -06:00
Ben Burdette
0e49de6a2b position for stdin, string; (string) for trace; fix tests 2020-05-21 14:28:45 -06:00
Eelco Dolstra
00b562c87e Fix GitHub test 2020-05-21 22:02:34 +02:00
Ben Burdette
92123c6c79 Merge remote-tracking branch 'upstream/master' into errors-phase-2 2020-05-15 07:00:36 -06:00
Eelco Dolstra
5f64655ff4 Move registry-related commands from 'nix flake' to 'nix registry'
This makes 'nix flake' less cluttered and more consistent (it's only
subcommands that operator on a flake). Also, the registry is not
inherently flake-related (e.g. fetchTree could also use it to remap
inputs).
2020-05-15 14:38:10 +02:00
Eelco Dolstra
5722f9690c tests/binary-cache.sh: Improve incomplete closure test
Issue #3373.
2020-05-12 13:56:00 +02:00
Ben Burdette
59b1f5c701 Merge branch 'master' into errors-phase-2 2020-05-11 14:35:30 -06:00
Ben Burdette
55eb717148 add pos to errorinfo, remove from hints 2020-05-08 18:18:28 -06:00
Matthew Kenigsberg
2852a486f8 rename run to shell in tests
(cherry picked from commit f459ca547f)
2020-05-01 12:17:53 +02:00
Matthew Kenigsberg
f459ca547f rename run to shell in tests 2020-04-30 08:31:37 -06:00
Eelco Dolstra
b69323f8c9 Revive 'nix search'
It uses the evaluation cache now rather than the ad hoc JSON cache.
2020-04-20 15:27:09 +02:00
Eelco Dolstra
0858738355 Merge remote-tracking branch 'origin/master' into flakes 2020-04-16 18:27:37 +02:00
Eelco Dolstra
a118293bd0
Merge pull request #3458 from zimbatm/nix-user-conf-dir
NIX_USER_CONF_FILES
2020-04-15 13:00:28 +02:00
zimbatm
895516cadf
add NIX_USER_CONF_FILES
Motivation: maintain project-level configuration files.

Document the whole situation a bit better so that it corresponds to the
implementation, and add NIX_USER_CONF_FILES that allows overriding
which user files Nix will load during startup.
2020-04-14 18:45:06 +02:00
Eelco Dolstra
c0c2cb871d Merge remote-tracking branch 'origin/master' into flakes 2020-04-14 13:02:55 +02:00
Bruce Toll
e8bd1bc732 Add test case for temporary directories on darwin
A test case for correct handling of temporary directory deletion that
was added to check.sh as part of PR #2689 was initially disabled for
Darwin because of a directory permission issue in PR #2688.

Now that the issue in PR #2688 is fixed, this commit enables the test
case for Darwin.
2020-04-10 18:20:12 -04:00
Bruce Toll
8132d0a12e Fix nix-build --check -K in sandbox w/o root
Temporarily add user-write permission to build directory so that it
can be moved out of the sandbox to the store with a .check suffix.

This is necessary because the build directory has already had its
permissions set read-only, but write permission is required
to update the directory's parent link to move it out of the sandbox.

Updated the related --check "derivation may not be deterministic"
messages to consistently use the real store paths.

Added test for non-root sandbox nix-build --check -K to demonstrate
issue and help prevent regressions.
2020-04-10 16:23:10 -04:00
Eelco Dolstra
e5ea01c1a8
Remove flake 'edition' field
Future editions of flakes or the Nix language can be supported by
renaming flake.nix (e.g. flake-v2.nix). This avoids a bootstrap
problem where we don't know which grammar to use to parse
flake*.nix. It also allows a project to support multiple flake
editions, in theory.
2020-04-10 10:24:09 +02:00
Bruce Toll
16a4864759 Delete temporary directory on successful build
With --check and the --keep-failed (-K) flag, the temporary directory
was being retained regardless of whether the build was successful and
reproducible.  This removes the temporary directory, as expected, on
a reproducible check build.

Added tests to verify that temporary build directories are not
retained unnecessarily, particularly when using --check with
--keep-failed.
2020-04-09 16:37:41 -04:00
Eelco Dolstra
d103c79144
Merge remote-tracking branch 'origin/master' into flakes 2020-04-09 15:54:21 +02:00
Eelco Dolstra
9ed097db7b
Merge pull request #3468 from Infinisil/functionArgsPositions
Make function arguments retain position info
2020-04-08 15:29:39 +02:00
Eelco Dolstra
03a4a3c95c Merge branch 'flakes' of github.com:NixOS/nix into flakes 2020-04-07 14:30:25 +02:00
Eelco Dolstra
54955867a6 Merge remote-tracking branch 'origin/master' into flakes 2020-04-07 14:29:45 +02:00
Eelco Dolstra
55cefd41d6 Merge branch 'fetchgit-recursive' of https://github.com/blitz/nix 2020-04-07 13:45:17 +02:00
Eelco Dolstra
462421d345 Backport libfetchers from the flakes branch
This provides a pluggable mechanism for defining new fetchers. It adds
a builtin function 'fetchTree' that generalizes existing fetchers like
'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a
set of attributes, e.g.

  fetchTree {
    type = "git";
    url = "https://example.org/repo.git";
    ref = "some-branch";
    rev = "abcdef...";
  }

The existing fetchers are just wrappers around this. Note that the
input attributes to fetchTree are the same as flake input
specifications and flake lock file entries.

All fetchers share a common cache stored in
~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching
mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}).

This also adds support for Git worktrees (c169ea5904).
2020-04-07 09:03:14 +02:00
Eelco Dolstra
3473b1950a
Fix tests.githubFlakes evaluation 2020-04-03 21:36:21 +02:00
Eelco Dolstra
9c78f7f196 getFlake: In pure mode, check that the argument is an immutable flakeref 2020-04-03 13:07:05 +02:00
Silvan Mosberger
c34e96f7e0
Make function arguments retain position info
This allows querying the location of function arguments. E.g.

  builtins.unsafeGetAttrPos "x" (builtins.functionArgs ({ x }: null))

  => { column = 57; file = "/home/infinisil/src/nix/inst/test.nix"; line = 1; }
2020-04-02 05:52:52 +02:00
Eelco Dolstra
d15d91cad1 Makefile cleanup 2020-03-30 16:50:13 +02:00
Eelco Dolstra
e322a16523 Remove global -I flags
(cherry picked from commit 2c692a3b14)
2020-03-30 15:30:19 +02:00
Eelco Dolstra
2c692a3b14 Remove global -I flags 2020-03-30 14:39:33 +02:00
Bjørn Forsman
be84049baf tests/fetchGitSubmodules.sh: more checks 2020-03-29 22:29:58 +02:00
Bjørn Forsman
6864ad7cf5 fetchGit: fix submodule output attribute
Before this change it would be false for all evaluations but the first.
Now it follows the input argument (as it should).
2020-03-29 22:29:58 +02:00
Bjørn Forsman
587e259bfd tests/fetchGitSubmodules: add more tests 2020-03-29 22:29:58 +02:00
Julian Stecklina
6c00a9545f Fix typo in submodule test 2020-03-29 22:29:58 +02:00
Julian Stecklina
435366ed3c Rename fetchGit fetchSubmodules to just submodules 2020-03-29 22:29:58 +02:00
Julian Stecklina
c846abb5cc Add more test for git submodule functionality 2020-03-29 22:29:58 +02:00
Julian Stecklina
c8d33de777 Add git submodule fixes from @bjornfor
This fixes fetching repositories with no submodules and also cleans up
.git files in checkouts.
2020-03-29 22:29:57 +02:00
Julian Stecklina
c2a24c2b88 Add test for fetchGit submodule support 2020-03-29 22:29:57 +02:00
Eelco Dolstra
bf0b7e5423 Add test for circular flake dependencies 2020-03-27 22:03:40 +01:00
Eelco Dolstra
015f8f1c13 Improve lock file generation
This is now done in a single pass. Also fixes some issues when
updating flakes with circular dependencies. Finally, when using
'--recreate-lock-file --commit-lock-file', the commit message now
correctly shows the differences.
2020-03-27 21:08:41 +01:00
Eelco Dolstra
0a10854f85 Misc changes from the flakes branch 2020-03-24 14:34:47 +01:00
Eelco Dolstra
7a8de57d3e Pretty-print 'nix why-depends' / 'nix-store -q --tree' output
Extracted from 678301072f.
2020-03-24 14:26:23 +01:00
Eelco Dolstra
4e67f89f38 Fix GitHub VM test 2020-03-20 15:03:18 +01:00
Eelco Dolstra
8aa354fdfd
Register flake-registry.json as a GC root again 2020-03-19 11:45:34 +01:00
Eelco Dolstra
1c127e6a82
downloadFile(): Use expired file if the download fails 2020-03-19 11:42:50 +01:00
Eelco Dolstra
c5ec95e2c7
tarball.cc: Use ETags 2020-03-18 15:14:23 +01:00
Eelco Dolstra
38e360154d
Git: Use unified caching system 2020-03-17 22:35:29 +01:00
Eelco Dolstra
d1165d8791
Require shallow clones to be requested explicitly
If you do a fetchTree on a Git repository, whether the result contains
a revCount attribute should not depend on whether that repository
happens to be a shallow clone or not. That would complicate caching a
lot and would be semantically messy. So applying fetchTree/fetchGit to
a shallow repository is now an error unless you pass the attribute
'shallow = true'. If 'shallow = true', we don't return revCount, even
if the repository is not actually shallow.

Note that Nix itself is not doing shallow clones at the moment. But it
could do so as an optimisation if the user specifies 'shallow = true'.

Issue #2988.
2020-03-17 22:35:29 +01:00
Eelco Dolstra
2a4e4f6a6e
Unified fetcher caching system 2020-03-17 22:35:29 +01:00
Eelco Dolstra
fbcb897e21
Add a test for shallow Git clones
Also, don't return a revCount anymore for shallow or dirty Git trees,
since it's incorrect.

Closes #2988.
2020-03-16 13:20:32 +01:00
Eelco Dolstra
34c7645a58
Fix re-running the fetchGit.sh test 2020-03-16 12:30:11 +01:00
Eelco Dolstra
ae9119167e Change the lock file to a graph
This enables support for cycles between flakes.
2020-03-12 22:06:57 +01:00
Eelco Dolstra
35f6651735 Merge remote-tracking branch 'origin/master' into flakes 2020-03-11 17:03:38 +01:00
Eelco Dolstra
9950cdec35 Move some corepkgs into the nix binary 2020-03-11 16:57:48 +01:00
Eelco Dolstra
73769b28e3 Move calling flakes into a Nix helper function (call-flake.nix) 2020-03-09 15:28:41 +01:00
Eelco Dolstra
73b6d87e17
Merge remote-tracking branch 'origin/master' into flakes 2020-03-04 13:58:42 +01:00
Eelco Dolstra
d700eecea9
Add test for foldl' 2020-03-04 11:43:48 +01:00
Eelco Dolstra
890df325c7
fetchTree: Use a feature flag 2020-02-20 13:36:16 +01:00
Maximilian Bosch
c169ea5904
builtins.fetchGit: Fix build when fetching a git worktree
Worktrees[1] are a feature of git which allow you to check out a ref in
a different directory.

While playing around with flakes I realized that git repositories in a
worktree checkout break when trying to build a flake:

```
$ git worktree add ../nixpkgs-flakes nixpkgs-flakes
$ cd ../nixpkgs-flakes
$ nix build .#hello
error: opening directory '/home/ma27/Projects/nixpkgs-flakes/.git/refs/heads': Not a directory
```

This issue has been fixed by determining with `git rev-parse --git-common-dir`
where the actual `.git` directory is.

Please note that this issue only exists on the `flakes` branch, fetching
worktree checkouts with Nix master seems to work fine.

[1] https://git-scm.com/docs/git-worktree
2020-02-19 14:00:36 +01:00
Eelco Dolstra
9659940659 Test narHash mismatch 2020-02-12 10:33:52 +01:00
Eelco Dolstra
26dacc0983 Add fetchTree builtin function
This allows all supported fetchers to be used, e.g.

  builtins.fetchTree {
    type = "github";
    owner = "NixOS";
    repo = "nix";
    rev = "d4df99a3349cf2228a8ee78dea320afef86eb3ba";
  }
2020-02-11 23:53:46 +01:00
Eelco Dolstra
379852a152 Registry: Use attr notation instead of URLs 2020-02-06 14:27:31 +01:00
Eelco Dolstra
9d7fb62db6 Add option --commit-lock-file 2020-02-05 14:48:49 +01:00
Eelco Dolstra
d5334c466b
Automatically do git/hg add on flake.lock 2020-02-02 16:33:22 +01:00
Eelco Dolstra
a9ebc3ea5d
Remove the git+ and hg+ prefixes from structured input refs 2020-02-02 13:06:00 +01:00
Eelco Dolstra
90ada8e31a
--tarball-ttl 0 -> --refresh 2020-02-02 12:47:21 +01:00
Eelco Dolstra
94a94da075
Substitute flake inputs
This improves reproducibility and may be faster than fetching from the
original source (especially for git/hg inputs, but probably also for
github inputs - our binary cache is probably faster than GitHub's
dynamically generated tarballs).

Unfortunately this doesn't work for the top-level flake since even if
we know the NAR hash of the tree, we don't know the other tree
attributes such as revCount and lastModified.

Fixes #3253.
2020-02-02 12:41:23 +01:00
Eelco Dolstra
b270869466
Renamed ref / resolvedRef -> lockedRef 2020-02-02 00:06:59 +01:00
Eelco Dolstra
5d70b454be
nix flake update: Imply --refresh 2020-02-01 12:30:49 +01:00
Eelco Dolstra
9640208d00
Fix test
https://hydra.nixos.org/build/111166467
2020-02-01 09:30:55 +01:00
Eelco Dolstra
54037f4e2d Allow flake input specification via attributes rather than a URL
E.g.

  inputs.dwarffs = {
    type = "github";
    owner = "edolstra";
    repo = "dwarffs";
  };

rather than

  inputs.dwarffs.url = github:edolstra/dwarffs;
2020-01-31 20:50:46 +01:00
Eelco Dolstra
8414685c0f Change lock file format to use an attribute representation of flake refs rather than URLs 2020-01-31 19:16:40 +01:00
Eelco Dolstra
dbefe9e6b8 Fix test
https://hydra.nixos.org/build/111146865
2020-01-31 14:55:37 +01:00
Eelco Dolstra
e91f32f2b5 Use light box drawing symbols 2020-01-31 14:09:27 +01:00
Eelco Dolstra
678301072f nix flake list-inputs: Pretty-print the tree 2020-01-31 14:09:27 +01:00
Eelco Dolstra
a6e2b6b360 nix flake deps -> nix flake list-inputs
Also add a --json flag.
2020-01-31 13:00:50 +01:00
Eelco Dolstra
b9fb372075 Add --update-input flag to update a specific flake input
Typical usage:

  $ nix flake update ~/Misc/eelco-configurations/hagbard --update-input nixpkgs

to update the 'nixpkgs' input of a flake while leaving every other
input unchanged.

The argument is an input path, so you can do e.g. '--update-input
dwarffs/nixpkgs' to update an input of an input.

Fixes #2928.
2020-01-29 23:14:27 +01:00
Eelco Dolstra
26f895a26d Clean up the lock file handling flags
Added a flag --no-update-lock-file to barf if the lock file needs any
changes. This is useful for CI systems if you're building a
checkout. Fixes #2947.

Renamed --no-save-lock-file to --no-write-lock-file. It is now a fatal
error if the lock file needs changes but --no-write-lock-file is not
given.
2020-01-29 21:04:28 +01:00
Eelco Dolstra
f68bed7f67 Add flag --override-input to override specific lock file entries
E.g.

  $ nix flake update ~/Misc/eelco-configurations/hagbard \
    --override-input 'dwarffs/nixpkgs' ../my-nixpkgs

overrides the 'nixpkgs' input of the 'dwarffs' input of the top-level
flake.

Fixes #2837.
2020-01-29 18:41:25 +01:00
Eelco Dolstra
e53c89a643 Hopefully fix macOS test
https://hydra.nixos.org/build/110879694
2020-01-28 17:36:56 +01:00
Eelco Dolstra
99c8e7a48d Simplify flake tests 2020-01-28 16:34:37 +01:00
Eelco Dolstra
1af7b94c1d Add support for tarball flake inputs
For example,

  $ nix flake info https://github.com/edolstra/dwarffs/archive/master.tar.gz

Fixes #2929.
2020-01-28 13:11:02 +01:00
Eelco Dolstra
5046233b5a Add Mercurial tests 2020-01-27 13:45:49 +01:00
Eelco Dolstra
cc22cf662b Respect lock files of inputs + fine-grained lock file control
When computing a lock file, we now respect the lock files of flake
inputs. This is important for usability / reproducibility. For
example, the 'nixops' flake depends on the 'nixops-aws' and
'nixops-hetzner' repositories. So when the 'nixops' flake is used in
another flake, we want the versions of 'nixops-aws' and
'nixops-hetzner' locked by the the 'nixops' flake because those
presumably have been tested.

This can lead to a proliferation of versions of flakes like 'nixpkgs'
(since every flake's lock file could depend on a different version of
'nixpkgs'). This is not a major issue when using Nixpkgs overlays or
NixOS modules, since then the top-level flake composes those
overlays/modules into *its* version of Nixpkgs and all other versions
are ignored. Lock file computation has been made a bit more lazy so it
won't try to fetch all those versions of 'nixpkgs'.

However, in case it's necessary to minimize flake versions, there now
are two input attributes that allow this. First, you can copy an input
from another flake, as follows:

  inputs.nixpkgs.follows = "dwarffs/nixpkgs";

This states that the calling flake's 'nixpkgs' input shall be the same
as the 'nixpkgs' input of the 'dwarffs' input.

Second, you can override inputs of inputs:

  inputs.nixpkgs.url = github:edolstra/nixpkgs/<hash>;
  inputs.nixops.inputs.nixpkgs.url = github:edolstra/nixpkgs/<hash>;

or equivalently, using 'follows':

  inputs.nixpkgs.url = github:edolstra/nixpkgs/<hash>;
  inputs.nixops.inputs.nixpkgs.follows = "nixpkgs";

This states that the 'nixpkgs' input of the 'nixops' input shall be
the same as the calling flake's 'nixpkgs' input.

Finally, at '-v' Nix now prints the changes to the lock file, e.g.

  $ nix flake update ~/Misc/eelco-configurations/hagbard
  inputs of flake 'git+file:///home/eelco/Misc/eelco-configurations?subdir=hagbard' changed:
    updated 'nixpkgs': 'github:edolstra/nixpkgs/7845bf5f4b3013df1cf036e9c9c3a55a30331db9' -> 'github:edolstra/nixpkgs/03f3def66a104a221aac8b751eeb7075374848fd'
    removed 'nixops'
    removed 'nixops/nixops-aws'
    removed 'nixops/nixops-hetzner'
    removed 'nixops/nixpkgs'
2020-01-24 22:05:11 +01:00
Eelco Dolstra
b5c9dbc84f Fix --override-flake and add a test 2020-01-22 20:00:58 +01:00
Eelco Dolstra
ad6e55d777 Fix GitHub test 2020-01-22 00:25:17 +01:00
Eelco Dolstra
b33b94748c Convert fetchMercurial to a input type
This enables Mercurial flakes. It also fixes a bug in pure mode where
you could use a branch/tag name rather than a revision.
2020-01-21 23:49:32 +01:00
Eelco Dolstra
9f4d8c6170 Pluggable fetchers
Flakes are now fetched using an extensible mechanism. Also lots of
other flake cleanups.
2020-01-21 22:56:04 +01:00
Eelco Dolstra
6fadb3fc03 Merge remote-tracking branch 'origin/master' into flakes 2020-01-21 21:18:52 +01:00
Eelco Dolstra
aef635da78 Fix derivation computation with __structuredAttrs and multiple outputs
Fixes

  error: derivation '/nix/store/klivma7r7h5lndb99f7xxmlh5whyayvg-zlib-1.2.11.drv' has incorrect output '/nix/store/fv98nnx5ykgbq8sqabilkgkbc4169q05-zlib-1.2.11-dev', should be '/nix/store/adm7pilzlj3z5k249s8b4wv3scprhzi1-zlib-1.2.11-dev'
2020-01-21 21:14:13 +01:00
Nikola Knezevic
52a8f9295b Add support for \u escape in fromJSON
As fromTOML supports \u and \U escapes, bring fromJSON on par. As JSON defaults
to UTF-8 encoding (every JSON parser must support UTF-8), this change parses the
`\u hex hex hex hex` sequence (\u followed by 4 hexadecimal digits) into an
UTF-8 representation.

Add a test to verify correct parsing, using all escape sequences from json.org.
2020-01-07 00:09:58 +01:00
Eelco Dolstra
1dc29df1d3 Merge remote-tracking branch 'origin/master' into flakes 2020-01-06 12:43:07 +01:00
edef
c65a6fa86a passAsFile: leave out the hash prefix
Having a colon in the path may cause issues, and having the hash
function indicated isn't actually necessary. We now verify the path 
format in the tests to prevent regressions.
2020-01-02 23:56:06 +00:00
Eelco Dolstra
dbb4bec003 Fix tests.githubFlakes 2019-12-18 14:38:02 +01:00
Eelco Dolstra
87873d0d65 Merge remote-tracking branch 'origin/master' into flakes 2019-12-18 14:25:25 +01:00
Eelco Dolstra
ad6b738ed8 Merge remote-tracking branch 'origin/master' into flakes 2019-12-16 20:17:21 +01:00
Eelco Dolstra
54bf5ba422 nix-store -r: Handle symlinks to store paths
Fixes #3270.
2019-12-16 19:11:47 +01:00
Eelco Dolstra
5a6d6da7ae Validate tarball components 2019-12-13 19:05:26 +01:00
Eelco Dolstra
4581159e3f Simplify tarball test 2019-12-13 17:26:58 +01:00
Tom Bereknyei
c6295a3afd Initial gzip support
Closes #3256
2019-12-13 03:34:15 -05:00
Eelco Dolstra
ecb3a1afa2 Merge remote-tracking branch 'origin/master' into flakes 2019-12-11 14:53:30 +01:00
Eelco Dolstra
cce218f950 Add base32 encoder/decoder 2019-12-10 13:37:23 +01:00
Eelco Dolstra
c3c23a52ee Merge remote-tracking branch 'origin/master' into flakes 2019-12-04 00:31:09 +01:00
Eelco Dolstra
c1d18050b4 Disable recursive Nix test on macOS
https://hydra.nixos.org/build/107724274
2019-12-03 19:19:14 +01:00
Eelco Dolstra
ac2bc721d8 Merge remote-tracking branch 'origin/recursive-nix' 2019-12-02 12:34:46 +01:00
Eelco Dolstra
ca8caaec5e nix: Add --expr flag
This replaces the '(...)' installable syntax, which is not very
discoverable. The downside is that you can't have multiple expressions
or mix expressions and other installables.
2019-11-27 00:05:30 +01:00
Eelco Dolstra
6a9c815734 Remove most of <nix/config.nix>
This is no longer needed.
2019-11-26 22:07:28 +01:00
Eelco Dolstra
872740cf60
Merge pull request #3238 from puckipedia/attrset-overrides-dynamic
Ensure enough space in attrset bindings
2019-11-26 20:14:55 +01:00
Eelco Dolstra
c13193017f
Disallow empty store path names
Fixes #3239.
2019-11-26 20:12:15 +01:00
Puck Meerburg
cdadbf7708 Add testcase for attrset using __overrides and dynamic attrs 2019-11-25 13:03:54 +00:00
Eelco Dolstra
2c6dbcd5e7 Fix 'nix flake init' test 2019-11-20 13:07:44 +01:00
Eric Culp
6c041e8413 Replace $TMPDIR with $TEST_ROOT in tests/fetchurl.sh
$TMPDIR isn't necessarily set and would cause this test to fail.
2019-11-08 12:08:10 -08:00
Eelco Dolstra
850f73045f
Fix GitHub test 2019-11-06 14:48:35 +01:00
Eelco Dolstra
88c452d160
Merge remote-tracking branch 'origin/master' into flakes 2019-11-06 10:56:33 +01:00
Eelco Dolstra
c119ab9db0
Enable recursive Nix using a feature
Derivations that want to use recursion should now set

  requiredSystemFeatures = [ "recursive-nix" ];

to make the daemon socket appear.

Also, Nix should be configured with "experimental-features =
recursive-nix".
2019-11-06 00:55:03 +01:00
Eelco Dolstra
2af9561316
Add a test for recursive Nix 2019-11-06 00:55:03 +01:00
Eelco Dolstra
e1725ba946
Fix VM tests 2019-11-05 11:12:25 +01:00
Eelco Dolstra
b81d9d26f5
Merge remote-tracking branch 'origin/master' into flakes 2019-11-04 22:29:31 +01:00
Eelco Dolstra
0e459d79a6
Merge branch 'issue-3147-inNixShell-arg' of https://github.com/hercules-ci/nix 2019-10-27 17:10:19 +01:00
Robert Hensing
9d612c393a Add inNixShell = true to nix-shell auto-call
This is an alternative to the IN_NIX_SHELL environment variable,
allowing the expression to adapt itself to nix-shell without
triggering those adaptations when used as a dependency of another
shell.

Closes #3147
2019-10-27 13:16:02 +01:00
Eelco Dolstra
e012384fe9
Merge branch 'tojson-tostring-fix' of https://github.com/mayflower/nix 2019-10-27 12:18:35 +01:00
Robin Gloster
e583df5280
builtins.toJSON: fix __toString usage 2019-10-27 10:15:51 +01:00
Eelco Dolstra
aabf5c86c9
Add experimental-features setting
Experimental features are now opt-in. There is currently one
experimental feature: "nix-command" (which enables the "nix"
command. This will allow us to merge experimental features more
quickly, without committing to supporting them indefinitely.

Typical usage:

$ nix build --experimental-features 'nix-command flakes' nixpkgs#hello

(cherry picked from commit 8e478c2341,
without the "flakes" feature)
2019-10-21 13:34:44 +02:00
Eelco Dolstra
8e478c2341
Add experimental-features setting
Experimental features are now opt-in. There are currently two
experimental features: "nix-command" (which enables the "nix"
command), and "flakes" (which enables support for flakes). This will
allow us to merge experimental features more quickly, without
committing to supporting them indefinitely.

Typical usage:

$ nix build --experimental-features 'nix-command flakes' nixpkgs#hello
2019-10-16 17:49:01 +02:00
Eelco Dolstra
a56036fa87 Fix repeated fetchGit.sh test 2019-10-16 00:21:19 +02:00
Eelco Dolstra
14a89aa8cd Fix 'nix flake init' 2019-10-15 19:53:29 +02:00
Eelco Dolstra
7d38060a0d Support non-x86_64-linux system types in flakes
A command like

  $ nix run nixpkgs#hello

will now build the attribute 'packages.${system}.hello' rather than
'packages.hello'. Note that this does mean that the flake needs to
export an attribute for every system type it supports, and you can't
build on unsupported systems. So 'packages' typically looks like this:

  packages = nixpkgs.lib.genAttrs ["x86_64-linux" "i686-linux"] (system: {
    hello = ...;
  });

The 'checks', 'defaultPackage', 'devShell', 'apps' and 'defaultApp'
outputs similarly are now attrsets that map system types to
derivations/apps. 'nix flake check' checks that the derivations for
all platforms evaluate correctly, but only builds the derivations in
'checks.${system}'.

Fixes #2861. (That issue also talks about access to ~/.config/nixpkgs
and --arg, but I think it's reasonable to say that flakes shouldn't
support those.)

The alternative to attribute selection is to pass the system type as
an argument to the flake's 'outputs' function, e.g. 'outputs = { self,
nixpkgs, system }: ...'. However, that approach would be at odds with
hermetic evaluation and make it impossible to enumerate the packages
provided by a flake.
2019-10-15 18:16:29 +02:00
Eelco Dolstra
e99bb91217
Merge remote-tracking branch 'origin/master' into flakes 2019-10-10 12:54:37 +02:00
Eelco Dolstra
9348f9291e
nix-env: Create ~/.nix-profile automatically 2019-10-09 23:35:01 +02:00
Eelco Dolstra
26762ceb86
nix-profile.sh: Don't create .nix-channels
This is already done by the installer, so no need to do it again.
2019-10-09 23:35:01 +02:00
Eelco Dolstra
5a303093dc
Remove world-writability from per-user directories
'nix-daemon' now creates subdirectories for users when they first
connect.

Fixes #509 (CVE-2019-17365).
Should also fix #3127.
2019-10-09 23:34:48 +02:00
Eelco Dolstra
21304c11f9
uri -> url for consistency 2019-10-08 17:00:55 +02:00
Eelco Dolstra
a323b7826c
Merge remote-tracking branch 'origin/master' into flakes 2019-10-04 17:26:32 +02:00
Eelco Dolstra
204291f059
Merge release.nix, shell.nix and release-common.nix into flake.nix
Also provide a Nixpkgs overlay, memoize Nixpkgs evaluation and fit the
githubFlakes test.
2019-10-04 10:45:33 +02:00
Eelco Dolstra
5a0e98d1e5 Use '#' instead of ':' to separate flakeref and attrpath
This is less ambiguous.
2019-09-20 16:01:40 +02:00
zimbatm
619cc4af85
function-trace: always show the trace
If the user invokes nix with --trace-function-calls it means that they
want to see the trace.
2019-09-18 23:23:21 +02:00
Eelco Dolstra
f0e77c8a6c
Test quoted attrpaths
Issue #3076.
2019-09-10 16:06:52 +02:00
Eelco Dolstra
f3f854dac1
nix flake check: Add some tests 2019-09-10 16:03:03 +02:00
Eelco Dolstra
e302ba0e65
Merge remote-tracking branch 'origin/master' into flakes 2019-09-04 13:30:11 +02:00
Eelco Dolstra
5fad9d01c2
gc-auto.sh: Increase sleep time 2019-09-04 12:52:54 +02:00
Eelco Dolstra
08ee364950
gc-auto.sh: More test fixes 2019-09-03 18:11:43 +02:00
Eelco Dolstra
cec50290bf
gc-auto.sh: Add some more instrumentation 2019-09-03 15:45:32 +02:00
Eelco Dolstra
80c36d4562 Remove 'name' attribute from flakes
This is no longer needed since flakes are given an identity in the
'inputs' attribute.
2019-08-30 16:38:27 +02:00
Eelco Dolstra
30ccf4e52d Turn flake inputs into an attrset
Instead of a list, inputs are now an attrset like

  inputs = {
    nixpkgs.uri = github:NixOS/nixpkgs;
  };

If 'uri' is omitted, than the flake is a lookup in the flake registry, e.g.

  inputs = {
    nixpkgs = {};
  };

but in that case, you can also just omit the input altogether and
specify it as an argument to the 'outputs' function, as in

  outputs = { self, nixpkgs }: ...

This also gets rid of 'nonFlakeInputs', which are now just a special
kind of input that have a 'flake = false' attribute, e.g.

  inputs = {
    someRepo = {
      uri = github:example/repo;
      flake = false;
    };
  };
2019-08-30 16:27:51 +02:00
Eelco Dolstra
0588d72286 Update tests 2019-08-30 13:11:33 +02:00
Eelco Dolstra
ebc4dae517 Merge remote-tracking branch 'origin/master' into flakes 2019-08-29 16:11:38 +02:00
Eelco Dolstra
a2c4fcd5e9 Don't rely on st_blocks
It doesn't seem very reliable on ZFS.
2019-08-29 14:49:58 +02:00
Eelco Dolstra
31f5ecfaa5 Maybe fix #3058 2019-08-29 12:35:15 +02:00
Eelco Dolstra
ecb0a23d51 Add some more instrumentation 2019-08-29 12:10:01 +02:00
Eelco Dolstra
b6120d26a8
gc-auto.sh: Increase verbosity 2019-08-28 22:19:31 +02:00
Eelco Dolstra
73728874ab Hopefully fix post-hook test on macOS
https://hydra.nixos.org/build/99262744
2019-08-27 17:01:54 +02:00
Eelco Dolstra
477f82e5a7
Merge pull request #2782 from grahamc/flames
Track function start and end
2019-08-15 14:20:42 +02:00
Graham Christensen
ee9c988a1b
Track function start and ends for flame graphs
With this patch, and this file I called `log.py`:

    #!/usr/bin/env nix-shell
    #!nix-shell -i python3 -p python3 --pure

    import sys
    from pprint import pprint

    stack = []
    timestack = []

    for line in open(sys.argv[1]):
        components = line.strip().split(" ", 2)
        if components[0] != "function-trace":
            continue

        direction = components[1]
        components = components[2].rsplit(" ", 2)

        loc = components[0]
        _at = components[1]
        time = int(components[2])

        if direction == "entered":
            stack.append(loc)
            timestack.append(time)
        elif direction == "exited":
            dur = time - timestack.pop()
            vst = ";".join(stack)
            print(f"{vst} {dur}")
            stack.pop()

and:

    nix-instantiate --trace-function-calls -vvvv ../nixpkgs/pkgs/top-level/release.nix -A unstable > log.matthewbauer 2>&1
    ./log.py ./log.matthewbauer > log.matthewbauer.folded
    flamegraph.pl --title matthewbauer-post-pr log.matthewbauer.folded > log.matthewbauer.folded.svg

I can make flame graphs like: http://gsc.io/log.matthewbauer.folded.svg

---

Includes test cases around function call failures and tryEval. Uses
RAII so the finish is always called at the end of the function.
2019-08-14 16:09:35 -04:00
Eelco Dolstra
ab16b3d076
Fix gc-auto test 2019-08-08 15:49:19 +02:00
Eelco Dolstra
1d750e0587
Merge remote-tracking branch 'origin/master' into flakes 2019-08-08 15:49:13 +02:00
Eelco Dolstra
05a10dd835
tests/post-hook.sh: Don't put result link in cwd 2019-08-08 15:47:56 +02:00
Eelco Dolstra
2053ac7747
Rename file for consistency 2019-08-08 12:18:46 +02:00
Eelco Dolstra
56df30cd3f
Merge pull request #2995 from tweag/post-build-hook
Add a post build hook
2019-08-07 15:02:29 +02:00
Eelco Dolstra
ec415d7166
Add a test for auto-GC
This currently fails because we're using POSIX file locks. So when the
garbage collector opens and closes its own temproots file, it causes
the lock to be released and then deleted by another GC instance.
2019-08-02 18:39:16 +02:00
regnat
7c5596734f
Add a post-build-hook
Passing `--post-build-hook /foo/bar` to a nix-* command will cause
`/foo/bar` to be executed after each build with the following
environment variables set:

    DRV_PATH=/nix/store/drv-that-has-been-built.drv
    OUT_PATHS=/nix/store/...build /nix/store/...build-bin /nix/store/...build-dev

This can be useful in particular to upload all the builded artifacts to
the cache (including the ones that don't appear in the runtime closure
of the final derivation or are built because of IFD).

This new feature prints the stderr/stdout output to the `nix-build`
and `nix build` client, and the output is printed in a Nix 2
compatible format:

    [nix]$ ./inst/bin/nix-build ./test.nix
    these derivations will be built:
      /nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv
    building '/nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv'...
    hello!
    bye!
    running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'...
    post-build-hook: + sleep 1
    post-build-hook: + echo 'Signing paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: Signing paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: + sleep 1
    post-build-hook: + echo 'Uploading paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: Uploading paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: + sleep 1
    post-build-hook: + printf 'very important stuff'
    /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation

    [nix-shell:~/projects/github.com/NixOS/nix]$ ./inst/bin/nix build -L -f ./test.nix
    my-example-derivation> hello!
    my-example-derivation> bye!
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + echo 'Signing paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> Signing paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + echo 'Uploading paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> Uploading paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + printf 'very important stuff'
    [1 built, 0.0 MiB DL]

Co-authored-by: Graham Christensen <graham@grahamc.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2019-08-02 10:48:15 -04:00
Eelco Dolstra
219d645987
Merge pull request #3013 from basvandijk/disable-lsof-for-darwin-tests
Disable findRuntimeRoots on darwin when running tests because lsof is slow
2019-07-30 11:34:18 +02:00
Bas van Dijk
ee1e3132ca Disable findRuntimeRoots on darwin when running tests because lsof is slow
See: https://github.com/NixOS/nix/issues/3011
2019-07-30 11:29:03 +02:00
Bas van Dijk
89865144c3 Allow builtins.pathExists to check the existence of /nix/store paths
This makes it consitent with builtins.readDir.
2019-07-30 11:27:35 +02:00
Matthew Bauer
03addc3b0a Use $HOME instead of $USER
$USER/.nix-profile will not be a path. I think $HOME/.nix-profile was
the origininal intent.

/cc @Grahamc
2019-07-25 09:44:01 -04:00
Eelco Dolstra
b45628a172
Fix flake tests 2019-07-12 14:37:45 +02:00
Eelco Dolstra
cc218b15ba Merge remote-tracking branch 'origin/master' into flakes 2019-07-06 21:06:22 +02:00
Graham Christensen
68bdd83dc8
timeout: test for error code 2019-07-02 11:18:36 -04:00
Daiderd Jordan
a52c331edb
build: replace 100 offset for build exit codes 2019-07-02 00:12:38 +02:00
Daiderd Jordan
99ee3755dd
build: add tests for --check status codes 2019-07-02 00:12:38 +02:00
Eelco Dolstra
2cc248c4fd
Merge remote-tracking branch 'origin/master' into flakes 2019-06-25 14:27:57 +02:00
Eelco Dolstra
94f11d0a61
Fix abort in fromTOML
Fixes #2969.
2019-06-24 17:09:09 +02:00
Eelco Dolstra
99cec651c9
Add more fromTOML tests 2019-06-24 16:52:11 +02:00
Eelco Dolstra
d4fe9daed6
Simplify getFlake() / fetchFlake() logic 2019-06-21 19:04:58 +02:00
Eelco Dolstra
aa2846198f
Don't update the global registry when building a locked flake
It's unnecessary and slows things down (e.g. when you're on a Thalys
with super-crappy Internet).
2019-06-21 18:34:43 +02:00
Eelco Dolstra
d132d057a8
Handle store symlinks in flake directories
E.g. 'nix path-info ./result' inside a flake directory now works
again.
2019-06-21 15:29:05 +02:00
Nick Van den Broeck
e3552f2bcf Added tests for the nix flake CLI 2019-06-19 17:15:58 +02:00
Eelco Dolstra
e75ffbf04a
Merge pull request #2937 from CSVdB/fuzzymatching-v2
FuzzyMatching works
2019-06-18 18:03:38 +02:00
Nick Van den Broeck
59714a15e0 FuzzyMatching works
Fixes #2843
2019-06-18 17:01:57 +02:00
Eelco Dolstra
5064971ded
Fix test failures when $TMPDIR changes
(cherry picked from commit c38c726eb5)
2019-06-15 16:36:33 +02:00
Eelco Dolstra
e2d7569685
Add a test for GitHub flakes
Fixes #2889.
2019-06-11 21:46:46 +02:00
Eelco Dolstra
1b05792988
Shorter syntax for referencing flake outputs
Fixes #2819.
2019-06-04 22:35:43 +02:00
Eelco Dolstra
1e53a07712
Make non-flake inputs lazy
Also add a proper test for non-flake inputs.
2019-06-04 20:56:42 +02:00
Eelco Dolstra
4ec1a9ab40
Fix test 2019-06-04 20:33:49 +02:00
Eelco Dolstra
278114d559
Fix GC closure generation 2019-06-04 19:45:16 +02:00
Eelco Dolstra
6dbd5c26e6
Make flake input fetching lazy
As long as the flake input is locked, it is now only fetched when it
is evaluated (e.g. "nixpkgs" is fetched when
"inputs.nixpkgs.<something>" is evaluated).

This required adding an "id" attribute to the members of "inputs" in
lockfiles, e.g.

  "inputs": {
    "nixpkgs/release-19.03": {
      "id": "nixpkgs",
      "inputs": {},
      "narHash": "sha256-eYtxncIMFVmOHaHBtTdPGcs/AnJqKqA6tHCm0UmPYQU=",
      "nonFlakeInputs": {},
      "uri": "github:edolstra/nixpkgs/e9d5882bb861dc48f8d46960e7c820efdbe8f9c1"
    }
  }

because the flake ID needs to be known beforehand to construct the
"inputs" attrset.

Fixes #2913.
2019-06-04 19:17:03 +02:00
Nick Van den Broeck
d9a6a75ed2 Made epochs more fine-grained
Fixes #2894
2019-06-03 14:47:47 +02:00
Eelco Dolstra
a2f86ac647
Merge remote-tracking branch 'origin/master' into flakes 2019-06-03 09:22:19 +02:00
Eelco Dolstra
aec545c20b Fix segfault in builtin fetchurl with hashed mirrors + SRI hashes 2019-06-01 15:27:43 +02:00
Eelco Dolstra
8cb3bbd504
Fix handling of bare flakerefs containing a colon 2019-05-31 22:17:39 +02:00
Eelco Dolstra
ccb1bad612
Allow bare flakerefs as installables
So now

  $ nix build blender-bin

works and builds the default package from that flake. You don't need
to add a colon at the end anymore.
2019-05-31 21:42:23 +02:00
Eelco Dolstra
094539ef4a
Rename requires -> inputs, provides -> outputs
Issue #2828.
2019-05-31 09:59:48 +02:00
Eelco Dolstra
6636808e90
Merge remote-tracking branch 'origin/master' into flakes 2019-05-29 12:36:44 +02:00
Eelco Dolstra
17ef3e6f41
Enable more fromTOML tests
cpptoml now parses almost all examples from the spec.
2019-05-29 12:22:52 +02:00
Eelco Dolstra
cfd74aef1e
Fix eval-okay-fromTOML test
Turns out we were mis-parsing single-quoted attributes, e.g. 'key2'.
2019-05-29 12:12:02 +02:00
Eelco Dolstra
0f840483c7
Add date of last commit to SourceInfo
This is primarily useful for version string generation, where we need
a monotonically increasing number. The revcount is the preferred thing
to use, but isn't available for GitHub flakes (since it requires
fetching the entire history). The last commit timestamp OTOH can be
extracted from GitHub tarballs.
2019-05-29 10:10:36 +02:00
Eelco Dolstra
9eaebbf575
Merge branch 'attrPaths' of https://github.com/NinjaTrappeur/nix 2019-05-28 22:59:05 +02:00
Nick Van den Broeck
4d030a8d96 Added nonFlakeRequires test
Fixes #2888
2019-05-28 10:51:45 +02:00
Eelco Dolstra
638c56caed
Remove outdated fetchGit test
It's no longer an error if we can't update our clone.
2019-05-26 15:59:50 +02:00
Eelco Dolstra
90fe1dfd2f Register flake source trees as GC roots
This ensures that flakes don't get garbage-collected, which is
important to get nix-channel-like behaviour.

For example, running

  $ nix build hydra:

will create a GC root

  ~/.cache/nix/flake-closures/hydra -> /nix/store/xarfiqcwa4w8r4qpz1a769xxs8c3phgn-flake-closure

where the contents/references of the linked file in the store are the
flake source trees used by the 'hydra' flake:

  /nix/store/n6d5f5lkpfjbmkyby0nlg8y1wbkmbc7i-source
  /nix/store/vbkg4zy1qd29fnhflsv9k2j9jnbqd5m2-source
  /nix/store/z46xni7d47s5wk694359mq9ay353ar94-source

Note that this in itself is not enough to allow offline use; the
fetcher for the flakeref (e.g. fetchGit or downloadCached) must not
fail if it cannot fetch the latest version of the file, so long as it
knows a cached version.

Issue #2868.
2019-05-23 23:53:25 +02:00
Eelco Dolstra
013f4928c8 Fix tests
https://hydra.nixos.org/eval/1521131
2019-05-23 16:36:12 +02:00
Eelco Dolstra
a4ba6e5590 Add a test for the registry GC root 2019-05-22 23:52:29 +02:00
Eelco Dolstra
e414bde6f9 Check the flake epoch
Closes #2883.
2019-05-22 14:31:40 +02:00
Eelco Dolstra
bc0fb109a9 Add some tests 2019-05-22 14:04:18 +02:00
Nick Van den Broeck
ef6ae61503 Lockfile handling in resolveFlake is fixed 2019-05-17 14:50:10 +02:00
Nick Van den Broeck
98f20dee41 Give errors in resolveFlake
If DontUpdate but the lockfile isn't correct
2019-05-16 15:02:25 +02:00
Nick Van den Broeck
d9ad3723d5 Fixed issue 65
lockfile updating
2019-05-16 15:02:05 +02:00
Eelco Dolstra
0f5032c5a4 Merge remote-tracking branch 'origin/master' into flakes 2019-05-15 20:51:29 +02:00
Eelco Dolstra
696a98af5a
Disable tests introduced by PR #82
This PR was not intended to be merged until those tests were actually
passing. So disable them for now to unbreak the flakes branch.

https://hydra.nixos.org/eval/1519271
2019-05-15 09:03:46 +02:00
Daiderd Jordan
d75bdb5793
build: add test for sandboxed --check 2019-05-12 16:46:21 +02:00
Eelco Dolstra
391e1f511d
Add test for indirect flake dependencies
I.e. flake3 depends on flake2 which depends on flake1. Currently this
fails with

  error: indirect flake reference 'flake1' is not allowed

because we're not propagating lockfiles downwards properly.
2019-05-09 13:59:50 +02:00
Eelco Dolstra
a746dc64d2
Add currently failing test for #81 2019-05-09 13:55:33 +02:00
Eelco Dolstra
2bc55aba1e
Merge remote-tracking branch 'origin/master' into flakes 2019-05-08 14:30:27 +02:00
Will Dietz
a834861876
fix hashfile test that wasn't failing due to eval laziness
See:
7becb1bf1c (r33450554)
2019-05-08 10:04:54 +02:00
Eelco Dolstra
2d5a219688 Add basic flake tests 2019-05-07 23:32:09 +02:00
Eelco Dolstra
c38c726eb5 Fix test failures when $TMPDIR changes 2019-05-07 22:49:13 +02:00
Daniel Schaefer
3f192ac80c Add builtins.hashFile
For text files it is possible to do it like so:
`builtins.hashString "sha256" (builtins.readFile /tmp/a)`
but that doesn't work for binary files.

With builtins.hashFile any kind of file can be conveniently hashed.
2019-05-03 17:23:36 +02:00
Eelco Dolstra
0cbda84f5b exportGit: Don't clone local repositories
This ensures that commands like 'nix flake info /my/nixpkgs' don't
copy a gigabyte of crap to ~/.cache/nix.

Fixes #60.
2019-04-19 14:10:57 +02:00
Eelco Dolstra
ee1254d4f5 nix: Add --impure as a shorter alias of --no-pure-eval 2019-04-08 23:19:19 +02:00
Eelco Dolstra
d2875f6782 Fix tests 2019-04-08 16:11:36 +02:00
Eelco Dolstra
e489f5cabf Update eval-okay-types.exp to match #1828 2019-03-27 21:07:04 +01:00
zimbatm
514b3c7f83
Add isPath primop
this is added for completeness' sake since all the other possible
`builtins.typeOf` results have a corresponding `builtins.is<Type>`
2019-03-24 11:36:49 +01:00
Guillaume Maudoux
115e2c8c67 Update tests to the new --roots format 2019-03-10 01:50:33 +01:00
Eelco Dolstra
b71e1fb342
Restore --init calls in tests 2019-03-04 18:27:45 +08:00
zimbatm
b402148d8f
remove noop uses of nix-store --init
the nix-store --init command is a noop apparently
2019-02-22 21:07:53 +01:00
Shea Levy
b30be6b450
Add builtins.appendContext.
A partner of builtins.getContext, useful for the same reasons.
2019-01-31 08:52:23 -05:00
Shea Levy
1d757292d0
Add builtins.getContext.
This can be very helpful when debugging, as well as enabling complex
black magic like surgically removing a single dependency from a
string's context.
2019-01-14 11:27:10 -05:00
Will Dietz
aa7e52abff tests/fetchurl: fix after changing default hash from 512 to 256 2018-12-14 22:37:20 -06:00
Eelco Dolstra
6024dc1d97
Support SRI hashes
SRI hashes (https://www.w3.org/TR/SRI/) combine the hash algorithm and
a base-64 hash. This allows more concise and standard hash
specifications. For example, instead of

  import <nix/fetchurl.nl> {
    url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz;
    sha256 = "5d22dad058d5c800d65a115f919da22938c50dd6ba98c5e3a183172d149840a4";
  };

you can write

  import <nix/fetchurl.nl> {
    url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz;
    hash = "sha256-XSLa0FjVyADWWhFfkZ2iKTjFDda6mMXjoYMXLRSYQKQ=";
  };

In fixed-output derivations, the outputHashAlgo is no longer mandatory
if outputHash specifies the hash (either as an SRI or in the old
"<type>:<hash>" format).

'nix hash-{file,path}' now print hashes in SRI format by default. I
also reverted them to use SHA-256 by default because that's what we're
using most of the time in Nixpkgs.

Suggested by @zimbatm.
2018-12-13 14:30:52 +01:00
Eelco Dolstra
3f4de91d80
Merge branch 'better-git-cache' of https://github.com/graham-at-target/nix 2018-11-20 20:41:19 +01:00
Eelco Dolstra
812e39313c
Enable sandboxing by default
Closes #179.
2018-11-07 17:08:28 +01:00
Eelco Dolstra
18b4c53f71
Restore old (dis)allowedRequisites behaviour for self-references
stdenv relies on this. So ignore self-references (but only in legacy non-structured attributes mode).
2018-10-27 15:41:53 +02:00
Eelco Dolstra
3cd15c5b1f
Per-output reference and closure size checks
In structured-attributes derivations, you can now specify per-output
checks such as:

  outputChecks."out" = {
    # The closure of 'out' must not be larger than 256 MiB.
    maxClosureSize = 256 * 1024 * 1024;

    # It must not refer to C compiler or to the 'dev' output.
    disallowedRequisites = [ stdenv.cc "dev" ];
  };

  outputChecks."dev" = {
    # The 'dev' output must not be larger than 128 KiB.
    maxSize = 128 * 1024;
  };

Also fixed a bug in allowedRequisites that caused it to ignore
self-references.
2018-10-23 01:29:16 +02:00
Eelco Dolstra
01bd66bf83
Merge pull request #2452 from ElvishJerricco/verify-sigs-overflow
Fix overflow when verifying signatures of content addressable paths
2018-10-08 12:03:03 +02:00
Eelco Dolstra
1e7b8deea7
Check requiredSystemFeatures for local builds
For example, this prevents a "kvm" build on machines that don't have
KVM.

Fixes #2012.
2018-09-28 16:10:27 +02:00
Will Fancher
b7091ce41e Add a test for signed content-addressed paths 2018-09-25 22:18:52 -04:00
Graham Christensen
0aca1ffb6e Prefer 'remote builder' over 'slave' 2018-09-10 18:57:46 +02:00
Eelco Dolstra
9b1bdf2db8
FIx floating point evaluation
Fixes #2361.
2018-08-19 11:59:49 +02:00
Graham Christensen
02098d2073 fetchGit: use a better caching scheme
The current usage technically works by putting multiple different
repos in to the same git directory. However, it is very slow as
Git tries very hard to find common commits between the two
repositories. If the two repositories are large (like Nixpkgs and
another long-running project,) it is maddeningly slow.

This change busts the cache for existing deployments, but users
will be promptly repaid in per-repository performance.
2018-08-17 11:27:34 -04:00
Eelco Dolstra
fa4def3d46
Require libbrotli 2018-08-06 14:06:54 +02:00
Eelco Dolstra
34c17fdae5
Add a test for LegacySSHStore that doesn't require a VM 2018-08-03 21:20:38 +02:00
Eelco Dolstra
bc65e02d96
Merge pull request #2326 from aszlig/fix-symlink-leak
Fix symlink leak in restricted eval mode
2018-08-03 17:01:34 +02:00
aszlig
43e28a1b75
Fix symlink leak in restricted eval mode
In EvalState::checkSourcePath, the path is checked against the list of
allowed paths first and later it's checked again *after* resolving
symlinks.

The resolving of the symlinks is done via canonPath, which also strips
out "../" and "./". However after the canonicalisation the error message
pointing out that the path is not allowed prints the symlink target in
the error message.

Even if we'd suppress the message, symlink targets could still be leaked
if the symlink target doesn't exist (in this case the error is thrown in
canonPath).

So instead, we now do canonPath() without symlink resolving first before
even checking against the list of allowed paths and then later do the
symlink resolving and checking the allowed paths again.

The first call to canonPath() should get rid of all the "../" and "./",
so in theory the only way to leak a symlink if the attacker is able to
put a symlink in one of the paths allowed by restricted evaluation mode.

For the latter I don't think this is part of the threat model, because
if the attacker can write to that path, the attack vector is even
larger.

Signed-off-by: aszlig <aszlig@nix.build>
2018-08-03 06:46:43 +02:00
Samuel Dionne-Riel
438e02529d Allows selectively adding environment variables to pure shells.
Includes documentation and test.
2018-08-02 08:21:30 -04:00
volth
403a76a18f lib.concatMap and lib.mapAttrs to be builtins 2018-07-05 02:54:09 +00:00
Will Dietz
cb9ef85852 tests/search.sh: minor fix to unbreak tests after search ux merge 2018-07-03 15:37:06 -05:00
Eelco Dolstra
3b1f54cf06
Add a fromTOML primop
This is primarily useful for processing Cargo.lock files.
2018-07-03 18:39:36 +02:00
Aleksandr Pashkov
5ee4472b8b tests: more robust check for user namespaces availability (canUseSandbox)
Issue https://github.com/NixOS/nix/issues/2165
2018-06-06 00:41:01 +03:00
Eelco Dolstra
4ac4f675df
Don't require --fallback to recover from disappeared binary cache NARs 2018-06-05 16:04:41 +02:00
Eelco Dolstra
ca06a9cea7
Improve binary cache fallback test 2018-06-05 15:26:12 +02:00
Eelco Dolstra
a936a19da3
Remove non-existent NIX_DEBUG_SUBST 2018-06-05 13:40:45 +02:00
Eelco Dolstra
737ed88f35
Modularize config settings
Allow global config settings to be defined in multiple Config
classes. For example, this means that libutil can have settings and
evaluator settings can be moved out of libstore. The Config classes
are registered in a new GlobalConfig class to which config files
etc. are applied.

Relevant to https://github.com/NixOS/nix/issues/2009 in that it
removes the need for ad hoc handling of useCaseHack, which was the
underlying cause of that issue.
2018-05-30 13:28:01 +02:00
volth
88c1ea30e4 add docs and tests 2018-05-24 12:51:34 +00:00
Eelco Dolstra
1ad19232c4
Don't return negative numbers from the flex tokenizer
Fixes #1374.
Closes #2129.
2018-05-11 12:05:12 +02:00
Eelco Dolstra
548ad391d9
Fix builtins.add test
Nix prints the floating point number 4.0 as "4".
2018-05-02 13:49:32 +02:00
Félix Baylac-Jacqué
b2f3a7411a
nix-lang: Add deep nested mixed attrs test case. 2018-05-01 14:42:34 +02:00
Tim Sears
9a714f75af
add tests for builtins.add 2018-04-29 12:03:42 -07:00
Shea Levy
e2b028353b
Merge branch 'pos-crash-fix' of git://github.com/dezgeg/nix 2018-04-23 08:43:54 -04:00
Shea Levy
0aae411eaa
Merge branch 'add-test-eval-okay-nested-with' of git://github.com/ryantrinkle/nix 2018-04-23 08:43:14 -04:00
Daniel Poelzleithner
f6e8ceafa6 add tests for multi search 2018-04-18 21:45:46 +02:00
Félix Baylac-Jacqué
00584bb091
parser: Allow mixed nested and top-level attrpaths
Fixes #2077.
2018-04-18 18:44:38 +02:00
Félix Baylac-Jacqué
10d33452e2
nix-lang parser: Add mixed nested attrs tests.
Part of issue #2077
2018-04-17 09:13:01 +02:00
Shea Levy
7459388448
Merge branch 'fix/ruby-shebang' of git://github.com/samueldr/nix 2018-04-09 10:54:39 -04:00
Samuel Dionne-Riel
a4c9b2595e Hardcodes nix-shell instead of /usr/bin/env nix-shell...
... in the ruby shebang test.
2018-04-09 09:36:54 -04:00
Samuel Dionne-Riel
9478f88681 Adds nix-shell test for special-cased ruby interpreter.
The test fakes the interpreter only to verify the arguments it would be
given.
2018-04-08 00:45:28 -04:00
Doug Beardsley
72902ec399 Add missing eval-okay-regex-split.exp test file 2018-04-06 12:18:59 -06:00
Tuomas Tynkkynen
af86132e1a libexpr: Make unsafeGetAttrPos not crash on noPos
Currently e.g. `builtins.unsafeGetAttrPos "abort" builtins` will
eventually segfault because pos->file is an unset Symbol.

Found by afl-fuzz.
2018-04-03 15:54:42 +03:00
Ryan Trinkle
f2273b11f5 Add test eval-okay-nested-with 2018-03-30 15:41:16 -04:00
Linus Heckemann
12913ccf45 nix-shell: allow symlinks to .drvs
This makes persistent shell environments easier to use.
2018-03-16 22:15:24 +00:00
Eelco Dolstra
64441f0551
Merge pull request #1939 from dezgeg/lexer-fix
libexpr: Recognize newline in more places in lexer
2018-03-16 12:14:58 +01:00
Eelco Dolstra
ca14b14200
Use boost::format from the boost package
Note that this only requires headers from boost so it doesn't add a
runtime dependency.

Also, use Nixpkgs 18.03.
2018-03-14 19:24:04 +01:00
Shea Levy
56f2ed0081
Merge pull request #1906 from dtzWill/fix/nix-search
nix search: tests and fix #1893 and part of #1892
2018-03-13 18:43:47 -06:00
Shea Levy
aa8bbbf69d
Merge branch 'write-failure-fixes' of git://github.com/lheckemann/nix 2018-03-02 10:59:59 -05:00
Tuomas Tynkkynen
a0e38c16bc libexpr: Recognize newline in more places in lexer
Flex's regexes have an annoying feature: the dot matches everything
except a newline. This causes problems for expressions like:

"${0}\
"

where the backslash-newline combination matches this rule instead of the
intended one mentioned in the comment:

    <STRING>\$|\\|\$\\ {
                    /* This can only occur when we reach EOF, otherwise the above
                    (...|\$[^\{\"\\]|\\.|\$\\.)+ would have triggered.
                    This is technically invalid, but we leave the problem to the
                    parser who fails with exact location. */
                    return STR;
                }
However, the parser actually accepts the resulting token sequence
('"' DOLLAR_CURLY 0 '}' STR '"'), which is a problem because the lexer
rule didn't assign anything to yylval. Ultimately this leads to a crash
when dereferencing a NULL pointer in ExprConcatStrings::bindVars().

The fix does change the syntax of the language in some corner cases
but I think it's only turning previously invalid (or crashing) syntax
to valid syntax. E.g.

"a\
b"

and

''a''\
b''

were previously syntax errors but now both result in "a\nb".

Found by afl-fuzz.
2018-03-02 17:30:48 +02:00
Will Dietz
c577186f59 tests/search.sh: simplify, don't rely on consistent ordering 2018-03-01 13:25:06 -06:00
Tuomas Tynkkynen
77e9e1ed91 libexpr: Fix prim_replaceStrings() to work on an empty source string
Otherwise, running e.g.

nix-instantiate --eval -E --strict 'builtins.replaceStrings [""] ["X"] "abc"'

would just hang in an infinite loop.

Found by afl-fuzz.

First attempt of this was reverted in e2d71bd186 because it caused
another infinite loop, which is fixed now and a test added.
2018-02-26 19:49:13 +02:00
Will Dietz
8282c60d74 tests: test nix search behavior 2018-02-25 16:40:05 -06:00
Eelco Dolstra
de4c03d201
Merge branch 'fix/dry-run-partially' of https://github.com/dtzWill/nix 2018-02-22 12:18:20 +01:00
Eelco Dolstra
e2d71bd186
Revert "libexpr: Fix prim_replaceStrings() to work on an empty source string"
This reverts commit 4ea9707591.

It causes an infinite loop in Nixpkgs evaluation,
e.g. "nix-instantiate -A hello" hung.

PR #1886.
2018-02-21 15:35:28 +01:00
Tuomas Tynkkynen
4ea9707591 libexpr: Fix prim_replaceStrings() to work on an empty source string
Otherwise, running e.g.

nix-instantiate --eval -E --strict 'builtins.replaceStrings [""] ["X"] "abc"'

would just hang in an infinite loop.

Found by afl-fuzz.
2018-02-19 23:20:26 +02:00
Eelco Dolstra
75a1d96cfd
Merge branch 'register-settings' of https://github.com/shlevy/nix 2018-02-19 13:58:34 +01:00