Commit graph

1479 commits

Author SHA1 Message Date
Eelco Dolstra
b5c9dbc84f Fix --override-flake and add a test 2020-01-22 20:00:58 +01:00
Eelco Dolstra
32f31a8c63 nix flake info: Don't show empty descriptions 2020-01-22 17:20:21 +01:00
Eelco Dolstra
b33b94748c Convert fetchMercurial to a input type
This enables Mercurial flakes. It also fixes a bug in pure mode where
you could use a branch/tag name rather than a revision.
2020-01-21 23:49:32 +01:00
Eelco Dolstra
9f4d8c6170 Pluggable fetchers
Flakes are now fetched using an extensible mechanism. Also lots of
other flake cleanups.
2020-01-21 22:56:04 +01:00
Eelco Dolstra
6fadb3fc03 Merge remote-tracking branch 'origin/master' into flakes 2020-01-21 21:18:52 +01:00
Eelco Dolstra
aef635da78 Fix derivation computation with __structuredAttrs and multiple outputs
Fixes

  error: derivation '/nix/store/klivma7r7h5lndb99f7xxmlh5whyayvg-zlib-1.2.11.drv' has incorrect output '/nix/store/fv98nnx5ykgbq8sqabilkgkbc4169q05-zlib-1.2.11-dev', should be '/nix/store/adm7pilzlj3z5k249s8b4wv3scprhzi1-zlib-1.2.11-dev'
2020-01-21 21:14:13 +01:00
Eelco Dolstra
c3181e21e7 Tweak error message 2020-01-13 21:52:03 +01:00
Eelco Dolstra
bfaa4db7bd Merge branch 'assert-show-expression' of https://github.com/LnL7/nix 2020-01-13 21:49:55 +01:00
Eelco Dolstra
835e541144 Fix build
https://hydra.nixos.org/eval/1564374
2020-01-13 21:34:54 +01:00
Daiderd Jordan
307bcb9a8e
libexpr: show expression in assertion errors
Includes the expression of the condition in the assertion message if
the assertion failed, making assertions much easier to debug. eg.

    error: assertion (withPython -> (python2Packages != null)) failed at pkgs/tools/security/nmap/default.nix:11:1
2020-01-11 15:45:41 +01:00
Yorick van Pelt
a350d0beb0
json-to-value: use unique_ptr instead of raw pointers 2020-01-09 22:46:41 +01:00
Yorick van Pelt
f1fac0b5c3
builtins.fromJSON: use nlohmann/json parser instead of custom parser 2020-01-09 17:38:27 +01:00
Nikola Knezevic
52a8f9295b Add support for \u escape in fromJSON
As fromTOML supports \u and \U escapes, bring fromJSON on par. As JSON defaults
to UTF-8 encoding (every JSON parser must support UTF-8), this change parses the
`\u hex hex hex hex` sequence (\u followed by 4 hexadecimal digits) into an
UTF-8 representation.

Add a test to verify correct parsing, using all escape sequences from json.org.
2020-01-07 00:09:58 +01:00
Nikola Knezevic
cb2d348d48 Remove redundant check in parseJSONString 2020-01-07 00:09:58 +01:00
Eelco Dolstra
1dc29df1d3 Merge remote-tracking branch 'origin/master' into flakes 2020-01-06 12:43:07 +01:00
Eelco Dolstra
cb90e382b5 Hide FunctionCallTrace constructor/destructor
This prevents them from being inlined. On gcc 9, this reduces the
stack size needed for

  nix-instantiate '<nixpkgs>' -A texlive.combined.scheme-full --dry-run

from 12.9 MiB to 4.8 MiB.
2020-01-05 16:21:34 +01:00
Eelco Dolstra
87873d0d65 Merge remote-tracking branch 'origin/master' into flakes 2019-12-18 14:25:25 +01:00
Eelco Dolstra
ad6b738ed8 Merge remote-tracking branch 'origin/master' into flakes 2019-12-16 20:17:21 +01:00
Eelco Dolstra
54bf5ba422 nix-store -r: Handle symlinks to store paths
Fixes #3270.
2019-12-16 19:11:47 +01:00
Eelco Dolstra
acb71aa5c6 Tweak error message 2019-12-15 10:44:53 +01:00
Dima
d89d9958a7 bugfix: Adding depth limit to resolveExprPath
There is no termination condition for evaluation of cyclical
expression paths which can lead to infinite loops. This addresses
one spot in the parser in a similar fashion as utils.cc/canonPath
does.

This issue can be reproduced by something like:

```
ln -s a b
ln -s b a

nix-instantiate -E 'import ./a'
```
2019-12-13 14:51:30 +01:00
Eelco Dolstra
ecb3a1afa2 Merge remote-tracking branch 'origin/master' into flakes 2019-12-11 14:53:30 +01:00
Eelco Dolstra
bbe97dff8b Make the Store API more type-safe
Most functions now take a StorePath argument rather than a Path (which
is just an alias for std::string). The StorePath constructor ensures
that the path is syntactically correct (i.e. it looks like
<store-dir>/<base32-hash>-<name>). Similarly, functions like
buildPaths() now take a StorePathWithOutputs, rather than abusing Path
by adding a '!<outputs>' suffix.

Note that the StorePath type is implemented in Rust. This involves
some hackery to allow Rust values to be used directly in C++, via a
helper type whose destructor calls the Rust type's drop()
function. The main issue is the dynamic nature of C++ move semantics:
after we have moved a Rust value, we should not call the drop function
on the original value. So when we move a value, we set the original
value to bitwise zero, and the destructor only calls drop() if the
value is not bitwise zero. This should be sufficient for most types.

Also lots of minor cleanups to the C++ API to make it more modern
(e.g. using std::optional and std::string_view in some places).
2019-12-10 22:06:05 +01:00
Eelco Dolstra
98ef11677c EvalState::callFunction(): Make FunctionCallTrace use less stack space
The FunctionCallTrace object consumes a few hundred bytes of stack
space, even when tracing is disabled. This was causing stack overflows:

  $ nix-instantiate '<nixpkgs> -A texlive.combined.scheme-full --dry-run
  error: stack overflow (possible infinite recursion)

This is with the default stack size of 8 MiB.

Putting the object on the heap reduces stack usage to < 5 MiB.
2019-12-10 13:32:30 +01:00
Eelco Dolstra
0678e4d56a Move #include
(cherry picked from commit 8beedd4486)
2019-12-05 20:30:29 +01:00
Eelco Dolstra
79142cbbe1 Bindings: Add convenience method for requiring an attribute
(cherry picked from commit fb692e5f7b)
2019-12-05 20:29:15 +01:00
Eelco Dolstra
0d118ef0c9 Bindings::get(): Add convenience method
This allows writing attribute lookups as

    if (auto name = value.attrs->get(state.sName))
      ...

(cherry picked from commit f216c76c56)
2019-12-05 20:29:00 +01:00
Eelco Dolstra
c3c23a52ee Merge remote-tracking branch 'origin/master' into flakes 2019-12-04 00:31:09 +01:00
Eelco Dolstra
f102d793f1
Merge pull request #2748 from edolstra/rust
Make nix/unpack-channel.nix a builtin builder
2019-11-29 19:33:31 +01:00
Eelco Dolstra
2d6f1ddbb5
Remove builtins.valueSize
Fixes #3246.
2019-11-28 13:52:42 +01:00
Eelco Dolstra
8918bae098 Drop remaining uses of external "tar"
Also, fetchGit now runs in O(1) memory since we pipe the output of
'git archive' directly into unpackTarball() (rather than first reading
it all into memory).
2019-11-26 22:07:28 +01:00
Eelco Dolstra
872740cf60
Merge pull request #3238 from puckipedia/attrset-overrides-dynamic
Ensure enough space in attrset bindings
2019-11-26 20:14:55 +01:00
Eelco Dolstra
1ec6e6e11e
Add feature to disable URL literals
E.g.

  $ nix-build '<nixpkgs>' -A hello --experimental-features no-url-literals
  error: URL literals are disabled, at /nix/store/vsjamkzh15r3c779q2711az826hqgvzr-nixpkgs-20.03pre194957.bef773ed53f/nixpkgs/pkgs/top-level/all-packages.nix:1236:11

Helps with implementing https://github.com/NixOS/rfcs/pull/45.
2019-11-26 19:48:34 +01:00
Puck Meerburg
cd55f91ad2 Ensure enough space in attrset bindings when using both __overrides and dynamic attributes 2019-11-25 12:37:14 +00:00
Eelco Dolstra
d12d69ea1a
Turn NIX_PATH into a config setting
This allows it to be set in nix.conf.
2019-11-22 23:07:35 +01:00
Eelco Dolstra
ec9dd9a5ae
Provide a default value for NIX_PATH 2019-11-22 22:08:51 +01:00
Eelco Dolstra
ba87b08f85
getEnv(): Return std::optional
This allows distinguishing between an empty value and no value.
2019-11-22 16:18:13 +01:00
Eelco Dolstra
fd8ee94ab2 Remove #include 2019-11-20 13:04:39 +01:00
Eelco Dolstra
8beedd4486 Move #include 2019-11-20 13:04:39 +01:00
Eelco Dolstra
5ee23c35b9
Merge pull request #3219 from Ericson2314/semicolons
Fix extra semicolons warnings
2019-11-11 12:13:51 +01:00
John Ericson
4c34054673 Remove unneeded semicolons 2019-11-10 11:24:47 -05:00
John Ericson
96e6e680c1 Fix extra ; warnings involving MakeError 2019-11-10 11:24:47 -05:00
Peter Kolloch
2ba9f22715
De-duplicate struct PrimOp forward declaration 2019-11-10 10:02:22 +01:00
Eelco Dolstra
e491efe9fb
Use more stable registry URL 2019-11-06 14:20:16 +01:00
Eelco Dolstra
d5f1cc3e94
Use revcount/last-modified for computing the flake fingerprint
The store path is not enough. For example, when we build a dirty tree,
commit, and build the clean tree, a re-evaluation is necessary because
the flake may depend on the lastModified or revCount attributes.
2019-11-06 12:01:37 +01:00
Eelco Dolstra
88c452d160
Merge remote-tracking branch 'origin/master' into flakes 2019-11-06 10:56:33 +01:00
Eelco Dolstra
852554bb16
Merge branch 'nix-repl-e' of https://github.com/zimbatm/nix 2019-11-05 11:20:53 +01:00
Eelco Dolstra
72748b4088
Merge pull request #3173 from mkenigs/typo
fix typo
2019-11-05 11:17:26 +01:00
Eelco Dolstra
b81d9d26f5
Merge remote-tracking branch 'origin/master' into flakes 2019-11-04 22:29:31 +01:00
Benjamin Hipple
80d5ec6ff4 Minor updates to inline comments
Add missing docstring on InstallableCommand. Also, some of these were wrapped
when they're right next to a line longer than the unwrapped line, so we can just
unwrap them to save vertical space.
2019-10-31 05:56:37 -04:00
Janne Heß
2d1d1e3083 Replace git show with git log
git show seems to print the entire tag message when being called on a tag
instead of a commit. git log instead always prints the correct timestamp
in my tests.
The error nix prints is: `error: stoull`.
2019-10-30 17:15:38 +01:00
matthew
e0bcacf79f fix typo 2019-10-28 18:02:47 -05:00
zimbatm
9a25059656
findDerivationFilename: add FIXME 2019-10-28 21:40:02 +01:00
zimbatm
ec448f8bb6
libexpr: findDerivationFilename return Pos instead of tuple 2019-10-28 21:29:54 +01:00
Eelco Dolstra
e012384fe9
Merge branch 'tojson-tostring-fix' of https://github.com/mayflower/nix 2019-10-27 12:18:35 +01:00
Robin Gloster
e583df5280
builtins.toJSON: fix __toString usage 2019-10-27 10:15:51 +01:00
zimbatm
59c7249769
libexpr: add findDerivationFilename
extract the derivation to filename:lineno heuristic
2019-10-23 17:21:16 +02:00
Eelco Dolstra
1e23b82a53 exportGitHub(): Don't rely on the ETag from GitHub
We relied on it being the Git revision, but that stopped being the
case.
2019-10-21 23:14:29 +02:00
Eelco Dolstra
45b740c18b Use upstream json_fwd.hpp to speed up compilation 2019-10-21 22:11:21 +02:00
Eelco Dolstra
a07da2fd7a Don't ignore revs/refs of local flakerefs
Fixes

  error: the content hash of flake '/home/eelco/Dev/nixpkgs-flake?ref=HEAD&rev=0000000000000000000000000000000000000000' doesn't match the hash recorded in the referring lockfile
2019-10-21 14:57:01 +02:00
Eelco Dolstra
8e478c2341
Add experimental-features setting
Experimental features are now opt-in. There are currently two
experimental features: "nix-command" (which enables the "nix"
command), and "flakes" (which enables support for flakes). This will
allow us to merge experimental features more quickly, without
committing to supporting them indefinitely.

Typical usage:

$ nix build --experimental-features 'nix-command flakes' nixpkgs#hello
2019-10-16 17:49:01 +02:00
Eelco Dolstra
0ab64729e9 Improve GitHub caching
In particular, when building a flake lock file, inputs like 'nixpkgs'
are now downloaded only once. Previously, it would fetch
https://api.github.com/repos/<owner>/<repo>/tarball/<ref> and then
later https://api.github.com/repos/<owner>/<repo>/tarball/<rev>, even
though they produce the same result.

Git and GitHub now also share a cache that maps revs to a store path
and other info.
2019-10-16 00:20:51 +02:00
Eelco Dolstra
e99bb91217
Merge remote-tracking branch 'origin/master' into flakes 2019-10-10 12:54:37 +02:00
Eelco Dolstra
926d3e5bb0
Fix Bison 2.4 warning 2019-10-09 22:57:37 +02:00
Eelco Dolstra
99b73fb507
OCD performance fix: {find,count}+insert => insert 2019-10-09 16:06:29 +02:00
Eelco Dolstra
21304c11f9
uri -> url for consistency 2019-10-08 17:00:55 +02:00
Eelco Dolstra
a15f9b37eb
fetchGit: Support Git trees without any commits
Fixes

  $ nix build
  fatal: bad revision 'HEAD'
  error: program 'git' failed with exit code 128

on a new flake. It is now detected as a dirty tree with revCount = 0.
2019-10-07 15:44:32 +02:00
Eelco Dolstra
a323b7826c
Merge remote-tracking branch 'origin/master' into flakes 2019-10-04 17:26:32 +02:00
Eelco Dolstra
90d6018509
Fix aborts when using builtins.getFlake
In that case, 'self' could refer to a value on the stack, so accessing
'self.rev' would abort.
2019-10-02 22:08:19 +02:00
Eelco Dolstra
168a887916
Fix fetchTarball with chroot stores
Fixes #2405.
2019-10-01 07:51:06 +00:00
Eelco Dolstra
15b888c9a5
cmatch -> smatch 2019-09-27 15:31:09 +02:00
Eelco Dolstra
454e3a541a
Fix sorting of non-flake input attributes 2019-09-26 17:51:51 +02:00
Eelco Dolstra
bd79c1f6f6 Don't catch exceptions by value
(cherry picked from commit 893be6f5e3)
2019-09-22 21:56:56 +02:00
Eelco Dolstra
893be6f5e3 Don't catch exceptions by value 2019-09-22 21:29:33 +02:00
Eelco Dolstra
5961c94097 Flake alias -> id 2019-09-20 14:46:37 +02:00
Eelco Dolstra
68e0f23edc Add flags to disallow dirty Git trees and to turn off warnings 2019-09-20 14:29:49 +02:00
Eelco Dolstra
aeb7148afd
Some effort to minimize flake dependencies
For example, if the top-level flake depends on
"nixpkgs/release-19.03", and one of its dependencies depends on
"nixpkgs", then the latter will be mapped to "nixpkgs/release-19.03",
rather than whatever the default branch of "nixpkgs" is. Thus you get
only one "nixpkgs" dependency rather than two.

This currently only works in a breadth-first way, so the other way
around (i.e. if the top-level flake depends on "nixpkgs", and a
dependency depends on "nixpkgs/release-19.03") still results in two
"nixpkgs" dependencies.
2019-09-18 23:59:45 +02:00
zimbatm
619cc4af85
function-trace: always show the trace
If the user invokes nix with --trace-function-calls it means that they
want to see the trace.
2019-09-18 23:23:21 +02:00
Eelco Dolstra
c67407172d
Record original flakerefs in the lock file again
If 'input.<name>.uri' changes, then the entry in the lockfile for
input <name> should be considered stale.

Also print some messages when lock file entries are added/updated.
2019-09-18 21:57:57 +02:00
Eelco Dolstra
f97d3753a1
Require flake.nix to be an attrset (not a non-trivial thunk) 2019-09-09 17:34:38 +02:00
Eelco Dolstra
c87840ae14
Don't allow arbitrary computations in flake attributes
E.g. you can write 'edition = 201909' but not 'edition = 201909 + 0'.

Fixes #3075.
2019-09-09 16:34:44 +02:00
Eelco Dolstra
2fa7f2a56a
Use git+ prefix in flake URI schemes
Fixes #3045.
2019-09-05 17:15:09 +02:00
Eelco Dolstra
61fdb16aac
Improve error message when a directory is not a flake
So you now get

  $ nix build
  error: path '.' is not a flake (because it does not reference a Git repository)

rather than

  $ nix build
  error: unsupported argument '.'
2019-09-02 17:35:35 +02:00
Eelco Dolstra
a49b6761a5 Fix sourceInfo 2019-08-30 17:27:51 +02:00
Eelco Dolstra
80c36d4562 Remove 'name' attribute from flakes
This is no longer needed since flakes are given an identity in the
'inputs' attribute.
2019-08-30 16:38:27 +02:00
Eelco Dolstra
30ccf4e52d Turn flake inputs into an attrset
Instead of a list, inputs are now an attrset like

  inputs = {
    nixpkgs.uri = github:NixOS/nixpkgs;
  };

If 'uri' is omitted, than the flake is a lookup in the flake registry, e.g.

  inputs = {
    nixpkgs = {};
  };

but in that case, you can also just omit the input altogether and
specify it as an argument to the 'outputs' function, as in

  outputs = { self, nixpkgs }: ...

This also gets rid of 'nonFlakeInputs', which are now just a special
kind of input that have a 'flake = false' attribute, e.g.

  inputs = {
    someRepo = {
      uri = github:example/repo;
      flake = false;
    };
  };
2019-08-30 16:27:51 +02:00
Eelco Dolstra
2341f30ec6 Clean up the 'outputs' interface 2019-08-30 13:06:23 +02:00
Eelco Dolstra
89468410d5 Extract flake dependencies from the 'outputs' arguments
That is, instead of

  inputs = [ "nixpkgs" ];

  outputs = inputs: ... inputs.nixpkgs ...;

you can write

  outputs = { nixpkgs }: ... inputs.nixpkgs ...;
2019-08-30 11:22:34 +02:00
Eelco Dolstra
ebc4dae517 Merge remote-tracking branch 'origin/master' into flakes 2019-08-29 16:11:38 +02:00
Graham Christensen
ee9c988a1b
Track function start and ends for flame graphs
With this patch, and this file I called `log.py`:

    #!/usr/bin/env nix-shell
    #!nix-shell -i python3 -p python3 --pure

    import sys
    from pprint import pprint

    stack = []
    timestack = []

    for line in open(sys.argv[1]):
        components = line.strip().split(" ", 2)
        if components[0] != "function-trace":
            continue

        direction = components[1]
        components = components[2].rsplit(" ", 2)

        loc = components[0]
        _at = components[1]
        time = int(components[2])

        if direction == "entered":
            stack.append(loc)
            timestack.append(time)
        elif direction == "exited":
            dur = time - timestack.pop()
            vst = ";".join(stack)
            print(f"{vst} {dur}")
            stack.pop()

and:

    nix-instantiate --trace-function-calls -vvvv ../nixpkgs/pkgs/top-level/release.nix -A unstable > log.matthewbauer 2>&1
    ./log.py ./log.matthewbauer > log.matthewbauer.folded
    flamegraph.pl --title matthewbauer-post-pr log.matthewbauer.folded > log.matthewbauer.folded.svg

I can make flame graphs like: http://gsc.io/log.matthewbauer.folded.svg

---

Includes test cases around function call failures and tryEval. Uses
RAII so the finish is always called at the end of the function.
2019-08-14 16:09:35 -04:00
Eelco Dolstra
1d750e0587
Merge remote-tracking branch 'origin/master' into flakes 2019-08-08 15:49:13 +02:00
Bas van Dijk
89865144c3 Allow builtins.pathExists to check the existence of /nix/store paths
This makes it consitent with builtins.readDir.
2019-07-30 11:27:35 +02:00
Eelco Dolstra
b29cec7697
Don't write lock files if they have dirty inputs 2019-07-12 13:29:54 +02:00
Eelco Dolstra
bd62290c23
fetchGit: Warn about dirty trees 2019-07-12 12:59:00 +02:00
Eelco Dolstra
0802e006f2
Use "git add --force --intent-to-add" for flake.lock
Fixes

  The following paths are ignored by one of your .gitignore files:
  flake.lock
2019-07-11 17:05:53 +02:00
Eelco Dolstra
ad42a78469
Rename 'epoch' -> 'edition' 2019-07-11 13:54:53 +02:00
Eelco Dolstra
b0c220c02e
Check for epochs < 201906 2019-07-10 10:27:33 +02:00
Eelco Dolstra
cc218b15ba Merge remote-tracking branch 'origin/master' into flakes 2019-07-06 21:06:22 +02:00
Eelco Dolstra
33db1d35ae
Merge pull request #2582 from LnL7/fetchgit-refs
fetchGit: allow fetching explicit refs
2019-07-02 15:44:31 +02:00
Eelco Dolstra
2cc248c4fd
Merge remote-tracking branch 'origin/master' into flakes 2019-06-25 14:27:57 +02:00
Eelco Dolstra
64ec087f58
Fix 32-bit overflow with --no-net
--no-net causes tarballTtl to be set to the largest 32-bit integer,
which causes comparison like 'time + tarballTtl < other_time' to
fail on 32-bit systems. So cast them to 64-bit first.

https://hydra.nixos.org/build/95076624
(cherry picked from commit 29ccb2e969)
2019-06-24 22:16:43 +02:00
Eelco Dolstra
f8b30338ac
Refactor downloadCached() interface
(cherry picked from commit df3f5a78d5)
2019-06-24 22:12:26 +02:00
Eelco Dolstra
b43e1e186e
CachedDownloadResult: Include store path
Also, make fetchGit and fetchMercurial update allowedPaths properly.

(Maybe the evaluator, rather than the caller of the evaluator, should
apply toRealPath(), but that's a bigger change.)

(cherry picked from commit 5c34d66538)
2019-06-24 21:59:27 +02:00
Eelco Dolstra
dc29e9fb47
downloadCached: Return ETag
(cherry picked from commit 529add316c)
2019-06-24 21:58:33 +02:00
Eelco Dolstra
94f11d0a61
Fix abort in fromTOML
Fixes #2969.
2019-06-24 17:09:09 +02:00
Eelco Dolstra
d4fe9daed6
Simplify getFlake() / fetchFlake() logic 2019-06-21 19:04:58 +02:00
Eelco Dolstra
aa2846198f
Don't update the global registry when building a locked flake
It's unnecessary and slows things down (e.g. when you're on a Thalys
with super-crappy Internet).
2019-06-21 18:34:43 +02:00
Eelco Dolstra
d132d057a8
Handle store symlinks in flake directories
E.g. 'nix path-info ./result' inside a flake directory now works
again.
2019-06-21 15:29:05 +02:00
Eelco Dolstra
29ccb2e969
Fix 32-bit overflow with --no-net
--no-net causes tarballTtl to be set to the largest 32-bit integer,
which causes comparison like 'time + tarballTtl < other_time' to
fail on 32-bit systems. So cast them to 64-bit first.

https://hydra.nixos.org/build/95076624
2019-06-21 12:53:52 +02:00
Eelco Dolstra
e75ffbf04a
Merge pull request #2937 from CSVdB/fuzzymatching-v2
FuzzyMatching works
2019-06-18 18:03:38 +02:00
Nick Van den Broeck
59714a15e0 FuzzyMatching works
Fixes #2843
2019-06-18 17:01:57 +02:00
Eelco Dolstra
9d1207c02c
nix flake check: Check apps 2019-06-17 17:59:57 +02:00
Eelco Dolstra
06010eaf19
Fix fetchTarball with chroot stores
Fixes #2405.
2019-06-12 10:34:13 +02:00
Eelco Dolstra
69b047f4ce
writeRegistry(): Write correct version 2019-06-11 21:32:57 +02:00
Eelco Dolstra
c47d2dac6c
Disable EvalCache in impure mode 2019-06-07 22:38:39 +02:00
Eelco Dolstra
6644b6099b
Add flake evaluation cache
This exploits the hermetic nature of flake evaluation to speed up
repeated evaluations of a flake output attribute.

For example (doing 'nix build' on an already present package):

  $ time nix build nixpkgs:firefox

  real    0m1.497s
  user    0m1.160s
  sys     0m0.139s

  $ time nix build nixpkgs:firefox

  real    0m0.052s
  user    0m0.038s
  sys     0m0.007s

The cache is ~/.cache/nix/eval-cache-v1.sqlite, which has entries like

  INSERT INTO Attributes VALUES(
    X'92a907d4efe933af2a46959b082cdff176aa5bfeb47a98fabd234809a67ab195',
    'packages.firefox',
    1,
    '/nix/store/pbalzf8x19hckr8cwdv62rd6g0lqgc38-firefox-67.0.drv /nix/store/g6q0gx0v6xvdnizp8lrcw7c4gdkzana0-firefox-67.0 out');

where the hash 92a9... is a fingerprint over the flake store path and
the contents of the lockfile. Because flakes are evaluated in pure
mode, this uniquely identifies the evaluation result.
2019-06-07 22:25:48 +02:00
Eelco Dolstra
54aff8430c
Move flake-related stuff to src/libexpr/flake 2019-06-05 16:51:54 +02:00
Eelco Dolstra
1b05792988
Shorter syntax for referencing flake outputs
Fixes #2819.
2019-06-04 22:35:43 +02:00
Eelco Dolstra
ce225615c3
Eliminate duplicate fetching of the top-level flake 2019-06-04 21:10:53 +02:00
Eelco Dolstra
087530dec4
Add comments 2019-06-04 21:07:55 +02:00
Eelco Dolstra
1e53a07712
Make non-flake inputs lazy
Also add a proper test for non-flake inputs.
2019-06-04 20:56:42 +02:00
Eelco Dolstra
1c5067b9a7
Check hash 2019-06-04 20:35:35 +02:00
Eelco Dolstra
5fe7be2409
Rename dep -> input
Also use nlohmann::json range-based for.
2019-06-04 20:08:13 +02:00
Eelco Dolstra
9e99b5205c
Move LockFile and related types to a separate file 2019-06-04 20:01:21 +02:00
Eelco Dolstra
278114d559
Fix GC closure generation 2019-06-04 19:45:16 +02:00
Eelco Dolstra
6dbd5c26e6
Make flake input fetching lazy
As long as the flake input is locked, it is now only fetched when it
is evaluated (e.g. "nixpkgs" is fetched when
"inputs.nixpkgs.<something>" is evaluated).

This required adding an "id" attribute to the members of "inputs" in
lockfiles, e.g.

  "inputs": {
    "nixpkgs/release-19.03": {
      "id": "nixpkgs",
      "inputs": {},
      "narHash": "sha256-eYtxncIMFVmOHaHBtTdPGcs/AnJqKqA6tHCm0UmPYQU=",
      "nonFlakeInputs": {},
      "uri": "github:edolstra/nixpkgs/e9d5882bb861dc48f8d46960e7c820efdbe8f9c1"
    }
  }

because the flake ID needs to be known beforehand to construct the
"inputs" attrset.

Fixes #2913.
2019-06-04 19:17:03 +02:00
Eelco Dolstra
653c4e439b
Fix clang compilation error
https://hydra.nixos.org/build/94332344

https://stackoverflow.com/questions/46114214/lambda-implicit-capture-fails-with-variable-declared-from-structured-binding
2019-06-03 21:51:06 +02:00
Nick Van den Broeck
d9a6a75ed2 Made epochs more fine-grained
Fixes #2894
2019-06-03 14:47:47 +02:00
Eelco Dolstra
fb692e5f7b
Bindings: Add convenience method for requiring an attribute 2019-06-03 09:27:03 +02:00
Eelco Dolstra
8abb8647a3
Automatically determine subdir for path flakes
This means that in a flake in a subdirectory of a Git repo, you can
now do

  $ nix build

rather than the inconvenient

  $ nix build ../..?dir=foo/bar
2019-05-31 21:52:02 +02:00
Eelco Dolstra
ccb1bad612
Allow bare flakerefs as installables
So now

  $ nix build blender-bin

works and builds the default package from that flake. You don't need
to add a colon at the end anymore.
2019-05-31 21:42:23 +02:00
Eelco Dolstra
7adb10d29b
Fix reading the lockfile of a flake in a subdirectory 2019-05-31 20:12:59 +02:00
Eelco Dolstra
9169046e64
Add operator << for LockFile
Useful for debugging.
2019-05-31 20:10:56 +02:00
Eelco Dolstra
b971e406de
Support 'dir' and other parameters in path flakerefs 2019-05-31 19:01:11 +02:00
Eelco Dolstra
094539ef4a
Rename requires -> inputs, provides -> outputs
Issue #2828.
2019-05-31 09:59:48 +02:00
Eelco Dolstra
6ae4437acb
Remove makeFlakeValue() 2019-05-29 15:44:48 +02:00
Eelco Dolstra
6e4a8c47f4
Put flake-related stuff in its own namespace 2019-05-29 15:31:07 +02:00
Eelco Dolstra
c356d034f3
Make unsupported flake attributes a fatal error 2019-05-29 15:12:22 +02:00
Eelco Dolstra
6636808e90
Merge remote-tracking branch 'origin/master' into flakes 2019-05-29 12:36:44 +02:00
Eelco Dolstra
ae7b56cd9a
Get last commit time of github flakes 2019-05-29 10:10:40 +02:00
Eelco Dolstra
0f840483c7
Add date of last commit to SourceInfo
This is primarily useful for version string generation, where we need
a monotonically increasing number. The revcount is the preferred thing
to use, but isn't available for GitHub flakes (since it requires
fetching the entire history). The last commit timestamp OTOH can be
extracted from GitHub tarballs.
2019-05-29 10:10:36 +02:00
Eelco Dolstra
22f2744afd
Iterate over references 2019-05-28 23:05:08 +02:00
Eelco Dolstra
9eaebbf575
Merge branch 'attrPaths' of https://github.com/NinjaTrappeur/nix 2019-05-28 22:59:05 +02:00
Eelco Dolstra
ecee759b80
callFlake(): Emit source info attributes for non-flake dependencies 2019-05-28 14:01:08 +02:00
Eelco Dolstra
dda4f7167b
Remove redundant resolvedRef fields since they're already in SourceInfo 2019-05-28 13:12:43 +02:00
Eelco Dolstra
4846304541
Rename contentHash -> narHash for consistency 2019-05-28 13:08:40 +02:00
Eelco Dolstra
894e007445
Move hash into SourceInfo and rename to narHash to avoid ambiguity 2019-05-28 13:07:15 +02:00
Eelco Dolstra
6d7efcfaeb
Store SourceInfo in Flake and NonFlake
This deduplicates some shared fields. Factoring out the commonality is
useful in places like makeFlakeValue().
2019-05-28 13:06:43 +02:00
Eelco Dolstra
de36cf3db9
Merge branch 'nonFlakeRequiresTest' of https://github.com/CSVdB/nix into flakes 2019-05-28 12:05:11 +02:00
Nick Van den Broeck
4d030a8d96 Added nonFlakeRequires test
Fixes #2888
2019-05-28 10:51:45 +02:00
Nick Van den Broeck
6fb7545fa1 Fixed relative path parsing
Fixed #2821
2019-05-28 10:17:28 +02:00
Eelco Dolstra
90fe1dfd2f Register flake source trees as GC roots
This ensures that flakes don't get garbage-collected, which is
important to get nix-channel-like behaviour.

For example, running

  $ nix build hydra:

will create a GC root

  ~/.cache/nix/flake-closures/hydra -> /nix/store/xarfiqcwa4w8r4qpz1a769xxs8c3phgn-flake-closure

where the contents/references of the linked file in the store are the
flake source trees used by the 'hydra' flake:

  /nix/store/n6d5f5lkpfjbmkyby0nlg8y1wbkmbc7i-source
  /nix/store/vbkg4zy1qd29fnhflsv9k2j9jnbqd5m2-source
  /nix/store/z46xni7d47s5wk694359mq9ay353ar94-source

Note that this in itself is not enough to allow offline use; the
fetcher for the flakeref (e.g. fetchGit or downloadCached) must not
fail if it cannot fetch the latest version of the file, so long as it
knows a cached version.

Issue #2868.
2019-05-23 23:53:25 +02:00
Eelco Dolstra
6b77bfc28d FlakeRef::to_string(): Check round trip 2019-05-23 23:39:58 +02:00
Eelco Dolstra
6e984431dd fetchGit: Don't barf if we can't update our Git clone
Instead print a warning that we're continuing with the most recently
fetched version.
2019-05-23 23:38:40 +02:00
Eelco Dolstra
f0d6d67af9 Prevent the global registry from being GC'ed
Issue #2868.
2019-05-22 23:43:58 +02:00
Eelco Dolstra
df3f5a78d5 Refactor downloadCached() interface 2019-05-22 23:36:29 +02:00
Eelco Dolstra
66f1d7ee95 Fetch the flake registry from the NixOS/flake-registry repo 2019-05-22 22:56:46 +02:00
Eelco Dolstra
e414bde6f9 Check the flake epoch
Closes #2883.
2019-05-22 14:31:40 +02:00
Eelco Dolstra
70136a9bf4 Move flake-related flags into a separate class
Also, rename --dont-save-lock-file to --no-save-lock-file and change
noRegistries to useRegistries.
2019-05-22 14:04:18 +02:00
Eelco Dolstra
2468672e30 Improve FlakeCommand
It now handles commonality like calling getFlake() and resolving
relative local flake refs.

Fixes #2822.
2019-05-22 14:03:28 +02:00
Eelco Dolstra
5990b86391 Use warn(), tweak messages 2019-05-21 15:03:54 +02:00
Eelco Dolstra
20a1a65d37 Only rewrite the lockfile if it changed
This removes spurious warnings about failure to write the lockfile.
2019-05-21 14:55:43 +02:00
Nick Van den Broeck
ef6ae61503 Lockfile handling in resolveFlake is fixed 2019-05-17 14:50:10 +02:00
Nick Van den Broeck
98f20dee41 Give errors in resolveFlake
If DontUpdate but the lockfile isn't correct
2019-05-16 15:02:25 +02:00
Nick Van den Broeck
d9ad3723d5 Fixed issue 65
lockfile updating
2019-05-16 15:02:05 +02:00
Eelco Dolstra
5c34d66538 Make flakes work with 'nix build --store ...'
It was getting confused between logical and real store paths.

Also, make fetchGit and fetchMercurial update allowedPaths properly.

(Maybe the evaluator, rather than the caller of the evaluator, should
apply toRealPath(), but that's a bigger change.)
2019-05-15 15:38:24 +02:00
Nick Van den Broeck
4c9ebd20d7 One FIXME was already fixed 2019-05-15 08:10:46 +02:00
Nick Van den Broeck
d209bdcd08 Fixed issue #47
Content hashes
2019-05-09 14:27:55 +02:00
Eelco Dolstra
8fc1c3f413
Fix assertion failure in FlakeRef::to_string()
Fixes

  $ nix build
  nix: src/libexpr/primops/flakeref.cc:169: std::__cxx11::string nix::FlakeRef::to_string() const: Assertion `!rev' failed.
  Aborted

e.g. when flake.nix doesn't exist.

Also use gitRev().
2019-05-08 22:09:18 +02:00
Eelco Dolstra
455aa8d9ea
Add newline at end of lockfile
Suggested by @grahamc.
2019-05-08 18:28:01 +02:00
Eelco Dolstra
2bc55aba1e
Merge remote-tracking branch 'origin/master' into flakes 2019-05-08 14:30:27 +02:00
Eelco Dolstra
54e54db2e2
Merge remote-tracking branch 'tweag/flake-test' into flakes 2019-05-08 13:55:55 +02:00
Eelco Dolstra
77e1f9010c
Export missing rev/shortRev attributes 2019-05-08 13:38:32 +02:00
Eelco Dolstra
9d07c3717b
updateLockFile: Do "git add" in a slightly nicer way
"--intent-to-add" ensures the change shows up in "git diff".
2019-05-08 13:26:20 +02:00
Eelco Dolstra
ddd42b7e94 Fix immutable flakeref construction
We were appending ref/revs incorrectly for the IsGit case (by
appending /<ref>/<rev> rather than ?ref=<ref>&rev=<rev).
2019-05-07 23:32:09 +02:00
Eelco Dolstra
2a41a567e2 Improve FlakeRef::to_string()
We were incorrectly using path syntax (i.e. /<ref>/<rev>) for Git
repositories. This is only valid for GitHub flakerefs.
2019-05-07 23:32:09 +02:00
Eelco Dolstra
3c171851a8 Make the URL/path of the global flake registry configurable 2019-05-07 22:49:13 +02:00
Nick Van den Broeck
201f92e02c Fixed Flake data type and flake fetching 2019-05-07 05:57:08 +02:00
Daniel Schaefer
3f192ac80c Add builtins.hashFile
For text files it is possible to do it like so:
`builtins.hashString "sha256" (builtins.readFile /tmp/a)`
but that doesn't work for binary files.

With builtins.hashFile any kind of file can be conveniently hashed.
2019-05-03 17:23:36 +02:00
Eelco Dolstra
f8c4742c2f Fix 'git add' when subdir is empty 2019-05-03 13:15:13 +02:00
Eelco Dolstra
2aafa6901e Merge remote-tracking branch 'tweag/subdir' into flakes 2019-05-03 12:54:59 +02:00
Nick Van den Broeck
e0d4aa75fc Fixed compile errors 2019-05-02 08:40:00 +02:00
Eelco Dolstra
5d6e8c008b Allow 'dir' parameter in github: URIs
E.g. 'github:edolstra/dwarffs/flake?dir=foo/bar'.
2019-05-01 20:44:30 +02:00
Eelco Dolstra
a37436d792 Accept empty directories 2019-05-01 20:44:30 +02:00
Eelco Dolstra
fa88f71520 Validate 'dir=' parameters
We reject any path element starting with a '.' (mostly to reject '.'
and '..').
2019-05-01 20:44:30 +02:00
Eelco Dolstra
ab9e47284a Improve error message 2019-05-01 20:44:30 +02:00
Nick Van den Broeck
43408d3cd6 flake.lock now uses flakeRef.subdir 2019-05-01 16:24:33 +02:00
Eelco Dolstra
00db8d4549 Support 'dir' parameters in https and ssh flake URIs 2019-05-01 14:24:22 +02:00
Nick Van den Broeck
9b3069a88c Fuzzymatching
Fixed issue #61
2019-05-01 12:59:35 +02:00
Nick Van den Broeck
eba85e2367 WIP: still need to adapt flakeref parsing 2019-05-01 12:59:12 +02:00
Eelco Dolstra
4588a6ff3c Merge remote-tracking branch 'tweag/flakeFlags' into flakes 2019-04-30 12:46:54 +02:00
Nick Van den Broeck
24b35bf9e7 Fixed issue #13 2019-04-30 11:26:45 +02:00
Nick Van den Broeck
35d1c95f7f Fix flag registry order 2019-04-30 11:04:31 +02:00
Nick Van den Broeck
e9c42c06ef Fixed lookupFlake bug 2019-04-25 10:50:59 +02:00
Eelco Dolstra
21d5abfc14 Merge remote-tracking branch 'tweag/flake-clone' into flakes 2019-04-24 12:47:35 +02:00
Eelco Dolstra
4fb594a375 Merge remote-tracking branch 'tweag/resolvedFlake' into flakes 2019-04-19 15:56:17 +02:00
Nick Van den Broeck
e51abb6631 Changed some names 2019-04-19 14:23:35 +02:00
Eelco Dolstra
bc259192b4 fetchGit: Return revCount for dirty working trees 2019-04-19 14:15:51 +02:00
Eelco Dolstra
0cbda84f5b exportGit: Don't clone local repositories
This ensures that commands like 'nix flake info /my/nixpkgs' don't
copy a gigabyte of crap to ~/.cache/nix.

Fixes #60.
2019-04-19 14:10:57 +02:00
Eelco Dolstra
160ce18a0e Improve missing flake.nix error message 2019-04-19 11:43:56 +02:00
Eelco Dolstra
6960ee929d Clean up exportGit argument handling 2019-04-19 11:34:23 +02:00
Eelco Dolstra
46cb15df9b Fix assertion failure in FlakeRef::to_string() 2019-04-19 11:16:14 +02:00
Eelco Dolstra
6e4210d8ce Fix assertion failure 2019-04-19 10:58:08 +02:00
Eelco Dolstra
160b974fb0 Fix mutability check 2019-04-17 13:54:06 +02:00
Nick Van den Broeck
b42ba08fc8 Add command flake clone 2019-04-17 13:31:09 +02:00
Eelco Dolstra
939bee06cd Pass a flake to itself as "self" 2019-04-16 16:29:44 +02:00
Eelco Dolstra
3ddb6d1833 Allow refs to start with a digit
E.g. we want to accept "19.03" as a ref.
2019-04-16 16:24:51 +02:00
Eelco Dolstra
260527a90c Use the lock file 2019-04-16 16:18:47 +02:00
Eelco Dolstra
3d0e81051f Fix lock file generation
Before:

   "requires": {
        "nixpkgs": {
            "uri": "nixpkgs"
        }
    },

After:

    "requires": {
        "nixpkgs": {
            "uri": "github:edolstra/nixpkgs/f10e8a02eb7fa2b4a070f30cf87f4efcc7f3186d"
        }
    },
2019-04-16 15:43:55 +02:00
Eelco Dolstra
8c4e759efd updateLockFile(): Make sure Git can see flake.lock 2019-04-16 15:11:17 +02:00
Eelco Dolstra
60834492ae Update lock files from InstallableFlake::toValue()
This ensures that the lock file is updated *before* evaluating it, and
that it gets updated for any nix command, not just 'nix build'.

Also, while computing the lock file, allow arbitrary registry lookups,
not just at top-level.

Also, improve some error messages slightly.
2019-04-16 15:02:02 +02:00
Eelco Dolstra
7b312a8762 Pass stuff by reference 2019-04-16 14:27:54 +02:00
Eelco Dolstra
e1d73edb10 writeLockFile(): Emit empty objects rather than null 2019-04-16 14:23:10 +02:00
Eelco Dolstra
ed9d725392 getFlake(): Use impureIsAllowed
This fixes 'nix build nixpkgs:hello' without --impure.
2019-04-16 14:16:20 +02:00
Eelco Dolstra
aecf07b1d6 Remove dead function 2019-04-16 14:08:14 +02:00
Eelco Dolstra
035ac44354 Fix makeFlakeValue() 2019-04-16 13:56:08 +02:00
Eelco Dolstra
3a5493bfe8 Merge remote-tracking branch 'tweag/privateGithub' into flakes 2019-04-16 12:23:56 +02:00
Nick Van den Broeck
7587d62d02 Fixed flake pin issues 2019-04-16 08:38:17 +02:00
Nick Van den Broeck
b3d33b02e3 Added support for private github repositories 2019-04-16 08:14:46 +02:00
Eelco Dolstra
b4e367bf4a FlakeRef::to_string(): Drop the "flake:" prefix
This is unnecessary in most contexts and makes 'nix flake list' output
less readable.
2019-04-15 14:13:10 +02:00
Eelco Dolstra
c179f668e5 Slight cleanup 2019-04-15 14:08:18 +02:00
Eelco Dolstra
f6d684b5e2 getFlakeRegistries(): Return registries regardless of pureEval
This makes e.g. 'nix flake list' work.
2019-04-15 13:48:56 +02:00
Eelco Dolstra
84c12dbd7c Move --impure to MixEvalArgs 2019-04-15 13:45:51 +02:00
Eelco Dolstra
bb6e6923f2 Add environment variable NIX_SHOW_SYMBOLS for dumping the symbol table 2019-04-11 23:04:13 +02:00
Nick Van den Broeck
4ad4e48668 FlakeRegistry = FlakeRef -> FlakeRef 2019-04-11 07:08:04 +02:00
Nick Van den Broeck
c64f98b883 FlakeAlias is implemented 2019-04-09 10:16:02 +02:00
Nick Van den Broeck
f39670c631 Took ref and rev out of FlakeRef 2019-04-09 09:51:42 +02:00
Nick Van den Broeck
3ec0c82fab Fixed dependency resolution 2019-04-09 09:51:42 +02:00
Nick Van den Broeck
641db127be FlakeIds are now properly looked up in registries 2019-04-09 09:51:36 +02:00
Nick Van den Broeck
18c019b616 Added nonFlakeRequires and the command nix flake deps 2019-04-09 08:38:09 +02:00
Eelco Dolstra
c996e04aca Allow relative paths in flakerefs
Also allow "." as an installable to refer to the flake in the current
directory. E.g.

  $ nix build .

will build 'provides.defaultPackage' in the flake in the current
directory.
2019-04-08 23:52:09 +02:00
Eelco Dolstra
ee1254d4f5 nix: Add --impure as a shorter alias of --no-pure-eval 2019-04-08 23:19:19 +02:00
Eelco Dolstra
6a4c7fb975 Add path flakeref variant
Unlike file://<path>, this allows the path to be a dirty Git tree, so

  nix build /path/to/flake:attr

is a convenient way to test building a local flake.
2019-04-08 23:09:18 +02:00
Eelco Dolstra
9b7eac332b Merge remote-tracking branch 'origin/master' into flakes 2019-04-08 14:23:48 +02:00
Eelco Dolstra
154244adc6 nix: New installables syntax
The general syntax for an installable is now
<flakeref>:<attrpath>. The attrpath is relative to the flake's
'provides.packages' or 'provides' if the former doesn't yield a
result. E.g.

  $ nix build nixpkgs:hello

is equivalent to

  $ nix build nixpkgs:packages.hello

Also, '<flakeref>:' can be omitted, in which case it defaults to
'nixpkgs', e.g.

  $ nix build hello
2019-04-08 14:21:13 +02:00
Eelco Dolstra
6b0ca8e803 findAlongAttrPath(): Throw AttrPathNotFound 2019-04-08 14:20:58 +02:00
Eelco Dolstra
f32fbf952d Fix Bison 3.3 warning 2019-03-27 21:09:31 +01:00
Eelco Dolstra
c101b29133 Merge remote-tracking branch 'tweag/flake-registries' into flakes 2019-03-27 14:40:36 +01:00
Eelco Dolstra
f9c7176a87 nix flake add: Handle ~/.config/nix not existing
Fixes

  $ nix flake add fnord github:edolstra/fnord
  error: opening file '/home/eelco/.config/nix/registry.json': No such file or directory
2019-03-26 12:48:57 +01:00
Eelco Dolstra
42be60c6af Merge remote-tracking branch 'tweag/flake-add' into flakes 2019-03-26 12:35:27 +01:00
Eelco Dolstra
edb3836696 Merge remote-tracking branch 'tweag/auto-update-flake' into flakes 2019-03-26 11:50:22 +01:00
Eelco Dolstra
b5565a7081 Merge remote-tracking branch 'origin/master' into flakes 2019-03-26 11:44:14 +01:00
zimbatm
514b3c7f83
Add isPath primop
this is added for completeness' sake since all the other possible
`builtins.typeOf` results have a corresponding `builtins.is<Type>`
2019-03-24 11:36:49 +01:00
Nick Van den Broeck
a554f523db Combining registries properly 2019-03-23 13:42:48 +01:00
Nick Van den Broeck
5e4d92d267 Issue #15 is finished 2019-03-22 15:22:18 +01:00
Nick Van den Broeck
e007f367bd Fixed minor things 2019-03-22 11:21:49 +01:00
Nick Van den Broeck
d4ee8afd59 Implemented --flake flag for nix build
Also fixed Eelco's PR comments
2019-03-22 11:16:20 +01:00
Linus Heckemann
2aa89daab3 eval: improve type description for primops and applied primops
This can make type errors a little easier to understand.
2019-03-21 15:31:46 +01:00
Eelco Dolstra
1f64f4c7c8
pkg-config files: Use c++17 2019-03-14 14:11:12 +01:00
Eelco Dolstra
ef52ccf035
experimental/optional -> optional 2019-03-14 14:10:52 +01:00
Nick Van den Broeck
6542de98c2 Implemented writeRegistry 2019-03-07 12:00:10 +01:00
Nick Van den Broeck
9ff1a9ea65 Implemented json flag for nix flake info 2019-03-07 11:56:43 +01:00
Nick Van den Broeck
cfb6ab80ce Implemented "nix flake info" 2019-02-26 13:12:51 +01:00
Eelco Dolstra
d342de02b9
fetchFlake: Use infinite TTL when the revision is specified 2019-02-25 23:23:45 +08:00
Eelco Dolstra
529add316c
downloadCached: Return ETag
This allows fetchFlake() to return the Git revision of a GitHub
archive.
2019-02-25 23:20:50 +08:00
Eelco Dolstra
6e9182fbc2
Add basic flake lock file support 2019-02-12 22:43:22 +01:00
Eelco Dolstra
e38ec77de8
Interpret all file:// URIs as Git repositories 2019-02-12 22:06:19 +01:00
Eelco Dolstra
beab05851b
nix: Add --flake flag
This allows using an arbitrary "provides" attribute from the specified
flake. For example:

  nix build --flake nixpkgs packages.hello

(Maybe provides.packages should be used for consistency...)
2019-02-12 21:55:43 +01:00
Eelco Dolstra
272b58220d
Enforce use of immutable flakes in pure mode
... plus a temporary hack to allow impure flakes at top-level for the
default installation source.
2019-02-12 21:07:48 +01:00
Eelco Dolstra
ba05f29838
nix: Enable pure mode by default
We want to encourage a brave new world of hermetic evaluation for
source-level reproducibility, so flakes should not poke around in the
filesystem outside of their explicit dependencies.

Note that the default installation source remains impure in that it
can refer to mutable flakes, so "nix build nixpkgs.hello" still works
(and fetches the latest nixpkgs, unless it has been pinned by the
user).

A problem with pure evaluation is that builtins.currentSystem is
unavailable. For the moment, I've hard-coded "x86_64-linux" in the
nixpkgs flake. Eventually, "system" should be a flake function
argument.
2019-02-12 20:35:03 +01:00
Eelco Dolstra
91a6a47b0e
Improve flake references 2019-02-12 18:23:11 +01:00
Eelco Dolstra
0cd7f2cd8d
pkg-config files: Use c++17 2019-02-12 13:44:20 +01:00
Eelco Dolstra
c8a0b9d5cb
experimental/optional -> optional 2019-02-12 13:43:32 +01:00
Eelco Dolstra
52419f8db3
Add flake registry
This will eventually be moved to nixos.org.
2019-02-11 13:01:39 +01:00
Eelco Dolstra
dcae46ab14
Add github URIs
For example, github:edolstra/dwarffs is more-or-less equivalent to
https://github.com/edolstra/dwarffs.git. It's a much faster way to get
GitHub repositories: it fetches tarballs rather than entire Git
repositories. It also allows fetching specific revisions by hash
without specifying a ref (e.g. a branch name):

  github:edolstra/dwarffs/41c0c1bf292ea3ac3858ff393b49ca1123dbd553
2019-02-11 12:05:17 +01:00
Eelco Dolstra
ef4cf4e681
Introduce flake URIs 2019-02-11 12:05:13 +01:00
Eelco Dolstra
7a5cf31060
Initial flake support 2019-02-11 12:00:13 +01:00
Eelco Dolstra
f216c76c56
Bindings::get(): Add convenience method
This allows writing attribute lookups as

    if (auto name = value.attrs->get(state.sName))
      ...
2019-02-11 11:55:35 +01:00
Eelco Dolstra
c02da99757
EvalState::allocAttr(): Add convenience method 2019-02-11 11:55:29 +01:00
Eelco Dolstra
aa0e2a2e70
Make constant primops lazy 2019-02-11 11:52:07 +01:00
Shea Levy
b30be6b450
Add builtins.appendContext.
A partner of builtins.getContext, useful for the same reasons.
2019-01-31 08:52:23 -05:00
Shea Levy
1d757292d0
Add builtins.getContext.
This can be very helpful when debugging, as well as enabling complex
black magic like surgically removing a single dependency from a
string's context.
2019-01-14 11:27:10 -05:00
Shea Levy
087be7281a
Treat plain derivation paths in context as normal paths.
Previously, plain derivation paths in the string context (e.g. those
that arose from builtins.storePath on a drv file, not those that arose
from accessing .drvPath of a derivation) were treated somewhat like
derivaiton paths derived from .drvPath, except their dependencies
weren't recursively added to the input set. With this change, such
plain derivation paths are simply treated as paths and added to the
source inputs set accordingly, simplifying context handling code and
removing the inconsistency. If drvPath-like behavior is desired, the
.drv file can be imported and then .drvPath can be accessed.

This is a backwards-incompatibility, but storePath is never used on
drv files within nixpkgs and almost never used elsewhere.
2019-01-13 11:29:55 -05:00
Eelco Dolstra
c7bf1cdb4e
Merge pull request #2608 from dtzWill/fix/issue-2546
EvalState::resetFileCache: clear parse cache as well as eval cache
2019-01-10 20:56:31 +01:00
John Ericson
fef9f5653b Remove mentions of libformat, it no longer exists 2019-01-05 14:31:29 -05:00
Will Dietz
21ea00d3ec EvalState::resetFileCache: clear parse cache as well as eval cache
Fixes #2546.

(at least the basic reproduction I've been testing)
2018-12-31 10:18:28 -06:00
Daiderd Jordan
7e35e914c1
fetchGit: allow fetching explicit refs
Trying to fetch refs that are not in refs/heads currently fails because
it looks for refs/heads/refs/foo instead of refs/foo.

eg.

	builtins.fetchGit {
	  url = https://github.com/NixOS/nixpkgs.git;
	  ref = "refs/pull/1024/head;
	}
2018-12-14 20:12:22 +01:00
Eelco Dolstra
6024dc1d97
Support SRI hashes
SRI hashes (https://www.w3.org/TR/SRI/) combine the hash algorithm and
a base-64 hash. This allows more concise and standard hash
specifications. For example, instead of

  import <nix/fetchurl.nl> {
    url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz;
    sha256 = "5d22dad058d5c800d65a115f919da22938c50dd6ba98c5e3a183172d149840a4";
  };

you can write

  import <nix/fetchurl.nl> {
    url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz;
    hash = "sha256-XSLa0FjVyADWWhFfkZ2iKTjFDda6mMXjoYMXLRSYQKQ=";
  };

In fixed-output derivations, the outputHashAlgo is no longer mandatory
if outputHash specifies the hash (either as an SRI or in the old
"<type>:<hash>" format).

'nix hash-{file,path}' now print hashes in SRI format by default. I
also reverted them to use SHA-256 by default because that's what we're
using most of the time in Nixpkgs.

Suggested by @zimbatm.
2018-12-13 14:30:52 +01:00
CHEIKH Chawki
fa5143c722 Solve hg "abandoned transaction" issue 2018-12-06 13:57:59 +01:00
Eelco Dolstra
4aee93d5ce
fetchGit: Drop unnecessary localRef 2018-11-20 20:59:44 +01:00
Eelco Dolstra
3f4de91d80
Merge branch 'better-git-cache' of https://github.com/graham-at-target/nix 2018-11-20 20:41:19 +01:00
Jan Path
d1b049c4ea Fix typo in comments 2018-10-31 20:50:18 +01:00
Guillaume Maudoux
6a5bf9b143 simplify handling of extra '}' 2018-10-27 00:14:51 +02:00
Eelco Dolstra
8327a7a8fa
Merge branch 'dirOf-relative' of https://github.com/lheckemann/nix 2018-09-13 14:33:12 +02:00
Eelco Dolstra
91405986f4
Convert NIX_COUNT_CALLS to JSON too 2018-09-05 21:57:54 +02:00
Eelco Dolstra
0a2545f95c
Log stats to stderr
We shouldn't pollute stdout.
2018-09-05 21:35:58 +02:00
Michael Bishop
4b034f390c remove the old text format output 2018-09-02 18:25:23 -03:00
Michael Bishop
2fd1008c70 add JSON to NIX_SHOW_STATS 2018-09-01 20:05:06 -03:00
Michael Bishop
c29e5fbb13 improve the stats when profiling 2018-09-01 17:11:56 -03:00
Eelco Dolstra
475a0a54a9
fetchGit/fetchMercurial: Don't absolutize paths
This is already done by coerceToString(), provided that the argument
is a path (e.g. 'fetchGit ./bla'). It fixes the handling of URLs like
git@github.com:owner/repo.git. It breaks 'fetchGit "./bla"', but that
was never intended to work anyway and is inconsistent with other
builtin functions (e.g. 'readFile "./bla"' fails).
2018-09-01 00:19:49 +02:00
aszlig
0ad643ed5c
libexpr: Use int64_t for NixInt
Using a 64bit integer on 32bit systems will come with a bit of a
performance overhead, but given that Nix doesn't use a lot of integers
compared to other types, I think the overhead is negligible also
considering that 32bit systems are in decline.

The biggest advantage however is that when we use a consistent integer
size across all platforms it's less likely that we miss things that we
break due to that. One example would be:

https://github.com/NixOS/nixpkgs/pull/44233

On Hydra it will evaluate, because the evaluator runs on a 64bit
machine, but when evaluating the same on a 32bit machine it will fail,
so using 64bit integers should make that consistent.

While the change of the type in value.hh is rather easy to do, we have a
few more options available for doing the conversion in the lexer:

  * Via an #ifdef on the architecture and using strtol() or strtoll()
    accordingly depending on which architecture we are. For the #ifdef
    we would need another AX_COMPILE_CHECK_SIZEOF in configure.ac.
  * Using istringstream, which would involve copying the value.
  * As we're already using boost, lexical_cast might be a good idea.

Spoiler: I went for the latter, first of all because lexical_cast does
have an overload for const char* and second of all, because it doesn't
involve copying around the input string. Also, because istringstream
seems to come with a bigger overhead than boost::lexical_cast:

https://www.boost.org/doc/libs/release/doc/html/boost_lexical_cast/performance.html

The first method (still using strtol/strtoll) also wasn't something I
pursued further, because it is also locale-aware which I doubt is what
we want, given that the regex for int is [0-9]+.

Signed-off-by: aszlig <aszlig@nix.build>
Fixes: #2339
2018-08-29 01:05:52 +02:00
Eelco Dolstra
c651b7bdc9
Revert "Fix parser/lexer generation with parallel make"
This reverts commit d277442df5.

Make sucks.
2018-08-23 00:23:35 +02:00
Eelco Dolstra
9b1bdf2db8
FIx floating point evaluation
Fixes #2361.
2018-08-19 11:59:49 +02:00
Graham Christensen
02098d2073 fetchGit: use a better caching scheme
The current usage technically works by putting multiple different
repos in to the same git directory. However, it is very slow as
Git tries very hard to find common commits between the two
repositories. If the two repositories are large (like Nixpkgs and
another long-running project,) it is maddeningly slow.

This change busts the cache for existing deployments, but users
will be promptly repaid in per-repository performance.
2018-08-17 11:27:34 -04:00
Eelco Dolstra
d277442df5
Fix parser/lexer generation with parallel make
Fun fact: rules with multiple targets don't work properly with 'make
-j'. For example, a rule like

  a b: c
    touch a b

is equivalent to

  a: c
    touch a b

  b: c
    touch a b

so with 'make -j', the 'touch' command will be run twice. See
e.g. https://stackoverflow.com/questions/2973445/gnu-makefile-rule-generating-a-few-targets-from-a-single-source-file.
2018-08-17 12:59:23 +02:00
Linus Heckemann
d7402c9cd5 dirOf: allow use on non-absolute paths 2018-08-13 11:27:35 +02:00
Eelco Dolstra
bc65e02d96
Merge pull request #2326 from aszlig/fix-symlink-leak
Fix symlink leak in restricted eval mode
2018-08-03 17:01:34 +02:00
aszlig
43e28a1b75
Fix symlink leak in restricted eval mode
In EvalState::checkSourcePath, the path is checked against the list of
allowed paths first and later it's checked again *after* resolving
symlinks.

The resolving of the symlinks is done via canonPath, which also strips
out "../" and "./". However after the canonicalisation the error message
pointing out that the path is not allowed prints the symlink target in
the error message.

Even if we'd suppress the message, symlink targets could still be leaked
if the symlink target doesn't exist (in this case the error is thrown in
canonPath).

So instead, we now do canonPath() without symlink resolving first before
even checking against the list of allowed paths and then later do the
symlink resolving and checking the allowed paths again.

The first call to canonPath() should get rid of all the "../" and "./",
so in theory the only way to leak a symlink if the attacker is able to
put a symlink in one of the paths allowed by restricted evaluation mode.

For the latter I don't think this is part of the threat model, because
if the attacker can write to that path, the attack vector is even
larger.

Signed-off-by: aszlig <aszlig@nix.build>
2018-08-03 06:46:43 +02:00
Eelco Dolstra
45bcf5416a
Merge branch 'prim_mapAttr-fix' of https://github.com/volth/nix 2018-07-31 20:05:07 +02:00
Eelco Dolstra
a7fb7d3cde
Merge pull request #2303 from volth/patch-4
parser.y: fix assoc of -> and < > <= >=
2018-07-23 11:38:15 +02:00