Commit graph

218 commits

Author SHA1 Message Date
regnat
7d651f5c3f throw_ -> reportError
This function might or might not throw depending on the value of
`keepGoing`, so naming it `throw_` was a bit confusing
2021-06-02 11:24:31 +02:00
regnat
d12b12a15b Let nix flake check keep going when keep-going is set
When the `keep-going` option is set to `true`, make `nix flake check`
continue as much as it can before failing.

The UI isn’t perfect as-it-is as all the lines currently start with a
mostly useless `error (ignored): error:` prefix, but I’m not sure what
the best output would be, so I’ll leave it as-it-is for the time being

(This is a bit hijacking the `keep-going` flag as it’s supposed to be a
build-time only thing. But I think it’s faire to reuse it here).

Fix https://github.com/NixOS/nix/issues/4450
2021-06-02 11:13:12 +02:00
John Ericson
9b805d36ac Rename Buildable 2021-04-05 09:52:25 -04:00
John Ericson
255d145ba7 Use BuildableReq for buildPaths and ensurePath
This avoids an ambiguity where the `StorePathWithOutputs { drvPath, {}
}` could mean "build `brvPath`" or "substitute `drvPath`" depending on
context.

It also brings the internals closer in line to the new CLI, by
generalizing the `Buildable` type is used there and makes that
distinction already.

In doing so, relegate `StorePathWithOutputs` to being a type just for
backwards compatibility (CLI and RPC).
2021-04-05 08:33:00 -04:00
Eelco Dolstra
ef83ced4e1 Restore 'nix flake info' as a deprecated alias 2021-03-19 17:21:37 +01:00
Eelco Dolstra
66fa1c7375 Merge 'nix flake {info,list-inputs}' into 'nix flake metadata'
Fixes #4613.
2021-03-16 17:19:04 +01:00
Eelco Dolstra
77f5d171e1 --override-input: Imply --no-write-lock-file
Fixes #3779.
2021-03-16 16:53:39 +01:00
Eelco Dolstra
453c3a603f nix flake update: Recreate the lock file
This is probably what most people expect it to do. Fixes #3781.

There is a new command 'nix flake lock' that has the old behaviour of
'nix flake update', i.e. it just adds missing lock file entries unless
overriden using --update-input.
2021-02-26 14:55:54 +01:00
Eelco Dolstra
13897afbe6 Throw an error if --arg / --argstr is used with a flake
Fixes #3949.
2021-02-17 17:32:10 +01:00
Eelco Dolstra
cced73496b nix flake show: Handle 'overlays' output
Fixes #4542.
2021-02-17 16:53:39 +01:00
Eelco Dolstra
3da9a9241c Convert option descriptions to Markdown 2021-01-13 14:18:04 +01:00
Eelco Dolstra
93ad6430ed nix store prefetch-tarball -> nix flake prefetch 2021-01-11 12:36:39 +01:00
Eelco Dolstra
c9279b831e
Add 'nix flake' manpages 2020-12-23 18:26:40 +01:00
Silvan Mosberger
12e65078ef
Rename Value::normalType() -> Value::type() 2020-12-17 14:45:45 +01:00
Silvan Mosberger
bf98903967
Add ValueType checking functions for types that have the same NormalType 2020-12-12 03:31:50 +01:00
Silvan Mosberger
22ead43a0b
Use Value::normalType on all forced values instead of Value::type 2020-12-12 03:31:48 +01:00
Matthew Kenigsberg
6a4bf535d8 Capitalize JSON for consistency 2020-10-28 17:54:28 -05:00
Eelco Dolstra
9d5e9ef0da Move Explicit 2020-10-26 17:01:20 +01:00
John Ericson
39de73550d Merge remote-tracking branch 'upstream/master' into fix-and-ci-static-builds 2020-10-09 18:26:47 +00:00
Eelco Dolstra
85c8be6286 Remove static variable name clashes
This was useful for an experiment with building Nix as a single
compilation unit. It's not very useful otherwise but also doesn't
hurt...
2020-10-06 13:49:20 +02:00
John Ericson
cfe791a638 stdout_ -> cout
Better to get creative than just sprinkle arbitrary underscores.
2020-09-25 11:30:04 -04:00
John Ericson
25f7ff16fa Merge remote-tracking branch 'upstream/master' into fix-and-ci-static-builds 2020-09-04 02:40:36 +00:00
Eelco Dolstra
a72a20d68f
Add 'nix dump-args' to dump all commands/flags for manpage generation 2020-08-17 17:44:52 +02:00
John Ericson
39ae9a3d4a Merge remote-tracking branch 'upstream/master' into fix-and-ci-static-builds 2020-08-07 14:44:47 +00:00
Eelco Dolstra
2ffc058950 Make --no-eval-cache a global setting 2020-08-07 14:13:24 +02:00
Matthew Bauer
fa2d1fb36e Pass system to bundler 2020-07-30 15:37:05 -05:00
Matthew Bauer
52407f83a1 exporter -> bundler 2020-07-30 15:16:23 -05:00
Matthew Bauer
d7ffe327ae Merge remote-tracking branch 'origin/master' into fix-and-ci-static-builds 2020-07-30 14:59:57 -05:00
Matthew Bauer
5d04a4db9b Handle exporters checking correctly 2020-07-30 11:45:47 -05:00
Matthew Bauer
2f4250a416 Add "export" to Nix
This adds a ‘nix export’ command which hooks into nix-bundle. It can
be used in a similar way as nix-bundle, with the benefit of hooking
into the new “app” functionality. For instance,

$ nix export nixpkgs#jq
$ ./jq --help
jq - commandline JSON processor [version 1.6]
...
$ scp jq machine-without-nix:
$ ssh machine-without-nix ./jq
jq - commandline JSON processor [version 1.6]
...

Note that nix-bundle currently requires Linux to run. Other exporters
might not have that requirement.

“exporters” are meant to be reusable, so that, other repos can
implement their own bundling.

Fixes #3705
2020-07-30 11:33:22 -05:00
Eelco Dolstra
54712aaf8a Merge remote-tracking branch 'origin/master' into flakes 2020-07-06 16:40:10 +02:00
Eelco Dolstra
26cf0c674f nix run: Use packages/legacyPackages as fallback if there is no app definition
'nix run' will try to run $out/bin/<name>, where <name> is the
derivation name (excluding the version). This often works well:

  $ nix run nixpkgs#hello
  Hello, world!

  $ nix run nix -- --version
  nix (Nix) 2.4pre20200626_adf2fbb

  $ nix run patchelf -- --version
  patchelf 0.11.20200623.e61654b

  $ nix run nixpkgs#firefox -- --version
  Mozilla Firefox 77.0.1

  $ nix run nixpkgs#gimp -- --version
  GNU Image Manipulation Program version 2.10.14

though not always:

  $ nix run nixpkgs#git
  error: unable to execute '/nix/store/kp7wp760l4gryq9s36x481b2x4rfklcy-git-2.25.4/bin/git-minimal': No such file or directory
2020-06-29 19:08:50 +02:00
Eelco Dolstra
09fc06daab nix flake init: Use git add --force 2020-06-23 16:25:32 +02:00
Eelco Dolstra
334e26bfc2 nix flake check: Don't build apps
This was inconsistent since we're not building 'packages' or
'defaultPackage' either.

Closes #3726.
2020-06-22 11:31:07 +02:00
Eelco Dolstra
1524752c17 Merge remote-tracking branch 'origin/master' into flakes 2020-06-17 10:26:52 +02:00
Eelco Dolstra
0c62b4ad0f Represent 'follows' inputs explicitly in the lock file
This fixes an issue where lockfile generation was not idempotent:
after updating a lockfile, a "follows" node would end up pointing to a
new copy of the node, rather than to the original node.
2020-06-11 14:40:21 +02:00
Eelco Dolstra
e073f2c584 nix flake: Require 'flakes' feature 2020-06-08 16:23:54 +02:00
Eelco Dolstra
6470450ab4 Add completion for --update-input 2020-06-08 16:20:00 +02:00
Eelco Dolstra
488ff83e6b Fix completion of --template 2020-06-05 14:09:12 +02:00
Eelco Dolstra
810b2c6a48 nix flake init: Add a '--template' flag
The initial contents of the flake is specified by the
'templates.<name>' or 'defaultTemplate' output of another flake. E.g.

  outputs = { self }: {

    templates = {

      nixos-container = {
        path = ./nixos-container;
        description = "An example of a NixOS container";
      };

    };

  };

allows

  $ nix flake init -t templates#nixos-container

Also add a command 'nix flake new', which is identical to 'nix flake
init' except that it initializes a specified directory rather than the
current directory.
2020-06-04 20:22:25 +02:00
Eelco Dolstra
950b46821f Remove TreeInfo
The attributes previously stored in TreeInfo (narHash, revCount,
lastModified) are now stored in Input. This makes it less arbitrary
what attributes are stored where.

As a result, the lock file format has changed. An entry like

    "info": {
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },
    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github"
    },

is now stored as

    "locked": {
      "owner": "NixOS",
      "repo": "nixpkgs",
      "rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
      "type": "github",
      "lastModified": 1585405475,
      "narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
    },

The 'Input' class is now a dumb set of attributes. All the fetcher
implementations subclass InputScheme, not Input. This simplifies the
API.

Also, fix substitution of flake inputs. This was broken since lazy
flake fetching started using fetchTree internally.
2020-05-30 00:44:11 +02:00
Matthew Kenigsberg
8d67794da1 handle circular flake dependencies in list-inputs 2020-05-21 17:06:11 -06:00
Eelco Dolstra
5f64655ff4 Move registry-related commands from 'nix flake' to 'nix registry'
This makes 'nix flake' less cluttered and more consistent (it's only
subcommands that operator on a flake). Also, the registry is not
inherently flake-related (e.g. fetchTree could also use it to remap
inputs).
2020-05-15 14:38:10 +02:00
Eelco Dolstra
649c2db308 nix flake: Add completion support 2020-05-11 22:10:33 +02:00
Eelco Dolstra
6f3244ce45 Merge remote-tracking branch 'origin/master' into flakes 2020-05-05 18:59:33 +02:00
Eelco Dolstra
6521c92ce8 Improve path:// handling
In particular, doing 'nix build /path/to/dir' now works if
/path/to/dir is not a Git tree (it only has to contain a flake.nix
file).

Also, 'nix flake init' no longer requires a Git tree (but it will do a
'git add flake.nix' if it's a Git tree)
2020-04-27 22:53:11 +02:00
Eelco Dolstra
b69323f8c9 Revive 'nix search'
It uses the evaluation cache now rather than the ad hoc JSON cache.
2020-04-20 15:27:09 +02:00
Eelco Dolstra
42a12f9232 Move eval-cache.{cc,hh} 2020-04-20 13:14:59 +02:00
Eelco Dolstra
539a9c1c5f Get rid of the old eval cache 2020-04-20 13:13:52 +02:00
Eelco Dolstra
0725ab2fd7 Store more stuff in the evaluation cache
In particular, we store whether an attribute failed to evaluate (threw
an exception) or was an unsupported type. This is to ensure that a
repeated 'nix flake show' never has to evaluate anything, so it can
execute without fetching the flake.

With this, 'nix flake show nixpkgs/nixos-20.03 --legacy' executes in
0.6s (was 3.4s).
2020-04-19 23:07:06 +02:00
Eelco Dolstra
3738bcb05e Eval cache: Don't replace real attributes with placeholders 2020-04-18 15:12:31 +02:00
Eelco Dolstra
69cb9f7eee Wrap eval cache creation in a giant transaction
This speeds up the creation of the cache for the nixpkgs flake from
21.2s to 10.2s. Oddly, it also speeds up querying the cache
(i.e. running 'nix flake show nixpkgs/nixos-20.03 --legacy') from 4.2s
to 3.4s.

(For comparison, running with --no-eval-cache takes 9.5s, so the
overhead of building the SQLite cache is only 0.7s.)
2020-04-17 23:17:21 +02:00
Eelco Dolstra
aaa109565e Use a more space/time-efficient representation for the eval cache 2020-04-17 23:04:21 +02:00
Eelco Dolstra
bdb3226607 Add flag to disable the eval cache 2020-04-17 14:30:04 +02:00
Eelco Dolstra
aa34c0ef51 nix flake show: Speed up eval cache bigly
In the fully cached case for the 'nixpkgs' flake, it went from 101s to
4.6s. Populating the cache went from 132s to 17.4s (which could
probably be improved further by combining INSERTs).
2020-04-17 13:57:02 +02:00
Eelco Dolstra
9ea4f93f88 nix flake show: Support apps 2020-04-17 01:21:24 +02:00
Eelco Dolstra
12b7eefbc5 nix flake show: Use evaluation cache 2020-04-17 01:02:29 +02:00
Eelco Dolstra
3b489e8843 Add 'nix flake show' command 2020-04-16 19:52:39 +02:00
Eelco Dolstra
29043e7e9e Fix 2020-04-16 19:01:49 +02:00
Eelco Dolstra
8f41847394 Use Logger::stdout() 2020-04-16 13:47:59 +02:00
Eelco Dolstra
e5ea01c1a8
Remove flake 'edition' field
Future editions of flakes or the Nix language can be supported by
renaming flake.nix (e.g. flake-v2.nix). This avoids a bootstrap
problem where we don't know which grammar to use to parse
flake*.nix. It also allows a project to support multiple flake
editions, in theory.
2020-04-10 10:24:09 +02:00
Eelco Dolstra
68b43e01dd
nix flake info: Show resolved URL
This is useful for finding out what a registry lookup resolves to, e.g

  $ nix flake info patchelf
  Resolved URL:  github:NixOS/patchelf
  Locked URL:    github:NixOS/patchelf/cd7955af31698c571c30b7a0f78e59fd624d0229
2020-04-06 14:56:13 +02:00
Eelco Dolstra
ce3173edc1
nix flake info --json: Don't evaluate
This makes its behaviour consistent with the non-json
variant. Querying the outputs should be done by another command
(e.g. 'nix search')
2020-04-06 14:39:47 +02:00
Eelco Dolstra
ed13457dbf nix flake info --json: Show TreeInfo 2020-04-02 11:51:34 +02:00
Eelco Dolstra
bd10a07d17 Registry: Use a struct instead of a tuple for entries 2020-04-01 23:03:27 +02:00
Eelco Dolstra
77ffaea4fa Add a system-wide flake registry /etc/nix/registry.json
One application for this is pinning the 'nixpkgs' flake to the exact
revision used to build the NixOS system, e.g.

  {
      "flakes": [
          {
              "from": {
                  "id": "nixpkgs",
                  "type": "indirect"
              },
              "to": {
                  "owner": "NixOS",
                  "repo": "nixpkgs",
                  "type": "github",
                  "rev": "b0c285807d6a9f1b7562ec417c24fa1a30ecc31a"
              }
          }
      ],
      "version": 2
  }
2020-04-01 22:56:50 +02:00
Eelco Dolstra
e0a0ae0467 Move fetchers from libstore to libfetchers 2020-03-30 14:04:53 +02:00
Eelco Dolstra
4989c04dd2 nix flake info --json: Dump attr-style flakeref 2020-03-28 22:59:38 +01:00
Eelco Dolstra
2287e2f279 nix flake info: Show flake subdirectory 2020-03-28 18:05:50 +01:00
Eelco Dolstra
2a4e4f6a6e
Unified fetcher caching system 2020-03-17 22:35:29 +01:00
Eelco Dolstra
ae9119167e Change the lock file to a graph
This enables support for cycles between flakes.
2020-03-12 22:06:57 +01:00
Eelco Dolstra
73c9840569 Restore subdir -> dir
Got this mixed up somewhere.
2020-02-20 23:44:06 +01:00
Eelco Dolstra
d068f9ffff Restore subdir support in registries
Hacky...
2020-02-20 22:14:44 +01:00
Eelco Dolstra
46a284263f Fix build 2020-02-14 22:45:33 +01:00
Eelco Dolstra
0b013a54dc findAlongAttrPath(): Return position 2020-02-07 14:08:24 +01:00
Eelco Dolstra
b270869466
Renamed ref / resolvedRef -> lockedRef 2020-02-02 00:06:59 +01:00
Eelco Dolstra
887730aab3
Remove superfluous TreeInfo::rev field 2020-02-01 23:54:20 +01:00
Eelco Dolstra
8451298b35
Factor out TreeInfo 2020-02-01 22:06:55 +01:00
Eelco Dolstra
5d70b454be
nix flake update: Imply --refresh 2020-02-01 12:30:49 +01:00
Eelco Dolstra
678301072f nix flake list-inputs: Pretty-print the tree 2020-01-31 14:09:27 +01:00
Eelco Dolstra
a6e2b6b360 nix flake deps -> nix flake list-inputs
Also add a --json flag.
2020-01-31 13:00:50 +01:00
Eelco Dolstra
ebfbfe9515 Use std::nullopt 2020-01-30 01:10:26 +01:00
Eelco Dolstra
3c54e9ba01 Add 'nix flake archive' command
This copies a flake and all its inputs recursively to a store (e.g. a
binary cache). This is intended to enable long-term reproducibility
for flakes. However this will also require #3253.

Example:

  $ nix flake archive --json --to file:///tmp/my-cache nixops
  {"path":"/nix/store/272igzkgl1gdzmabsjvb2kb2zqbphb3p-source","inputs":{"nixops-aws":{"path":"/nix/store/ybcykw13gr7iq1pzg18iyibbcv8k9q1v-source","inputs":{}},"nixops-hetzner":{"path":"/nix/store/6yn0205x3nz55w8ms3335p2841javz2d-source","inputs":{}},"nixpkgs":{"path":"/nix/store/li3lkr2ajrzphqqz3jj2avndnyd3i5lc-source","inputs":{}}}}

  $ ll /tmp/my-cache
  total 16
  -rw-r--r-- 1 eelco users 403 Jan 30 01:01 272igzkgl1gdzmabsjvb2kb2zqbphb3p.narinfo
  -rw-r--r-- 1 eelco users 403 Jan 30 01:01 6yn0205x3nz55w8ms3335p2841javz2d.narinfo
  -rw-r--r-- 1 eelco users 408 Jan 30 01:01 li3lkr2ajrzphqqz3jj2avndnyd3i5lc.narinfo
  drwxr-xr-x 2 eelco users   6 Jan 30 01:01 nar
  -rw-r--r-- 1 eelco users  21 Jan 30 01:01 nix-cache-info
  -rw-r--r-- 1 eelco users 404 Jan 30 01:01 ybcykw13gr7iq1pzg18iyibbcv8k9q1v.narinfo

Fixes #3336.
2020-01-30 01:00:34 +01:00
Eelco Dolstra
26f895a26d Clean up the lock file handling flags
Added a flag --no-update-lock-file to barf if the lock file needs any
changes. This is useful for CI systems if you're building a
checkout. Fixes #2947.

Renamed --no-save-lock-file to --no-write-lock-file. It is now a fatal
error if the lock file needs changes but --no-write-lock-file is not
given.
2020-01-29 21:04:28 +01:00
Eelco Dolstra
f68bed7f67 Add flag --override-input to override specific lock file entries
E.g.

  $ nix flake update ~/Misc/eelco-configurations/hagbard \
    --override-input 'dwarffs/nixpkgs' ../my-nixpkgs

overrides the 'nixpkgs' input of the 'dwarffs' input of the top-level
flake.

Fixes #2837.
2020-01-29 18:41:25 +01:00
Eelco Dolstra
872a22fa23 resolveFlake -> lockFlake
"resolve" is ambiguous (also used for registry resolution).
2020-01-22 20:59:59 +01:00
Eelco Dolstra
90d55ed275 Fix 'nix flake update' 2020-01-22 19:28:27 +01:00
Eelco Dolstra
32f31a8c63 nix flake info: Don't show empty descriptions 2020-01-22 17:20:21 +01:00
Eelco Dolstra
9f4d8c6170 Pluggable fetchers
Flakes are now fetched using an extensible mechanism. Also lots of
other flake cleanups.
2020-01-21 22:56:04 +01:00
Eelco Dolstra
ecb3a1afa2 Merge remote-tracking branch 'origin/master' into flakes 2019-12-11 14:53:30 +01:00
Eelco Dolstra
ce27920936 Add start of 'nix profile' command 2019-10-22 00:22:15 +02:00
Eelco Dolstra
7d38060a0d Support non-x86_64-linux system types in flakes
A command like

  $ nix run nixpkgs#hello

will now build the attribute 'packages.${system}.hello' rather than
'packages.hello'. Note that this does mean that the flake needs to
export an attribute for every system type it supports, and you can't
build on unsupported systems. So 'packages' typically looks like this:

  packages = nixpkgs.lib.genAttrs ["x86_64-linux" "i686-linux"] (system: {
    hello = ...;
  });

The 'checks', 'defaultPackage', 'devShell', 'apps' and 'defaultApp'
outputs similarly are now attrsets that map system types to
derivations/apps. 'nix flake check' checks that the derivations for
all platforms evaluate correctly, but only builds the derivations in
'checks.${system}'.

Fixes #2861. (That issue also talks about access to ~/.config/nixpkgs
and --arg, but I think it's reasonable to say that flakes shouldn't
support those.)

The alternative to attribute selection is to pass the system type as
an argument to the flake's 'outputs' function, e.g. 'outputs = { self,
nixpkgs, system }: ...'. However, that approach would be at odds with
hermetic evaluation and make it impossible to enumerate the packages
provided by a flake.
2019-10-15 18:16:29 +02:00
Eelco Dolstra
a0bd088d84
Move addRegistrOverrides 2019-10-08 17:00:55 +02:00
Eelco Dolstra
21304c11f9
uri -> url for consistency 2019-10-08 17:00:55 +02:00
Eelco Dolstra
382aa05ff7 nix flake info --json: Get rid of duplicate getFlake() call
Also fix some gcc warnings.
2019-09-22 21:53:01 +02:00
Eelco Dolstra
5573365dff
nix flake check: Validate nixosConfigurations outputs 2019-09-19 20:15:42 +02:00
Eelco Dolstra
c67407172d
Record original flakerefs in the lock file again
If 'input.<name>.uri' changes, then the entry in the lockfile for
input <name> should be considered stale.

Also print some messages when lock file entries are added/updated.
2019-09-18 21:57:57 +02:00
Eelco Dolstra
55e55b34e6
nix flake check: Check hydraJobs 2019-09-10 17:39:55 +02:00
Eelco Dolstra
4b9dee6bcc
nix flake check: Do some basic checks on NixOS modules
Also show more position info.
2019-09-10 15:25:32 +02:00
Eelco Dolstra
dc3f52a144
nix flake check: Check overlays 2019-09-10 14:52:22 +02:00