Commit graph

458 commits

Author SHA1 Message Date
Eelco Dolstra
731bc65ec0
Refactor a bit 2019-07-12 16:16:27 +02:00
Eelco Dolstra
7ba928116e
nix dev-shell: Add --profile flag
This is useful to prevent the shell environment from being
garbage-collected.
2019-07-12 16:10:58 +02:00
Eelco Dolstra
990b5b2dcf
nix build: Add '--profile' flag
This replaces 'nix-env --set'. For example:

  $ nix build --profile /nix/var/nix/profiles/system \
      ~/Misc/eelco-configurations:nixosConfigurations.vyr.config.system.build.toplevel

updates the NixOS system profile from a flake.

This could have been a separate command (e.g. 'nix set-profile') but
1) '--profile' is pretty similar to '--out-link'; and 2) '--profile'
could be useful for other command (like 'nix dev-shell').
2019-07-12 15:32:17 +02:00
Eelco Dolstra
ad42a78469
Rename 'epoch' -> 'edition' 2019-07-11 13:54:53 +02:00
Eelco Dolstra
4205234f26
Merge remote-tracking branch 'origin/master' into flakes 2019-07-11 13:13:19 +02:00
Eelco Dolstra
aa739e7839
nix copy: Rename --substitute to --substitute-on-destination
'--substitute' was being shadowed by the regular '--substitute' (the
short-hand for '--option substitute true').

Fixes #2983.
2019-07-10 11:28:37 +02:00
Eelco Dolstra
cc218b15ba Merge remote-tracking branch 'origin/master' into flakes 2019-07-06 21:06:22 +02:00
Niklas Hambüchen
a96006d97f Get BOOST_LDFLAGS from autoconf, fix Ubuntu 16.04 build.
Our use of boost::coroutine2 depends on -lboost_context,
which in turn depends on `-lboost_thread`, which in turn depends
on `-lboost_system`.

I suspect that this builds on nix only because of low-level hacks
like NIX_LDFLAGS.

This commit passes the proper linker flags, thus fixing bootstrap
builds on non-nix distributions like Ubuntu 16.04.

With these changes, I can build Nix on Ubuntu 16.04 using:

    ./bootstrap.sh
    ./configure --prefix=$HOME/editline-prefix \
      --disable-doc-gen \
      CXX=g++-7 \
      --with-boost=$HOME/boost-prefix \
      EDITLINE_CFLAGS=-I$HOME/editline-prefix/include \
      EDITLINE_LIBS=-leditline \
      LDFLAGS=-L$HOME/editline-prefix/lib
    make

where

* g++-7 comes from gcc-7 from
  https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test,
* editline 1.14 from https://github.com/troglobit/editline/releases/tag/1.14.0
	was installed into `$HOME/editline-prefix`
  (because Ubuntu 16.04's `editline` is too old to have the function nix uses),
* boost 1.66 from
	https://www.boost.org/doc/libs/1_66_0/more/getting_started/unix-variants.html
	was installed into $HOME/boost-prefix (because Ubuntu 16.04 only has 1.58)
2019-07-03 04:32:25 +02:00
Niklas Hambüchen
d203c554fa Fix C++ compatibility with older editline versions.
For example, Ubuntu 16.04 and many similar long-term-support distros
have older versions.
2019-07-03 04:32:25 +02:00
Eelco Dolstra
6847c92788
Fix macOS build failure
Issue #2976.
2019-06-28 15:38:23 +02:00
Eelco Dolstra
e5f881a7e4
Merge remote-tracking branch 'origin/master' into flakes 2019-06-26 10:19:28 +02:00
Eelco Dolstra
324a5dc92f
ProgressBar: Fix updating
'updateCV.notify_one()' does nothing if the update thread is not
waiting for updateCV (in particular this happens when it is sleeping
on quitCV). So also set a variable to ensure that the update isn't
lost.
2019-06-25 21:59:20 +02:00
Eelco Dolstra
09dde33c19
Automatically use --no-net if there are no network interfaces
(cherry picked from commit 04a5976996)
2019-06-25 12:45:59 +02:00
Eelco Dolstra
5600b070a7
Add "warning" verbosity level
This ensures that "nix" shows warnings. Previously these were hidden
because they were at "info" level.

(cherry picked from commit 615a9d031d)
2019-06-25 12:44:17 +02:00
Eelco Dolstra
96c6b08ed7 nix doctor: Fix typo 2019-06-23 22:19:14 +02:00
Eelco Dolstra
d4fe9daed6
Simplify getFlake() / fetchFlake() logic 2019-06-21 19:04:58 +02:00
Eelco Dolstra
d132d057a8
Handle store symlinks in flake directories
E.g. 'nix path-info ./result' inside a flake directory now works
again.
2019-06-21 15:29:05 +02:00
Eelco Dolstra
a0de58f471
Make subcommand construction in MultiCommand lazy 2019-06-18 16:25:31 +02:00
Eelco Dolstra
eb18aedccb
Merge pull request #2951 from NixOS/app-improvements
App / check improvements
2019-06-18 16:15:40 +02:00
Eelco Dolstra
d4a48b12fa
Merge pull request #2917 from CSVdB/docs
Updated flake documentation
2019-06-18 16:05:55 +02:00
Eelco Dolstra
3871131308
Merge pull request #2949 from NixOS/no-net
Add --no-net flag
2019-06-18 15:59:42 +02:00
Eelco Dolstra
8a6704d826 Updated documentation 2019-06-18 11:08:04 +02:00
Eelco Dolstra
556f33422d
nix flake check: Ignore legacyPackages 2019-06-18 09:45:14 +02:00
Eelco Dolstra
f2fcc163fa
nix flake check: Warn about unknown flake outputs 2019-06-17 18:05:32 +02:00
Eelco Dolstra
9d1207c02c
nix flake check: Check apps 2019-06-17 17:59:57 +02:00
Eelco Dolstra
3b2ebd029c
nix flake info --json: Revive enumerating the outputs 2019-06-17 17:31:34 +02:00
Eelco Dolstra
0d69f7f3f0
nix app: Accept arguments
Example:

  $ nix app blender-bin -- --version
  Blender 2.80 (sub 74)
2019-06-17 17:05:37 +02:00
Eelco Dolstra
2467c98375
nix app: Search for installable in the 'apps' output
I.e. you can write

  $ nix app blender-bin:blender_2_80

which is equivalent to

  $ nix app blender-bin:apps.blender_2_80
2019-06-17 16:58:59 +02:00
Eelco Dolstra
d6c4fe55db
Merge remote-tracking branch 'origin/master' into flakes 2019-06-17 14:45:12 +02:00
Eelco Dolstra
04a5976996
Automatically use --no-net if there are no network interfaces 2019-06-17 10:14:27 +02:00
Eelco Dolstra
615a9d031d
Add "warning" verbosity level
This ensures that "nix" shows warnings. Previously these were hidden
because they were at "info" level.
2019-06-17 09:12:03 +02:00
Eelco Dolstra
26bc876ae6
nix: Add -L alias for --print-build-logs 2019-06-15 16:45:00 +02:00
Eelco Dolstra
415fc233e3
For nixpkgs.<attr>, use legacyPackages
This makes commands like 'nix run nixpkgs.chromium' work again.
2019-06-13 14:07:25 +02:00
Eelco Dolstra
6644b6099b
Add flake evaluation cache
This exploits the hermetic nature of flake evaluation to speed up
repeated evaluations of a flake output attribute.

For example (doing 'nix build' on an already present package):

  $ time nix build nixpkgs:firefox

  real    0m1.497s
  user    0m1.160s
  sys     0m0.139s

  $ time nix build nixpkgs:firefox

  real    0m0.052s
  user    0m0.038s
  sys     0m0.007s

The cache is ~/.cache/nix/eval-cache-v1.sqlite, which has entries like

  INSERT INTO Attributes VALUES(
    X'92a907d4efe933af2a46959b082cdff176aa5bfeb47a98fabd234809a67ab195',
    'packages.firefox',
    1,
    '/nix/store/pbalzf8x19hckr8cwdv62rd6g0lqgc38-firefox-67.0.drv /nix/store/g6q0gx0v6xvdnizp8lrcw7c4gdkzana0-firefox-67.0 out');

where the hash 92a9... is a fingerprint over the flake store path and
the contents of the lockfile. Because flakes are evaluated in pure
mode, this uniquely identifies the evaluation result.
2019-06-07 22:25:48 +02:00
Eelco Dolstra
54aff8430c
Move flake-related stuff to src/libexpr/flake 2019-06-05 16:51:54 +02:00
Eelco Dolstra
1e53a07712
Make non-flake inputs lazy
Also add a proper test for non-flake inputs.
2019-06-04 20:56:42 +02:00
Eelco Dolstra
45b5c606ac
Don't register invalid paths as GC roots
Unfortunately this doesn't work. Maybe we should keep separate roots
for each path.
2019-06-04 20:34:08 +02:00
Eelco Dolstra
5fe7be2409
Rename dep -> input
Also use nlohmann::json range-based for.
2019-06-04 20:08:13 +02:00
Eelco Dolstra
278114d559
Fix GC closure generation 2019-06-04 19:45:16 +02:00
Eelco Dolstra
6dbd5c26e6
Make flake input fetching lazy
As long as the flake input is locked, it is now only fetched when it
is evaluated (e.g. "nixpkgs" is fetched when
"inputs.nixpkgs.<something>" is evaluated).

This required adding an "id" attribute to the members of "inputs" in
lockfiles, e.g.

  "inputs": {
    "nixpkgs/release-19.03": {
      "id": "nixpkgs",
      "inputs": {},
      "narHash": "sha256-eYtxncIMFVmOHaHBtTdPGcs/AnJqKqA6tHCm0UmPYQU=",
      "nonFlakeInputs": {},
      "uri": "github:edolstra/nixpkgs/e9d5882bb861dc48f8d46960e7c820efdbe8f9c1"
    }
  }

because the flake ID needs to be known beforehand to construct the
"inputs" attrset.

Fixes #2913.
2019-06-04 19:17:03 +02:00
Nick Van den Broeck
d9a6a75ed2 Made epochs more fine-grained
Fixes #2894
2019-06-03 14:47:47 +02:00
Eelco Dolstra
5fbd9fee0b
Add 'nix app' command
This is like 'nix run', except that the command to execute is defined
in a flake output, e.g.

  defaultApp = {
    type = "app";
    program = "${packages.blender_2_80}/bin/blender";
  };

Thus you can do

  $ nix app blender-bin

to start Blender from the 'blender-bin' flake.

In the future, we can extend this with sandboxing. (For example we
would want to be able to specify that Blender should not have network
access by default and should only have access to certain paths in the
user's home directory.)
2019-06-03 09:27:03 +02:00
Eelco Dolstra
15f241775a
Doh 2019-05-31 23:21:53 +02:00
Eelco Dolstra
8cb3bbd504
Fix handling of bare flakerefs containing a colon 2019-05-31 22:17:39 +02:00
Eelco Dolstra
ccb1bad612
Allow bare flakerefs as installables
So now

  $ nix build blender-bin

works and builds the default package from that flake. You don't need
to add a colon at the end anymore.
2019-05-31 21:42:23 +02:00
Eelco Dolstra
094539ef4a
Rename requires -> inputs, provides -> outputs
Issue #2828.
2019-05-31 09:59:48 +02:00
Eelco Dolstra
49436bdbb7
nix flake info --json: List the "provides"
It also lists the contents of "checks" and "packages".

For example:

  $ nix flake info --json | jq
  {
    "branch": "HEAD",
    "description": "The purely functional package manager",
    "epoch": 2019,
    "id": "nix",
    "lastModified": 1559161142,
    "path": "/nix/store/2w2qla8735dbxah8gai8r1nsbf5x4f5d-source",
    "provides": {
      "checks": {
        "binaryTarball": {},
        "nix-copy-closure": {},
        "perlBindings": {},
        "remoteBuilds": {},
        "setuid": {}
      },
      "defaultPackage": {},
      "devShell": {},
      "hydraJobs": {},
      "packages": {
        "nix": {},
        "nix-perl-bindings": {}
      }
    },
    "revCount": 6955,
    "revision": "8cb24e04e8b6cc60e2504733afe78e0eadafcd98",
    "uri": "/home/eelco/Dev/nix"
  }

Fixes #2820.
2019-05-29 22:22:28 +02:00
Eelco Dolstra
3488fa7c6c
Hack: Use legacyPackages from Nixpkgs
Nixpkgs doesn't provide a clean "packages" set yet, so until that's
the case, look for packages in "legacyPackages" as well.
2019-05-29 22:22:28 +02:00
Eelco Dolstra
a9d3524e1f
nix flake check: Use read-only mode if we're not building 2019-05-29 21:00:44 +02:00
Eelco Dolstra
0e32b32fa3
nix flake check: Check defaultPackage, devShell and packages 2019-05-29 20:57:08 +02:00
Eelco Dolstra
e0aaf05f4f
Add 'nix flake check' command
This evaluates all the 'provides' of a flake and builds the 'check'
attributes.
2019-05-29 17:25:41 +02:00
Eelco Dolstra
6e4a8c47f4
Put flake-related stuff in its own namespace 2019-05-29 15:31:07 +02:00
Eelco Dolstra
6636808e90
Merge remote-tracking branch 'origin/master' into flakes 2019-05-29 12:36:44 +02:00
Eelco Dolstra
0f840483c7
Add date of last commit to SourceInfo
This is primarily useful for version string generation, where we need
a monotonically increasing number. The revcount is the preferred thing
to use, but isn't available for GitHub flakes (since it requires
fetching the entire history). The last commit timestamp OTOH can be
extracted from GitHub tarballs.
2019-05-29 10:10:36 +02:00
Eelco Dolstra
444786e6d3
nix flake info: Add missing newline 2019-05-28 14:11:19 +02:00
Eelco Dolstra
46294d60cd
printFlakeInfo: Separate JSON output 2019-05-28 14:01:57 +02:00
Eelco Dolstra
e7e7a03baf
nix flake deps: Remove --json flag for now
It doesn't produce valid JSON at the moment (but a concatenation of
JSON objects). Anyway we probably should merge this command info 'nix
flake info'.
2019-05-28 13:22:11 +02:00
Eelco Dolstra
fdf06ce72f
printFlakeInfo/printNonFlakeInfo: Factor out commonality 2019-05-28 13:21:06 +02:00
Eelco Dolstra
25e497bf9c
nix flake info/deps: Stop progress bar before printing output 2019-05-28 13:14:27 +02:00
Eelco Dolstra
dda4f7167b
Remove redundant resolvedRef fields since they're already in SourceInfo 2019-05-28 13:12:43 +02:00
Eelco Dolstra
6d7efcfaeb
Store SourceInfo in Flake and NonFlake
This deduplicates some shared fields. Factoring out the commonality is
useful in places like makeFlakeValue().
2019-05-28 13:06:43 +02:00
Eelco Dolstra
90fe1dfd2f Register flake source trees as GC roots
This ensures that flakes don't get garbage-collected, which is
important to get nix-channel-like behaviour.

For example, running

  $ nix build hydra:

will create a GC root

  ~/.cache/nix/flake-closures/hydra -> /nix/store/xarfiqcwa4w8r4qpz1a769xxs8c3phgn-flake-closure

where the contents/references of the linked file in the store are the
flake source trees used by the 'hydra' flake:

  /nix/store/n6d5f5lkpfjbmkyby0nlg8y1wbkmbc7i-source
  /nix/store/vbkg4zy1qd29fnhflsv9k2j9jnbqd5m2-source
  /nix/store/z46xni7d47s5wk694359mq9ay353ar94-source

Note that this in itself is not enough to allow offline use; the
fetcher for the flakeref (e.g. fetchGit or downloadCached) must not
fail if it cannot fetch the latest version of the file, so long as it
knows a cached version.

Issue #2868.
2019-05-23 23:53:25 +02:00
Eelco Dolstra
66f1d7ee95 Fetch the flake registry from the NixOS/flake-registry repo 2019-05-22 22:56:46 +02:00
Eelco Dolstra
e414bde6f9 Check the flake epoch
Closes #2883.
2019-05-22 14:31:40 +02:00
Eelco Dolstra
3e8ef9eb22 nix flake deps: Print flake dependencies 2019-05-22 14:04:18 +02:00
Eelco Dolstra
70136a9bf4 Move flake-related flags into a separate class
Also, rename --dont-save-lock-file to --no-save-lock-file and change
noRegistries to useRegistries.
2019-05-22 14:04:18 +02:00
Eelco Dolstra
2468672e30 Improve FlakeCommand
It now handles commonality like calling getFlake() and resolving
relative local flake refs.

Fixes #2822.
2019-05-22 14:03:28 +02:00
Eelco Dolstra
4d829916e7
Merge pull request #2800 from flokli/progress-bar-hide-unknown-expected
progress-bar: hide expected if expected is 0 (unknown)
2019-05-21 13:29:28 +02:00
Nick Van den Broeck
ef6ae61503 Lockfile handling in resolveFlake is fixed 2019-05-17 14:50:10 +02:00
Nick Van den Broeck
98f20dee41 Give errors in resolveFlake
If DontUpdate but the lockfile isn't correct
2019-05-16 15:02:25 +02:00
Nick Van den Broeck
d9ad3723d5 Fixed issue 65
lockfile updating
2019-05-16 15:02:05 +02:00
Eelco Dolstra
0f5032c5a4 Merge remote-tracking branch 'origin/master' into flakes 2019-05-15 20:51:29 +02:00
Eelco Dolstra
8f6c72faee
Merge pull request #2810 from NixOS/print-build-logs
nix: Add --print-build-logs flag
2019-05-15 20:38:38 +02:00
Eelco Dolstra
66b8a62101 nix: Add --print-build-logs flag
This causes 'nix' to print build log output to stderr rather than
showing the last log line in the progress bar. Log lines are prefixed
by the name of the derivation (minus the version string), e.g.

  binutils> make[1]: Leaving directory '/build/binutils-2.31.1'
  binutils-wrapper> unpacking sources
  binutils-wrapper> patching sources
  ...
  binutils-wrapper> Using dynamic linker: '/nix/store/kr51dlsj9v5cr4n8700jliyz8v5b2q7q-bootstrap-stage0-glibc/lib/ld-linux-x86-64.so.2'
  bootstrap-stage2-gcc-wrapper> unpacking sources
  ...
  linux-headers> unpacking sources
  linux-headers> unpacking source archive /nix/store/8javli69jhj3bkql2c35gsj5vl91p382-linux-4.19.16.tar.xz
2019-05-15 17:33:56 +02:00
Florian Klink
6ade7ec022 progress-bar: hide expected if expected is 0 (unknown)
Sometimes, "expected" can be "0", but in fact means "unknown".

This is for example the case when downloading a file while the http
server doesn't send the `Content-Length` header, like when running `nix
build` pointing to a nixpkgs checkout streamed from GitHub:

⇒  nix build -f https://github.com/NixOS/nixpkgs/archive/master.tar.gz hello
[1.8/0.0 MiB DL] downloading 'https://github.com/NixOS/nixpkgs/archive/master.tar.gz'

In that case, don't show that weird progress bar, but only the (slowly
increasing) downloaded size ("done").

⇒  nix build -f https://github.com/NixOS/nixpkgs/archive/master.tar.gz hello
[1.8 MiB DL] downloading 'https://github.com/NixOS/nixpkgs/archive/master.tar.gz'

This commit also updates fmt calls with three numbers (when something is
currently 'running' too) - I'm not sure if this can be provoked, but
showing "0" as expected doesn't make any sense, as we're obviously doing
more than nothing.
2019-05-11 12:34:39 +02:00
Eelco Dolstra
156e3a9daa
nix dev-shell: Ignore SSL_CERT_FILE 2019-05-11 01:50:28 +02:00
Eelco Dolstra
cb5ebc5c11
nix dev-shell: Keep $TERM 2019-05-08 17:07:35 +02:00
Eelco Dolstra
2bc55aba1e
Merge remote-tracking branch 'origin/master' into flakes 2019-05-08 14:30:27 +02:00
Eelco Dolstra
2fc8a29a9c
Merge remote-tracking branch 'tweag/nix-shell' into flakes 2019-05-08 13:46:44 +02:00
Nick Van den Broeck
201f92e02c Fixed Flake data type and flake fetching 2019-05-07 05:57:08 +02:00
Eelco Dolstra
7ba0f98e64 nix dev-shell: Less purity 2019-05-02 21:28:52 +02:00
Eelco Dolstra
8ec77614f6 Move createTempFile to libutil 2019-05-02 21:28:41 +02:00
Eelco Dolstra
dea18ff999 nix dev-shell: Execute shellHook 2019-05-02 21:13:19 +02:00
Eelco Dolstra
2919c496ea nix dev-shell: Use 'provides.devShell' by default
Thus

  $ nix dev-shell

will now build the 'provides.devShell' attribute from the flake in the
current directory. If it doesn't exist, it falls back to
'provides.defaultPackage'.
2019-05-02 21:10:13 +02:00
Eelco Dolstra
2b8c63f303 Add 'nix dev-shell' and 'nix print-dev-env' command
'nix dev-shell' is intended to replace nix-shell. It supports flakes,
e.g.

  $ nix dev-shell nixpkgs:hello

starts a bash shell providing an environment for building 'hello'.

Like Lorri (and unlike nix-shell), it computes the build environment
by building a modified top-level derivation that writes the
environment after running $stdenv/setup to $out and exits. This
provides some caching, so it's faster than nix-shell in some cases
(especially for packages with lots of dependencies, where the setup
script takes a long time).

There also is a command 'nix print-dev-env' that prints out shell code
for setting up the build environment in an existing shell, e.g.

  $ . <(nix print-dev-env nixpkgs:hello)

https://github.com/tweag/nix/issues/21
2019-05-02 20:22:14 +02:00
Eelco Dolstra
4588a6ff3c Merge remote-tracking branch 'tweag/flakeFlags' into flakes 2019-04-30 12:46:54 +02:00
Nick Van den Broeck
24b35bf9e7 Fixed issue #13 2019-04-30 11:26:45 +02:00
Nick Van den Broeck
35d1c95f7f Fix flag registry order 2019-04-30 11:04:31 +02:00
Nick Van den Broeck
d867e1804a Fix printing FlakeList 2019-04-25 11:43:16 +02:00
Eelco Dolstra
21d5abfc14 Merge remote-tracking branch 'tweag/flake-clone' into flakes 2019-04-24 12:47:35 +02:00
Eelco Dolstra
cc51e37ad0 Use "." as the default installable
This makes e.g. "nix build" do something more or less reasonable
(namely, build the default package of the flake in the current
directory).
2019-04-19 16:16:04 +02:00
Eelco Dolstra
4fb594a375 Merge remote-tracking branch 'tweag/resolvedFlake' into flakes 2019-04-19 15:56:17 +02:00
Eelco Dolstra
3392f1b778 Shut up clang warning 2019-04-19 14:41:06 +02:00
Nick Van den Broeck
e51abb6631 Changed some names 2019-04-19 14:23:35 +02:00
Eelco Dolstra
50ec2bed9e nix flake info: Show revcount 2019-04-19 14:19:46 +02:00
Eelco Dolstra
160ce18a0e Improve missing flake.nix error message 2019-04-19 11:43:56 +02:00
Nick Van den Broeck
b42ba08fc8 Add command flake clone 2019-04-17 13:31:09 +02:00
Eelco Dolstra
3c28cb1b8f Improve 'nix flake info' a bit
Example:

  $ nix flake info dwarffs
  ID:          dwarffs
  URI:         github:edolstra/dwarffs/a83d182fe3fe528ed6366a5cec3458bcb1a5f6e1
  Description: A filesystem that fetches DWARF debug info from the Internet on demand
  Revision:    a83d182fe3fe528ed6366a5cec3458bcb1a5f6e1
  Path:        /nix/store/grgd14kxxk8q4n503j87mpz48gcqpqw7-source
2019-04-16 16:18:19 +02:00
Eelco Dolstra
3d0e81051f Fix lock file generation
Before:

   "requires": {
        "nixpkgs": {
            "uri": "nixpkgs"
        }
    },

After:

    "requires": {
        "nixpkgs": {
            "uri": "github:edolstra/nixpkgs/f10e8a02eb7fa2b4a070f30cf87f4efcc7f3186d"
        }
    },
2019-04-16 15:43:55 +02:00
Eelco Dolstra
cfca793a20 Remove unneeded pureEval flags 2019-04-16 15:06:40 +02:00