Commit graph

4539 commits

Author SHA1 Message Date
Eelco Dolstra
34fa8ce917
nix: Support -j flag 2019-06-15 16:34:06 +02:00
Félix Baylac-Jacqué
9e0f5f803f
Daemon: warn when an untrusted user cannot override a setting
In a daemon-based Nix setup, some options cannot be overridden by a
client unless the client's user is considered trusted.

Currently, if an untrusted user tries to override one of those
options, we are silently ignoring it.

This can be pretty confusing in certain situations.

e.g. a user thinks he disabled the sandbox when in reality he did not.

We are now sending a warning message letting know the user some options
have been ignored.

Related to #1761.
2019-06-15 03:59:12 -04:00
Eelco Dolstra
415fc233e3
For nixpkgs.<attr>, use legacyPackages
This makes commands like 'nix run nixpkgs.chromium' work again.
2019-06-13 14:07:25 +02:00
Eelco Dolstra
06010eaf19
Fix fetchTarball with chroot stores
Fixes #2405.
2019-06-12 10:34:13 +02:00
Eelco Dolstra
69b047f4ce
writeRegistry(): Write correct version 2019-06-11 21:32:57 +02:00
Eelco Dolstra
c47d2dac6c
Disable EvalCache in impure mode 2019-06-07 22:38:39 +02:00
Eelco Dolstra
6644b6099b
Add flake evaluation cache
This exploits the hermetic nature of flake evaluation to speed up
repeated evaluations of a flake output attribute.

For example (doing 'nix build' on an already present package):

  $ time nix build nixpkgs:firefox

  real    0m1.497s
  user    0m1.160s
  sys     0m0.139s

  $ time nix build nixpkgs:firefox

  real    0m0.052s
  user    0m0.038s
  sys     0m0.007s

The cache is ~/.cache/nix/eval-cache-v1.sqlite, which has entries like

  INSERT INTO Attributes VALUES(
    X'92a907d4efe933af2a46959b082cdff176aa5bfeb47a98fabd234809a67ab195',
    'packages.firefox',
    1,
    '/nix/store/pbalzf8x19hckr8cwdv62rd6g0lqgc38-firefox-67.0.drv /nix/store/g6q0gx0v6xvdnizp8lrcw7c4gdkzana0-firefox-67.0 out');

where the hash 92a9... is a fingerprint over the flake store path and
the contents of the lockfile. Because flakes are evaluated in pure
mode, this uniquely identifies the evaluation result.
2019-06-07 22:25:48 +02:00
Matthew Bauer
5011a52cf3 Just enable hack on macOS
This is not needed on linux at all! Tried to explain as much as I
understand with the problem.
2019-06-05 20:20:50 -04:00
Eelco Dolstra
54aff8430c
Move flake-related stuff to src/libexpr/flake 2019-06-05 16:51:54 +02:00
Matthew Bauer
4a3e96281d Handle SIGWINCH in main thread
For the SIGWINCH signal to be caught, it needs to be set in sigaction
on the main thread. Previously, this was broken, and updateWindowSize
was never being called. Tested on macOS 10.14.
2019-06-05 00:40:45 -04:00
Eelco Dolstra
1b05792988
Shorter syntax for referencing flake outputs
Fixes #2819.
2019-06-04 22:35:43 +02:00
Eelco Dolstra
ce225615c3
Eliminate duplicate fetching of the top-level flake 2019-06-04 21:10:53 +02:00
Eelco Dolstra
087530dec4
Add comments 2019-06-04 21:07:55 +02:00
Eelco Dolstra
1e53a07712
Make non-flake inputs lazy
Also add a proper test for non-flake inputs.
2019-06-04 20:56:42 +02:00
Eelco Dolstra
1c5067b9a7
Check hash 2019-06-04 20:35:35 +02:00
Eelco Dolstra
45b5c606ac
Don't register invalid paths as GC roots
Unfortunately this doesn't work. Maybe we should keep separate roots
for each path.
2019-06-04 20:34:08 +02:00
Eelco Dolstra
5fe7be2409
Rename dep -> input
Also use nlohmann::json range-based for.
2019-06-04 20:08:13 +02:00
Eelco Dolstra
9e99b5205c
Move LockFile and related types to a separate file 2019-06-04 20:01:21 +02:00
Eelco Dolstra
278114d559
Fix GC closure generation 2019-06-04 19:45:16 +02:00
Eelco Dolstra
6dbd5c26e6
Make flake input fetching lazy
As long as the flake input is locked, it is now only fetched when it
is evaluated (e.g. "nixpkgs" is fetched when
"inputs.nixpkgs.<something>" is evaluated).

This required adding an "id" attribute to the members of "inputs" in
lockfiles, e.g.

  "inputs": {
    "nixpkgs/release-19.03": {
      "id": "nixpkgs",
      "inputs": {},
      "narHash": "sha256-eYtxncIMFVmOHaHBtTdPGcs/AnJqKqA6tHCm0UmPYQU=",
      "nonFlakeInputs": {},
      "uri": "github:edolstra/nixpkgs/e9d5882bb861dc48f8d46960e7c820efdbe8f9c1"
    }
  }

because the flake ID needs to be known beforehand to construct the
"inputs" attrset.

Fixes #2913.
2019-06-04 19:17:03 +02:00
Eelco Dolstra
653c4e439b
Fix clang compilation error
https://hydra.nixos.org/build/94332344

https://stackoverflow.com/questions/46114214/lambda-implicit-capture-fails-with-variable-declared-from-structured-binding
2019-06-03 21:51:06 +02:00
Nick Van den Broeck
d9a6a75ed2 Made epochs more fine-grained
Fixes #2894
2019-06-03 14:47:47 +02:00
Eelco Dolstra
5fbd9fee0b
Add 'nix app' command
This is like 'nix run', except that the command to execute is defined
in a flake output, e.g.

  defaultApp = {
    type = "app";
    program = "${packages.blender_2_80}/bin/blender";
  };

Thus you can do

  $ nix app blender-bin

to start Blender from the 'blender-bin' flake.

In the future, we can extend this with sandboxing. (For example we
would want to be able to specify that Blender should not have network
access by default and should only have access to certain paths in the
user's home directory.)
2019-06-03 09:27:03 +02:00
Eelco Dolstra
fb692e5f7b
Bindings: Add convenience method for requiring an attribute 2019-06-03 09:27:03 +02:00
Eelco Dolstra
a2f86ac647
Merge remote-tracking branch 'origin/master' into flakes 2019-06-03 09:22:19 +02:00
Eelco Dolstra
aec545c20b Fix segfault in builtin fetchurl with hashed mirrors + SRI hashes 2019-06-01 15:27:43 +02:00
Eelco Dolstra
15f241775a
Doh 2019-05-31 23:21:53 +02:00
Eelco Dolstra
8cb3bbd504
Fix handling of bare flakerefs containing a colon 2019-05-31 22:17:39 +02:00
Eelco Dolstra
8abb8647a3
Automatically determine subdir for path flakes
This means that in a flake in a subdirectory of a Git repo, you can
now do

  $ nix build

rather than the inconvenient

  $ nix build ../..?dir=foo/bar
2019-05-31 21:52:02 +02:00
Eelco Dolstra
ccb1bad612
Allow bare flakerefs as installables
So now

  $ nix build blender-bin

works and builds the default package from that flake. You don't need
to add a colon at the end anymore.
2019-05-31 21:42:23 +02:00
Eelco Dolstra
7adb10d29b
Fix reading the lockfile of a flake in a subdirectory 2019-05-31 20:12:59 +02:00
Eelco Dolstra
9169046e64
Add operator << for LockFile
Useful for debugging.
2019-05-31 20:10:56 +02:00
Eelco Dolstra
b971e406de
Support 'dir' and other parameters in path flakerefs 2019-05-31 19:01:11 +02:00
Eelco Dolstra
63c5c91cc0
Show hash mismatch warnings in SRI format 2019-05-31 18:48:28 +02:00
Eelco Dolstra
094539ef4a
Rename requires -> inputs, provides -> outputs
Issue #2828.
2019-05-31 09:59:48 +02:00
Eelco Dolstra
49436bdbb7
nix flake info --json: List the "provides"
It also lists the contents of "checks" and "packages".

For example:

  $ nix flake info --json | jq
  {
    "branch": "HEAD",
    "description": "The purely functional package manager",
    "epoch": 2019,
    "id": "nix",
    "lastModified": 1559161142,
    "path": "/nix/store/2w2qla8735dbxah8gai8r1nsbf5x4f5d-source",
    "provides": {
      "checks": {
        "binaryTarball": {},
        "nix-copy-closure": {},
        "perlBindings": {},
        "remoteBuilds": {},
        "setuid": {}
      },
      "defaultPackage": {},
      "devShell": {},
      "hydraJobs": {},
      "packages": {
        "nix": {},
        "nix-perl-bindings": {}
      }
    },
    "revCount": 6955,
    "revision": "8cb24e04e8b6cc60e2504733afe78e0eadafcd98",
    "uri": "/home/eelco/Dev/nix"
  }

Fixes #2820.
2019-05-29 22:22:28 +02:00
Eelco Dolstra
3488fa7c6c
Hack: Use legacyPackages from Nixpkgs
Nixpkgs doesn't provide a clean "packages" set yet, so until that's
the case, look for packages in "legacyPackages" as well.
2019-05-29 22:22:28 +02:00
Eelco Dolstra
a9d3524e1f
nix flake check: Use read-only mode if we're not building 2019-05-29 21:00:44 +02:00
Eelco Dolstra
0e32b32fa3
nix flake check: Check defaultPackage, devShell and packages 2019-05-29 20:57:08 +02:00
Eelco Dolstra
e0aaf05f4f
Add 'nix flake check' command
This evaluates all the 'provides' of a flake and builds the 'check'
attributes.
2019-05-29 17:25:41 +02:00
Eelco Dolstra
6ae4437acb
Remove makeFlakeValue() 2019-05-29 15:44:48 +02:00
Eelco Dolstra
6e4a8c47f4
Put flake-related stuff in its own namespace 2019-05-29 15:31:07 +02:00
Eelco Dolstra
c356d034f3
Make unsupported flake attributes a fatal error 2019-05-29 15:12:22 +02:00
Eelco Dolstra
6636808e90
Merge remote-tracking branch 'origin/master' into flakes 2019-05-29 12:36:44 +02:00
Ding Xiang Fei
abdedcdb38 bump cpptoml to v0.1.1 2019-05-29 17:01:39 +08:00
Eelco Dolstra
ae7b56cd9a
Get last commit time of github flakes 2019-05-29 10:10:40 +02:00
Eelco Dolstra
0f840483c7
Add date of last commit to SourceInfo
This is primarily useful for version string generation, where we need
a monotonically increasing number. The revcount is the preferred thing
to use, but isn't available for GitHub flakes (since it requires
fetching the entire history). The last commit timestamp OTOH can be
extracted from GitHub tarballs.
2019-05-29 10:10:36 +02:00
Eelco Dolstra
22f2744afd
Iterate over references 2019-05-28 23:05:08 +02:00
Eelco Dolstra
9eaebbf575
Merge branch 'attrPaths' of https://github.com/NinjaTrappeur/nix 2019-05-28 22:59:05 +02:00
Eelco Dolstra
444786e6d3
nix flake info: Add missing newline 2019-05-28 14:11:19 +02:00
Eelco Dolstra
46294d60cd
printFlakeInfo: Separate JSON output 2019-05-28 14:01:57 +02:00
Eelco Dolstra
ecee759b80
callFlake(): Emit source info attributes for non-flake dependencies 2019-05-28 14:01:08 +02:00
Eelco Dolstra
e7e7a03baf
nix flake deps: Remove --json flag for now
It doesn't produce valid JSON at the moment (but a concatenation of
JSON objects). Anyway we probably should merge this command info 'nix
flake info'.
2019-05-28 13:22:11 +02:00
Eelco Dolstra
fdf06ce72f
printFlakeInfo/printNonFlakeInfo: Factor out commonality 2019-05-28 13:21:06 +02:00
Eelco Dolstra
25e497bf9c
nix flake info/deps: Stop progress bar before printing output 2019-05-28 13:14:27 +02:00
Eelco Dolstra
dda4f7167b
Remove redundant resolvedRef fields since they're already in SourceInfo 2019-05-28 13:12:43 +02:00
Eelco Dolstra
4846304541
Rename contentHash -> narHash for consistency 2019-05-28 13:08:40 +02:00
Eelco Dolstra
894e007445
Move hash into SourceInfo and rename to narHash to avoid ambiguity 2019-05-28 13:07:15 +02:00
Eelco Dolstra
6d7efcfaeb
Store SourceInfo in Flake and NonFlake
This deduplicates some shared fields. Factoring out the commonality is
useful in places like makeFlakeValue().
2019-05-28 13:06:43 +02:00
Eelco Dolstra
de36cf3db9
Merge branch 'nonFlakeRequiresTest' of https://github.com/CSVdB/nix into flakes 2019-05-28 12:05:11 +02:00
Nick Van den Broeck
4d030a8d96 Added nonFlakeRequires test
Fixes #2888
2019-05-28 10:51:45 +02:00
Nick Van den Broeck
6fb7545fa1 Fixed relative path parsing
Fixed #2821
2019-05-28 10:17:28 +02:00
Eelco Dolstra
90fe1dfd2f Register flake source trees as GC roots
This ensures that flakes don't get garbage-collected, which is
important to get nix-channel-like behaviour.

For example, running

  $ nix build hydra:

will create a GC root

  ~/.cache/nix/flake-closures/hydra -> /nix/store/xarfiqcwa4w8r4qpz1a769xxs8c3phgn-flake-closure

where the contents/references of the linked file in the store are the
flake source trees used by the 'hydra' flake:

  /nix/store/n6d5f5lkpfjbmkyby0nlg8y1wbkmbc7i-source
  /nix/store/vbkg4zy1qd29fnhflsv9k2j9jnbqd5m2-source
  /nix/store/z46xni7d47s5wk694359mq9ay353ar94-source

Note that this in itself is not enough to allow offline use; the
fetcher for the flakeref (e.g. fetchGit or downloadCached) must not
fail if it cannot fetch the latest version of the file, so long as it
knows a cached version.

Issue #2868.
2019-05-23 23:53:25 +02:00
Eelco Dolstra
6b77bfc28d FlakeRef::to_string(): Check round trip 2019-05-23 23:39:58 +02:00
Eelco Dolstra
6e984431dd fetchGit: Don't barf if we can't update our Git clone
Instead print a warning that we're continuing with the most recently
fetched version.
2019-05-23 23:38:40 +02:00
Eelco Dolstra
f0d6d67af9 Prevent the global registry from being GC'ed
Issue #2868.
2019-05-22 23:43:58 +02:00
Eelco Dolstra
df3f5a78d5 Refactor downloadCached() interface 2019-05-22 23:36:29 +02:00
Eelco Dolstra
66f1d7ee95 Fetch the flake registry from the NixOS/flake-registry repo 2019-05-22 22:56:46 +02:00
Eelco Dolstra
e414bde6f9 Check the flake epoch
Closes #2883.
2019-05-22 14:31:40 +02:00
Eelco Dolstra
3e8ef9eb22 nix flake deps: Print flake dependencies 2019-05-22 14:04:18 +02:00
Eelco Dolstra
70136a9bf4 Move flake-related flags into a separate class
Also, rename --dont-save-lock-file to --no-save-lock-file and change
noRegistries to useRegistries.
2019-05-22 14:04:18 +02:00
Eelco Dolstra
2468672e30 Improve FlakeCommand
It now handles commonality like calling getFlake() and resolving
relative local flake refs.

Fixes #2822.
2019-05-22 14:03:28 +02:00
Eelco Dolstra
5990b86391 Use warn(), tweak messages 2019-05-21 15:03:54 +02:00
Eelco Dolstra
20a1a65d37 Only rewrite the lockfile if it changed
This removes spurious warnings about failure to write the lockfile.
2019-05-21 14:55:43 +02:00
Eelco Dolstra
4d829916e7
Merge pull request #2800 from flokli/progress-bar-hide-unknown-expected
progress-bar: hide expected if expected is 0 (unknown)
2019-05-21 13:29:28 +02:00
Nick Van den Broeck
ef6ae61503 Lockfile handling in resolveFlake is fixed 2019-05-17 14:50:10 +02:00
Nick Van den Broeck
98f20dee41 Give errors in resolveFlake
If DontUpdate but the lockfile isn't correct
2019-05-16 15:02:25 +02:00
Nick Van den Broeck
d9ad3723d5 Fixed issue 65
lockfile updating
2019-05-16 15:02:05 +02:00
Eelco Dolstra
b531695331
Merge pull request #96 from tweag/support-chroot-store
Make flakes work with 'nix build --store ...'
2019-05-16 10:51:40 +02:00
Eelco Dolstra
0f5032c5a4 Merge remote-tracking branch 'origin/master' into flakes 2019-05-15 20:51:29 +02:00
Eelco Dolstra
8f6c72faee
Merge pull request #2810 from NixOS/print-build-logs
nix: Add --print-build-logs flag
2019-05-15 20:38:38 +02:00
Eelco Dolstra
66b8a62101 nix: Add --print-build-logs flag
This causes 'nix' to print build log output to stderr rather than
showing the last log line in the progress bar. Log lines are prefixed
by the name of the derivation (minus the version string), e.g.

  binutils> make[1]: Leaving directory '/build/binutils-2.31.1'
  binutils-wrapper> unpacking sources
  binutils-wrapper> patching sources
  ...
  binutils-wrapper> Using dynamic linker: '/nix/store/kr51dlsj9v5cr4n8700jliyz8v5b2q7q-bootstrap-stage0-glibc/lib/ld-linux-x86-64.so.2'
  bootstrap-stage2-gcc-wrapper> unpacking sources
  ...
  linux-headers> unpacking sources
  linux-headers> unpacking source archive /nix/store/8javli69jhj3bkql2c35gsj5vl91p382-linux-4.19.16.tar.xz
2019-05-15 17:33:56 +02:00
Eelco Dolstra
5c34d66538 Make flakes work with 'nix build --store ...'
It was getting confused between logical and real store paths.

Also, make fetchGit and fetchMercurial update allowedPaths properly.

(Maybe the evaluator, rather than the caller of the evaluator, should
apply toRealPath(), but that's a bigger change.)
2019-05-15 15:38:24 +02:00
Nick Van den Broeck
4c9ebd20d7 One FIXME was already fixed 2019-05-15 08:10:46 +02:00
Eelco Dolstra
5f6840fbb4
Merge pull request #2802 from LnL7/fix-needs-hashrewrite
build: move needsHashRewrite initialization to startBuilder
2019-05-13 08:53:48 +02:00
Graham Christensen
f1b8e9efe7
runProgram: Uncomment chdir support 2019-05-12 17:03:01 -04:00
Daiderd Jordan
ce02fc74b2
build: make needsHashRewrite a method 2019-05-12 22:56:27 +02:00
Graham Christensen
73b797c207
handleDiffHook: stop passing allowVfork 2019-05-12 13:44:22 -04:00
Graham Christensen
b4a05edbfe
runProgram: support gid, uid, chdir 2019-05-12 13:17:27 -04:00
Graham Christensen
dde8eeb39a
chdir, setgroups 2019-05-12 13:17:26 -04:00
Graham Christensen
6df61db060
diff hook: execute as the build user, and pass the temp dir 2019-05-12 13:17:26 -04:00
Graham Christensen
c78686e411
build: run diff-hook under --check and document diff-hook 2019-05-12 13:17:24 -04:00
Daiderd Jordan
ff6867ab94
build: move needsHashRewrite initialization to startBuilder
The value of useChroot is not set yet in the constructor, resulting in
hash rewriting being enabled in certain cases where it should not be.

Fixes #2801
2019-05-12 15:53:40 +02:00
Florian Klink
6ade7ec022 progress-bar: hide expected if expected is 0 (unknown)
Sometimes, "expected" can be "0", but in fact means "unknown".

This is for example the case when downloading a file while the http
server doesn't send the `Content-Length` header, like when running `nix
build` pointing to a nixpkgs checkout streamed from GitHub:

⇒  nix build -f https://github.com/NixOS/nixpkgs/archive/master.tar.gz hello
[1.8/0.0 MiB DL] downloading 'https://github.com/NixOS/nixpkgs/archive/master.tar.gz'

In that case, don't show that weird progress bar, but only the (slowly
increasing) downloaded size ("done").

⇒  nix build -f https://github.com/NixOS/nixpkgs/archive/master.tar.gz hello
[1.8 MiB DL] downloading 'https://github.com/NixOS/nixpkgs/archive/master.tar.gz'

This commit also updates fmt calls with three numbers (when something is
currently 'running' too) - I'm not sure if this can be provoked, but
showing "0" as expected doesn't make any sense, as we're obviously doing
more than nothing.
2019-05-11 12:34:39 +02:00
Eelco Dolstra
156e3a9daa
nix dev-shell: Ignore SSL_CERT_FILE 2019-05-11 01:50:28 +02:00
Nick Van den Broeck
d209bdcd08 Fixed issue #47
Content hashes
2019-05-09 14:27:55 +02:00
Eelco Dolstra
8fc1c3f413
Fix assertion failure in FlakeRef::to_string()
Fixes

  $ nix build
  nix: src/libexpr/primops/flakeref.cc:169: std::__cxx11::string nix::FlakeRef::to_string() const: Assertion `!rev' failed.
  Aborted

e.g. when flake.nix doesn't exist.

Also use gitRev().
2019-05-08 22:09:18 +02:00
Eelco Dolstra
455aa8d9ea
Add newline at end of lockfile
Suggested by @grahamc.
2019-05-08 18:28:01 +02:00
Eelco Dolstra
cb5ebc5c11
nix dev-shell: Keep $TERM 2019-05-08 17:07:35 +02:00
Eelco Dolstra
2bc55aba1e
Merge remote-tracking branch 'origin/master' into flakes 2019-05-08 14:30:27 +02:00
Eelco Dolstra
7c6391ddc7
nix-shell: Don't absolutize '-p' expressions
This prevents spurious syscalls like

  25011 lstat("/home/eelco/with import <nixpkgs> { }; (pkgs.runCommandCC or pkgs.runCommand) \"shell\" { buildInputs = [ (hello) ]; } \"\"", 0x7ffe9c67f580) = -1 ENOENT (No such file or directory)
2019-05-08 14:29:36 +02:00
Eelco Dolstra
a887892eb6
nix-shell: Don't fail if run from a path containing the string "nix-shell" 2019-05-08 14:17:32 +02:00
Eelco Dolstra
54e54db2e2
Merge remote-tracking branch 'tweag/flake-test' into flakes 2019-05-08 13:55:55 +02:00
Eelco Dolstra
2fc8a29a9c
Merge remote-tracking branch 'tweag/nix-shell' into flakes 2019-05-08 13:46:44 +02:00
Eelco Dolstra
77e1f9010c
Export missing rev/shortRev attributes 2019-05-08 13:38:32 +02:00
Eelco Dolstra
9d07c3717b
updateLockFile: Do "git add" in a slightly nicer way
"--intent-to-add" ensures the change shows up in "git diff".
2019-05-08 13:26:20 +02:00
Eelco Dolstra
92caa60c49
Merge branch 'repl/ctrlc' of https://github.com/xbreak/nix 2019-05-08 10:18:45 +02:00
Eelco Dolstra
3e940bbf2d
Merge pull request #2790 from samueldr/fix/minor-comment-NIX_ROOT_FINDER
findRootsNoTemp: fixes comment about findRuntimeRoots
2019-05-08 10:13:09 +02:00
Eelco Dolstra
ddd42b7e94 Fix immutable flakeref construction
We were appending ref/revs incorrectly for the IsGit case (by
appending /<ref>/<rev> rather than ?ref=<ref>&rev=<rev).
2019-05-07 23:32:09 +02:00
Eelco Dolstra
2a41a567e2 Improve FlakeRef::to_string()
We were incorrectly using path syntax (i.e. /<ref>/<rev>) for Git
repositories. This is only valid for GitHub flakerefs.
2019-05-07 23:32:09 +02:00
Eelco Dolstra
3c171851a8 Make the URL/path of the global flake registry configurable 2019-05-07 22:49:13 +02:00
Nick Van den Broeck
201f92e02c Fixed Flake data type and flake fetching 2019-05-07 05:57:08 +02:00
Daniel Schaefer
3f192ac80c Add builtins.hashFile
For text files it is possible to do it like so:
`builtins.hashString "sha256" (builtins.readFile /tmp/a)`
but that doesn't work for binary files.

With builtins.hashFile any kind of file can be conveniently hashed.
2019-05-03 17:23:36 +02:00
Eelco Dolstra
f8c4742c2f Fix 'git add' when subdir is empty 2019-05-03 13:15:13 +02:00
Eelco Dolstra
2aafa6901e Merge remote-tracking branch 'tweag/subdir' into flakes 2019-05-03 12:54:59 +02:00
Eelco Dolstra
f9a2ea4486
Fix "Bad system call" running i686-linux binaries on x86_64-linux
To determine which seccomp filters to install, we were incorrectly
using settings.thisSystem, which doesn't denote the actual system when
--system is used.

Fixes #2791.
2019-05-03 10:48:33 +02:00
Eelco Dolstra
7ba0f98e64 nix dev-shell: Less purity 2019-05-02 21:28:52 +02:00
Eelco Dolstra
8ec77614f6 Move createTempFile to libutil 2019-05-02 21:28:41 +02:00
Eelco Dolstra
dea18ff999 nix dev-shell: Execute shellHook 2019-05-02 21:13:19 +02:00
Eelco Dolstra
2919c496ea nix dev-shell: Use 'provides.devShell' by default
Thus

  $ nix dev-shell

will now build the 'provides.devShell' attribute from the flake in the
current directory. If it doesn't exist, it falls back to
'provides.defaultPackage'.
2019-05-02 21:10:13 +02:00
Eelco Dolstra
7dcf5b011a Add function for quoting strings 2019-05-02 21:09:52 +02:00
Eelco Dolstra
2b8c63f303 Add 'nix dev-shell' and 'nix print-dev-env' command
'nix dev-shell' is intended to replace nix-shell. It supports flakes,
e.g.

  $ nix dev-shell nixpkgs:hello

starts a bash shell providing an environment for building 'hello'.

Like Lorri (and unlike nix-shell), it computes the build environment
by building a modified top-level derivation that writes the
environment after running $stdenv/setup to $out and exits. This
provides some caching, so it's faster than nix-shell in some cases
(especially for packages with lots of dependencies, where the setup
script takes a long time).

There also is a command 'nix print-dev-env' that prints out shell code
for setting up the build environment in an existing shell, e.g.

  $ . <(nix print-dev-env nixpkgs:hello)

https://github.com/tweag/nix/issues/21
2019-05-02 20:22:14 +02:00
Nick Van den Broeck
e0d4aa75fc Fixed compile errors 2019-05-02 08:40:00 +02:00
Eelco Dolstra
5d6e8c008b Allow 'dir' parameter in github: URIs
E.g. 'github:edolstra/dwarffs/flake?dir=foo/bar'.
2019-05-01 20:44:30 +02:00
Eelco Dolstra
a37436d792 Accept empty directories 2019-05-01 20:44:30 +02:00
Eelco Dolstra
fa88f71520 Validate 'dir=' parameters
We reject any path element starting with a '.' (mostly to reject '.'
and '..').
2019-05-01 20:44:30 +02:00
Eelco Dolstra
ab9e47284a Improve error message 2019-05-01 20:44:30 +02:00
Nick Van den Broeck
43408d3cd6 flake.lock now uses flakeRef.subdir 2019-05-01 16:24:33 +02:00
Eelco Dolstra
00db8d4549 Support 'dir' parameters in https and ssh flake URIs 2019-05-01 14:24:22 +02:00
Nick Van den Broeck
9b3069a88c Fuzzymatching
Fixed issue #61
2019-05-01 12:59:35 +02:00
Nick Van den Broeck
eba85e2367 WIP: still need to adapt flakeref parsing 2019-05-01 12:59:12 +02:00
Samuel Dionne-Riel
cbc7d9a412 findRootsNoTemp: fixes comment about findRuntimeRoots
The NIX_ROOT_FINDER environment variable was removed in
3c46fe62b8 when porting from perl to C.
2019-04-30 22:43:24 -04:00
Eelco Dolstra
4588a6ff3c Merge remote-tracking branch 'tweag/flakeFlags' into flakes 2019-04-30 12:46:54 +02:00
Nick Van den Broeck
24b35bf9e7 Fixed issue #13 2019-04-30 11:26:45 +02:00
Nick Van den Broeck
35d1c95f7f Fix flag registry order 2019-04-30 11:04:31 +02:00
Eelco Dolstra
e6109ec765 Merge remote-tracking branch 'tweag/fixFlakeList' into flakes 2019-04-25 14:05:40 +02:00
Nick Van den Broeck
d867e1804a Fix printing FlakeList 2019-04-25 11:43:16 +02:00
Nick Van den Broeck
e9c42c06ef Fixed lookupFlake bug 2019-04-25 10:50:59 +02:00
Eelco Dolstra
21d5abfc14 Merge remote-tracking branch 'tweag/flake-clone' into flakes 2019-04-24 12:47:35 +02:00
Eelco Dolstra
cc51e37ad0 Use "." as the default installable
This makes e.g. "nix build" do something more or less reasonable
(namely, build the default package of the flake in the current
directory).
2019-04-19 16:16:04 +02:00
Eelco Dolstra
4fb594a375 Merge remote-tracking branch 'tweag/resolvedFlake' into flakes 2019-04-19 15:56:17 +02:00
Eelco Dolstra
f8a52cc598 nlohmann-json: 3.5.0 -> 3.6.1
https://github.com/nlohmann/json/releases/tag/v3.6.1

This fixes some clang warnings.
2019-04-19 14:53:58 +02:00
Eelco Dolstra
3392f1b778 Shut up clang warning 2019-04-19 14:41:06 +02:00
Nick Van den Broeck
e51abb6631 Changed some names 2019-04-19 14:23:35 +02:00
Eelco Dolstra
50ec2bed9e nix flake info: Show revcount 2019-04-19 14:19:46 +02:00
Eelco Dolstra
bc259192b4 fetchGit: Return revCount for dirty working trees 2019-04-19 14:15:51 +02:00
Eelco Dolstra
0cbda84f5b exportGit: Don't clone local repositories
This ensures that commands like 'nix flake info /my/nixpkgs' don't
copy a gigabyte of crap to ~/.cache/nix.

Fixes #60.
2019-04-19 14:10:57 +02:00
Eelco Dolstra
160ce18a0e Improve missing flake.nix error message 2019-04-19 11:43:56 +02:00
Eelco Dolstra
6960ee929d Clean up exportGit argument handling 2019-04-19 11:34:23 +02:00
Eelco Dolstra
46cb15df9b Fix assertion failure in FlakeRef::to_string() 2019-04-19 11:16:14 +02:00
Eelco Dolstra
6e4210d8ce Fix assertion failure 2019-04-19 10:58:08 +02:00
Daiderd Jordan
b614e0e53d
build: only skip hash rewriting for sandboxing on linux
The sandbox on darwin, and possibly other future platforms, doesn't have
enough isolation to redirect outputs without hash rewriting.
2019-04-17 23:41:23 +02:00
Eelco Dolstra
160b974fb0 Fix mutability check 2019-04-17 13:54:06 +02:00
Nick Van den Broeck
b42ba08fc8 Add command flake clone 2019-04-17 13:31:09 +02:00
Eelco Dolstra
939bee06cd Pass a flake to itself as "self" 2019-04-16 16:29:44 +02:00
Eelco Dolstra
3ddb6d1833 Allow refs to start with a digit
E.g. we want to accept "19.03" as a ref.
2019-04-16 16:24:51 +02:00
Eelco Dolstra
260527a90c Use the lock file 2019-04-16 16:18:47 +02:00
Eelco Dolstra
3c28cb1b8f Improve 'nix flake info' a bit
Example:

  $ nix flake info dwarffs
  ID:          dwarffs
  URI:         github:edolstra/dwarffs/a83d182fe3fe528ed6366a5cec3458bcb1a5f6e1
  Description: A filesystem that fetches DWARF debug info from the Internet on demand
  Revision:    a83d182fe3fe528ed6366a5cec3458bcb1a5f6e1
  Path:        /nix/store/grgd14kxxk8q4n503j87mpz48gcqpqw7-source
2019-04-16 16:18:19 +02:00
Eelco Dolstra
3d0e81051f Fix lock file generation
Before:

   "requires": {
        "nixpkgs": {
            "uri": "nixpkgs"
        }
    },

After:

    "requires": {
        "nixpkgs": {
            "uri": "github:edolstra/nixpkgs/f10e8a02eb7fa2b4a070f30cf87f4efcc7f3186d"
        }
    },
2019-04-16 15:43:55 +02:00
Eelco Dolstra
8c4e759efd updateLockFile(): Make sure Git can see flake.lock 2019-04-16 15:11:17 +02:00
Eelco Dolstra
cfca793a20 Remove unneeded pureEval flags 2019-04-16 15:06:40 +02:00
Eelco Dolstra
60834492ae Update lock files from InstallableFlake::toValue()
This ensures that the lock file is updated *before* evaluating it, and
that it gets updated for any nix command, not just 'nix build'.

Also, while computing the lock file, allow arbitrary registry lookups,
not just at top-level.

Also, improve some error messages slightly.
2019-04-16 15:02:02 +02:00
Eelco Dolstra
7b312a8762 Pass stuff by reference 2019-04-16 14:27:54 +02:00
Eelco Dolstra
e1d73edb10 writeLockFile(): Emit empty objects rather than null 2019-04-16 14:23:10 +02:00
Eelco Dolstra
ed9d725392 getFlake(): Use impureIsAllowed
This fixes 'nix build nixpkgs:hello' without --impure.
2019-04-16 14:16:20 +02:00
Eelco Dolstra
ba66455636 Improve incremental build 2019-04-16 14:10:05 +02:00
Eelco Dolstra
aecf07b1d6 Remove dead function 2019-04-16 14:08:14 +02:00
Eelco Dolstra
035ac44354 Fix makeFlakeValue() 2019-04-16 13:56:08 +02:00
Eelco Dolstra
d8fa2fc429 Add FIXME for pureEval 2019-04-16 12:26:17 +02:00
Eelco Dolstra
3a5493bfe8 Merge remote-tracking branch 'tweag/privateGithub' into flakes 2019-04-16 12:23:56 +02:00
Nick Van den Broeck
7587d62d02 Fixed flake pin issues 2019-04-16 08:38:17 +02:00
Nick Van den Broeck
b3d33b02e3 Added support for private github repositories 2019-04-16 08:14:46 +02:00
Eelco Dolstra
b4e367bf4a FlakeRef::to_string(): Drop the "flake:" prefix
This is unnecessary in most contexts and makes 'nix flake list' output
less readable.
2019-04-15 14:13:10 +02:00
Eelco Dolstra
c179f668e5 Slight cleanup 2019-04-15 14:08:18 +02:00
Eelco Dolstra
f6d684b5e2 getFlakeRegistries(): Return registries regardless of pureEval
This makes e.g. 'nix flake list' work.
2019-04-15 13:48:56 +02:00
Eelco Dolstra
84c12dbd7c Move --impure to MixEvalArgs 2019-04-15 13:45:51 +02:00
Eelco Dolstra
a12cd53567 Merge remote-tracking branch 'tweag/automatedUpdate' into flakes 2019-04-15 12:11:16 +02:00
Eelco Dolstra
bb6e6923f2 Add environment variable NIX_SHOW_SYMBOLS for dumping the symbol table 2019-04-11 23:04:13 +02:00
Nick Van den Broeck
4bf3a8226b Automated lockfile updating with nix build 2019-04-11 14:34:04 +02:00
Nick Van den Broeck
4ad4e48668 FlakeRegistry = FlakeRef -> FlakeRef 2019-04-11 07:08:04 +02:00
Nick Van den Broeck
c64f98b883 FlakeAlias is implemented 2019-04-09 10:16:02 +02:00
Nick Van den Broeck
f39670c631 Took ref and rev out of FlakeRef 2019-04-09 09:51:42 +02:00
Nick Van den Broeck
3ec0c82fab Fixed dependency resolution 2019-04-09 09:51:42 +02:00
Nick Van den Broeck
641db127be FlakeIds are now properly looked up in registries 2019-04-09 09:51:36 +02:00
Nick Van den Broeck
18c019b616 Added nonFlakeRequires and the command nix flake deps 2019-04-09 08:38:09 +02:00
Eelco Dolstra
87033f2c4e Whitespace 2019-04-08 23:58:33 +02:00
Eelco Dolstra
c996e04aca Allow relative paths in flakerefs
Also allow "." as an installable to refer to the flake in the current
directory. E.g.

  $ nix build .

will build 'provides.defaultPackage' in the flake in the current
directory.
2019-04-08 23:52:09 +02:00
Eelco Dolstra
507da65900 Move flake template into a separate file 2019-04-08 23:39:38 +02:00
Eelco Dolstra
47727252ff Add "nix flake init" command for creating a flake 2019-04-08 23:36:12 +02:00
Eelco Dolstra
ee1254d4f5 nix: Add --impure as a shorter alias of --no-pure-eval 2019-04-08 23:19:19 +02:00
Eelco Dolstra
6a4c7fb975 Add path flakeref variant
Unlike file://<path>, this allows the path to be a dirty Git tree, so

  nix build /path/to/flake:attr

is a convenient way to test building a local flake.
2019-04-08 23:09:18 +02:00
Eelco Dolstra
4023ae4cdf nix: Support nixpkgs.<attrpath> for compatibility 2019-04-08 16:22:04 +02:00
Eelco Dolstra
101d964a59 nix: Make -f work for compatibility 2019-04-08 16:11:17 +02:00
Eelco Dolstra
9b7eac332b Merge remote-tracking branch 'origin/master' into flakes 2019-04-08 14:23:48 +02:00
Eelco Dolstra
154244adc6 nix: New installables syntax
The general syntax for an installable is now
<flakeref>:<attrpath>. The attrpath is relative to the flake's
'provides.packages' or 'provides' if the former doesn't yield a
result. E.g.

  $ nix build nixpkgs:hello

is equivalent to

  $ nix build nixpkgs:packages.hello

Also, '<flakeref>:' can be omitted, in which case it defaults to
'nixpkgs', e.g.

  $ nix build hello
2019-04-08 14:21:13 +02:00
Eelco Dolstra
6b0ca8e803 findAlongAttrPath(): Throw AttrPathNotFound 2019-04-08 14:20:58 +02:00
Eelco Dolstra
41ba5135e0 primeCache(): Barf if builds are needed but not allowed
Fixes #2753.
2019-04-01 21:09:49 +02:00
Eelco Dolstra
2bc6304793 getMachines(): Cache result 2019-04-01 21:09:35 +02:00
Eelco Dolstra
f32fbf952d Fix Bison 3.3 warning 2019-03-27 21:09:31 +01:00
Eelco Dolstra
c101b29133 Merge remote-tracking branch 'tweag/flake-registries' into flakes 2019-03-27 14:40:36 +01:00
Eelco Dolstra
be7fd63595 Remove debug line 2019-03-26 14:25:43 +01:00
Eelco Dolstra
f9c7176a87 nix flake add: Handle ~/.config/nix not existing
Fixes

  $ nix flake add fnord github:edolstra/fnord
  error: opening file '/home/eelco/.config/nix/registry.json': No such file or directory
2019-03-26 12:48:57 +01:00
Eelco Dolstra
42be60c6af Merge remote-tracking branch 'tweag/flake-add' into flakes 2019-03-26 12:35:27 +01:00
Eelco Dolstra
edb3836696 Merge remote-tracking branch 'tweag/auto-update-flake' into flakes 2019-03-26 11:50:22 +01:00
Eelco Dolstra
b5565a7081 Merge remote-tracking branch 'origin/master' into flakes 2019-03-26 11:44:14 +01:00
Eelco Dolstra
5c05c238e6
Merge pull request #1828 from zimbatm/isPath
Add isPath primop
2019-03-25 19:58:11 +01:00
Will Dietz
42e2d5e7b7
store-api.hh: add missing include for unordered_map 2019-03-25 09:18:30 +01:00
zimbatm
514b3c7f83
Add isPath primop
this is added for completeness' sake since all the other possible
`builtins.typeOf` results have a corresponding `builtins.is<Type>`
2019-03-24 11:36:49 +01:00
xbreak
fcd7660976 repl: Restore CTRL-C behaviour
Install signal handler during `readline` to handle SIGINT to abort
partially typed expressions.
2019-03-24 09:39:48 +00:00
Nick Van den Broeck
a554f523db Combining registries properly 2019-03-23 13:42:48 +01:00
Nick Van den Broeck
5e4d92d267 Issue #15 is finished 2019-03-22 15:22:18 +01:00
Nick Van den Broeck
e007f367bd Fixed minor things 2019-03-22 11:21:49 +01:00
Nick Van den Broeck
d4ee8afd59 Implemented --flake flag for nix build
Also fixed Eelco's PR comments
2019-03-22 11:16:20 +01:00
Linus Heckemann
2aa89daab3 eval: improve type description for primops and applied primops
This can make type errors a little easier to understand.
2019-03-21 15:31:46 +01:00
Chaker Benhamed
81a23fa7e2 Add a 5 seconds of timeouts to connect to S3.
The default is 1000ms, but we can hit it a lot of we don't have direct
link to AWS (e.g. using VPN).
2019-03-15 13:23:58 +01:00
Eelco Dolstra
1f64f4c7c8
pkg-config files: Use c++17 2019-03-14 14:11:12 +01:00
Eelco Dolstra
ef52ccf035
experimental/optional -> optional 2019-03-14 14:10:52 +01:00
Eelco Dolstra
86f3b94c8c
nix-store --gc --print-roots: Sort output 2019-03-14 13:59:10 +01:00
Eelco Dolstra
578ed7a259
findRoots(): Don't censor for trusted users
They're pretty much root anyway.
2019-03-14 13:53:34 +01:00
Eelco Dolstra
53522cb6ac
findRoots(): Add 'censor' parameter
This is less brittle than filtering paths after the fact in
nix-daemon.
2019-03-14 13:53:34 +01:00
Eelco Dolstra
a3f37d87ea
findRuntimeRoots: Simplify/fix handling of /proc files
Scanning of /proc/<pid>/{exe,cwd} was broken because '{memory:' was
prepended twice. Also, get rid of the whole '{memory:...}' thing
because it's unnecessary, we can just list the file in /proc directly.
2019-03-14 13:30:25 +01:00
Guillaume Maudoux
4f4391193c Fix warning about unused variable 2019-03-10 00:56:09 +01:00
Guillaume Maudoux
38ee16ae9c Unify internal findRootsXxx() api 2019-03-10 00:56:09 +01:00
Guillaume Maudoux
9d7221183a unify naming of roots as links & targets 2019-03-10 00:56:09 +01:00
Guillaume Maudoux
5c56570726 Also obfuscate the number of memory roots 2019-03-10 00:56:09 +01:00
Guillaume Maudoux
fc02b1b3ee Also print rooted path in nix-store -q --roots 2019-03-10 00:56:09 +01:00
Guillaume Maudoux
8574b70342 fixup! Make roots a map of store paths to pinning links 2019-03-10 00:56:09 +01:00
Guillaume Maudoux
ebc86550f9 Make roots a map of store paths to pinning links
This new structure makes more sense as there may be many sources rooting
the same store path. Many profiles can reference the same path but this
is even more true with /proc/<pid>/maps where distinct pids can and
often do map the same store path.
This implementation is also more efficient as the `Roots` map contains
only one entry per rooted store path.
2019-03-10 00:56:09 +01:00
Guillaume Maudoux
a17f86ce3a Obfuscate memory roots for non-root users 2019-03-10 00:56:09 +01:00
Guillaume Maudoux
43331d6344 detail memory roots 2019-03-10 00:56:09 +01:00
Nick Van den Broeck
6542de98c2 Implemented writeRegistry 2019-03-07 12:00:10 +01:00
Nick Van den Broeck
9ff1a9ea65 Implemented json flag for nix flake info 2019-03-07 11:56:43 +01:00
volth
fff8db205c
canBuildLocally: check for features
It could happen that the local builder match the system but lacks some features.
Now it results a failure.
The fix gracefully excludes the local builder from the set of available builders for derivation which requires the feature, so the derivation is built on remote builders only (as though it has incompatible system, like ```aarch64-linux``` when local is x86)
2019-03-06 05:03:25 +00:00
Will Dietz
0963479741
archive.cc: ignore more posix_fallocate "not supported" error codes
Fixes w/musl.
2019-03-01 10:31:17 +08:00
Nick Van den Broeck
cfb6ab80ce Implemented "nix flake info" 2019-02-26 13:12:51 +01:00
Dzmitry Zaitsau
06d6335987 fix indentation 2019-02-26 11:07:37 +01:00
Dzmitry Zaitsau
ac200c3678 Apply param values on S3Helper initialization 2019-02-25 18:06:19 +01:00
Dzmitry Zaitsau
56c18c67d9 Extend S3 URL parsing with parameters extraction 2019-02-25 18:00:55 +01:00
Dzmitry Zaitsau
07f992a74b Extract and expose splitUriAndParams function
which splits a URL into localtor and parameter parts
2019-02-25 17:59:26 +01:00
Eelco Dolstra
d342de02b9
fetchFlake: Use infinite TTL when the revision is specified 2019-02-25 23:23:45 +08:00
Eelco Dolstra
529add316c
downloadCached: Return ETag
This allows fetchFlake() to return the Git revision of a GitHub
archive.
2019-02-25 23:20:50 +08:00
Eelco Dolstra
d8fe447139
Merge pull request #2579 from catern/dumpdb
nix-store: make --dump-db take a list of paths to dump
2019-02-14 13:08:02 +01:00
Eelco Dolstra
6e9182fbc2
Add basic flake lock file support 2019-02-12 22:43:22 +01:00
Eelco Dolstra
e38ec77de8
Interpret all file:// URIs as Git repositories 2019-02-12 22:06:19 +01:00
Eelco Dolstra
beab05851b
nix: Add --flake flag
This allows using an arbitrary "provides" attribute from the specified
flake. For example:

  nix build --flake nixpkgs packages.hello

(Maybe provides.packages should be used for consistency...)
2019-02-12 21:55:43 +01:00
Eelco Dolstra
272b58220d
Enforce use of immutable flakes in pure mode
... plus a temporary hack to allow impure flakes at top-level for the
default installation source.
2019-02-12 21:07:48 +01:00
Eelco Dolstra
ba05f29838
nix: Enable pure mode by default
We want to encourage a brave new world of hermetic evaluation for
source-level reproducibility, so flakes should not poke around in the
filesystem outside of their explicit dependencies.

Note that the default installation source remains impure in that it
can refer to mutable flakes, so "nix build nixpkgs.hello" still works
(and fetches the latest nixpkgs, unless it has been pinned by the
user).

A problem with pure evaluation is that builtins.currentSystem is
unavailable. For the moment, I've hard-coded "x86_64-linux" in the
nixpkgs flake. Eventually, "system" should be a flake function
argument.
2019-02-12 20:35:03 +01:00
Eelco Dolstra
91a6a47b0e
Improve flake references 2019-02-12 18:23:11 +01:00
Eelco Dolstra
7a7ec22298
Merge pull request #2628 from shlevy/context-introspection
Context introspection
2019-02-12 17:57:35 +01:00
Eelco Dolstra
0cd7f2cd8d
pkg-config files: Use c++17 2019-02-12 13:44:20 +01:00
Eelco Dolstra
c8a0b9d5cb
experimental/optional -> optional 2019-02-12 13:43:32 +01:00
Eelco Dolstra
52419f8db3
Add flake registry
This will eventually be moved to nixos.org.
2019-02-11 13:01:39 +01:00
Eelco Dolstra
dcae46ab14
Add github URIs
For example, github:edolstra/dwarffs is more-or-less equivalent to
https://github.com/edolstra/dwarffs.git. It's a much faster way to get
GitHub repositories: it fetches tarballs rather than entire Git
repositories. It also allows fetching specific revisions by hash
without specifying a ref (e.g. a branch name):

  github:edolstra/dwarffs/41c0c1bf292ea3ac3858ff393b49ca1123dbd553
2019-02-11 12:05:17 +01:00
Eelco Dolstra
ef4cf4e681
Introduce flake URIs 2019-02-11 12:05:13 +01:00
Eelco Dolstra
7a5cf31060
Initial flake support 2019-02-11 12:00:13 +01:00
Eelco Dolstra
f216c76c56
Bindings::get(): Add convenience method
This allows writing attribute lookups as

    if (auto name = value.attrs->get(state.sName))
      ...
2019-02-11 11:55:35 +01:00
Eelco Dolstra
c02da99757
EvalState::allocAttr(): Add convenience method 2019-02-11 11:55:29 +01:00
Eelco Dolstra
f70434b1fb
Move Command and MultiCommand to libutil 2019-02-11 11:55:25 +01:00
Eelco Dolstra
15a16e5c05
MultiCommand: Simplify construction 2019-02-11 11:55:22 +01:00
Eelco Dolstra
aa0e2a2e70
Make constant primops lazy 2019-02-11 11:52:07 +01:00
Eelco Dolstra
01d07b1e92
Revert "Restore parent mount namespace before executing a child process"
This reverts commit a0ef21262f. This
doesn't work in 'nix run' and nix-shell because setns() fails in
multithreaded programs, and Boehm GC mark threads are uncancellable.

Fixes #2646.
2019-02-05 10:49:19 +01:00
Shea Levy
b30be6b450
Add builtins.appendContext.
A partner of builtins.getContext, useful for the same reasons.
2019-01-31 08:52:23 -05:00
Spencer Baugh
5f1891b795 nix-store: make --dump-db take a list of paths to dump
Inside a derivation, exportReferencesGraph already provides a way to
dump the Nix database for a specific closure. On the command line,
--dump-db gave us the same information, but only for the entire Nix
database at once.

With this change, one can now pass a list of paths to --dump-db to get
the Nix database dumped for just those paths. (The user is responsible
for ensuring this is a closure, like for --export).

Among other things, this is useful for deploying a closure to a new
host without using --import/--export; one can use tar to transfer the
store paths, and --dump-db/--load-db to transfer the validity
information. This is useful if the new host doesn't actually have Nix
yet, and the closure that is being deployed itself contains Nix.
2019-01-23 01:24:34 +00:00
Eelco Dolstra
7cc1a2593e
unsupported(): Show the name of the unsupported operation 2019-01-18 13:34:53 +01:00
Shea Levy
1d757292d0
Add builtins.getContext.
This can be very helpful when debugging, as well as enabling complex
black magic like surgically removing a single dependency from a
string's context.
2019-01-14 11:27:10 -05:00
Shea Levy
087be7281a
Treat plain derivation paths in context as normal paths.
Previously, plain derivation paths in the string context (e.g. those
that arose from builtins.storePath on a drv file, not those that arose
from accessing .drvPath of a derivation) were treated somewhat like
derivaiton paths derived from .drvPath, except their dependencies
weren't recursively added to the input set. With this change, such
plain derivation paths are simply treated as paths and added to the
source inputs set accordingly, simplifying context handling code and
removing the inconsistency. If drvPath-like behavior is desired, the
.drv file can be imported and then .drvPath can be accessed.

This is a backwards-incompatibility, but storePath is never used on
drv files within nixpkgs and almost never used elsewhere.
2019-01-13 11:29:55 -05:00
Eelco Dolstra
c7bf1cdb4e
Merge pull request #2608 from dtzWill/fix/issue-2546
EvalState::resetFileCache: clear parse cache as well as eval cache
2019-01-10 20:56:31 +01:00
Eelco Dolstra
30906122e2
Merge pull request #2617 from veprbl/pr/V547_nix_repl.cc
remove some dead code in nix/repl.cc
2019-01-10 20:55:27 +01:00
Eelco Dolstra
013dd28b15
Merge pull request #2601 from dtzWill/update/json-3.5.0
Update nlohmann-json 3.0.1 -> 3.5.0
2019-01-07 23:13:35 +01:00
John Ericson
fef9f5653b Remove mentions of libformat, it no longer exists 2019-01-05 14:31:29 -05:00
John Ericson
e10d6ed2a7 brotli is only used as a library now 2019-01-05 14:25:54 -05:00
Will Dietz
21ea00d3ec EvalState::resetFileCache: clear parse cache as well as eval cache
Fixes #2546.

(at least the basic reproduction I've been testing)
2018-12-31 10:18:28 -06:00
Dmitry Kalinkin
93c9ba3e78 remove some dead code in nix/repl.cc
Fixes a static analyzer warning:

Expression 'isDrv' is always false.

Fixes: 3beb6f6e7 ('Show derivations more concisely')
2018-12-25 22:38:40 +03:00
Will Dietz
3f8b78a84d nlohmann-json: 3.4.0 -> 3.5.0
https://github.com/nlohmann/json/releases/tag/v3.5.0
2018-12-21 22:38:06 -06:00
Will Dietz
e78511743e nlohmann: 3.0.1 -> 3.4.0
```
$ curl -L
"https://github.com/nlohmann/json/releases/download/v3.4.0/json.hpp" -o
src/nlohmann/json.hpp
```
2018-12-21 22:36:53 -06:00
Will Dietz
c1112ae9a2 nix-store: remove debugging print 2018-12-15 09:59:47 -06:00
Daiderd Jordan
7e35e914c1
fetchGit: allow fetching explicit refs
Trying to fetch refs that are not in refs/heads currently fails because
it looks for refs/heads/refs/foo instead of refs/foo.

eg.

	builtins.fetchGit {
	  url = https://github.com/NixOS/nixpkgs.git;
	  ref = "refs/pull/1024/head;
	}
2018-12-14 20:12:22 +01:00
Eelco Dolstra
6024dc1d97
Support SRI hashes
SRI hashes (https://www.w3.org/TR/SRI/) combine the hash algorithm and
a base-64 hash. This allows more concise and standard hash
specifications. For example, instead of

  import <nix/fetchurl.nl> {
    url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz;
    sha256 = "5d22dad058d5c800d65a115f919da22938c50dd6ba98c5e3a183172d149840a4";
  };

you can write

  import <nix/fetchurl.nl> {
    url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz;
    hash = "sha256-XSLa0FjVyADWWhFfkZ2iKTjFDda6mMXjoYMXLRSYQKQ=";
  };

In fixed-output derivations, the outputHashAlgo is no longer mandatory
if outputHash specifies the hash (either as an SRI or in the old
"<type>:<hash>" format).

'nix hash-{file,path}' now print hashes in SRI format by default. I
also reverted them to use SHA-256 by default because that's what we're
using most of the time in Nixpkgs.

Suggested by @zimbatm.
2018-12-13 14:30:52 +01:00
Eelco Dolstra
378e89360d
Merge pull request #2576 from LnL7/repl-no-link
nix repl: don't create result symlinks
2018-12-13 10:40:39 +01:00
volth
21d494da83
probably typo
...at least MSVC unable to compile this
2018-12-13 02:45:50 +00:00
Daiderd Jordan
0e6c84a771
nix repl: don't create result symlinks 2018-12-12 23:00:07 +01:00
zimbatm
5e6fa9092f
libstore: improve hash mismatch error messages
Use the same output ordering and format everywhere.

This is such a common issue that we trade the single-line error message for
more readability.

Old message:
```
fixed-output derivation produced path '/nix/store/d4nw9x2sy9q3r32f3g5l5h1k833c01vq-example.com' with sha256 hash '08y4734bm2zahw75b16bcmcg587vvyvh0n11gwiyir70divwp1rm' instead of the expected hash '1xzwnipjd54wl8g93vpw6hxnpmdabq0wqywriiwmh7x8k0lvpq5m'
```

New message:
```
hash mismatch in fixed-output derivation '/nix/store/d4nw9x2sy9q3r32f3g5l5h1k833c01vq-example.com':
  wanted: sha256:1xzwnipjd54wl8g93vpw6hxnpmdabq0wqywriiwmh7x8k0lvpq5m
  got:    sha256:08y4734bm2zahw75b16bcmcg587vvyvh0n11gwiyir70divwp1rm

```
2018-12-12 17:42:00 +01:00
Eelco Dolstra
f7425d55df
Fix assertion failure in NarInfoDiskCache
https://hydra.nixos.org/build/85827920
2018-12-12 17:13:22 +01:00
Daiderd Jordan
6f89053108
nar-info-disk-cache: include ca in the cache entries
Without this information the content addressable state and hashes are
lost after the first request, this causes signatures to be required for
everything even tho the path could be verified without signing.
2018-12-12 01:08:47 +01:00
Daiderd Jordan
898823b67d
s3: make scheme configurable
This enables using for http for S3 request for debugging or
implementations that don't have https configured.  This is not a problem
for binary caches since they should not contain sensitive information.
Both package signatures and AWS auth already protect against tampering.
2018-12-07 23:55:38 +01:00
CHEIKH Chawki
fa5143c722 Solve hg "abandoned transaction" issue 2018-12-06 13:57:59 +01:00
Eelco Dolstra
4aee93d5ce
fetchGit: Drop unnecessary localRef 2018-11-20 20:59:44 +01:00
Eelco Dolstra
3f4de91d80
Merge branch 'better-git-cache' of https://github.com/graham-at-target/nix 2018-11-20 20:41:19 +01:00
Eelco Dolstra
ebeea068d5
Merge pull request #2216 from dtzWill/fix/curl-760-timeout
download: if there are active requests, never sleep for 10s
2018-11-20 19:07:56 +01:00
Eelco Dolstra
5e64470b19
Fix typo 2018-11-20 16:54:17 +01:00
Kai Harries
de5997332d repl: give user the choice between libeditline and libreadline
The goal is to support libeditline AND libreadline and let the user
decide at compile time which one to use.

Add a compile time option to use libreadline instead of
libeditline. If compiled against libreadline completion functionality
is lost because of a incompatibility between libeditlines and
libreadlines completion function. Completion with libreadline is
possible and can be added later.

To use libreadline instead of libeditline the environment
variables 'EDITLINE_LIBS' and 'EDITLINE_CFLAGS' have to been set
during the ./configure step.

Example:

  EDITLINE_LIBS="/usr/lib/x86_64-linux-gnu/libhistory.so /usr/lib/x86_64-linux-gnu/libreadline.so"
  EDITLINE_CFLAGS="-DREADLINE"

The reason for this change is that for example on Debian already three
different editline libraries exist but none of those is compatible the
flavor used by nix. My hope is that with this change it would be
easier to port nix to systems that have already libreadline available.
2018-11-20 15:43:21 +01:00
Kai Harries
b289d86cd1 repl: Remove code that was commented out 2018-11-20 15:42:58 +01:00
Linus Heckemann
40e0c9e925 nix ls-nar: allow reading from FIFOs
fixes #2528
2018-11-16 16:24:36 +01:00
Eelco Dolstra
a32ff2573b
Fix 'Read-only file system' when building a derivation 2018-11-15 13:20:39 +01:00
Eelco Dolstra
6c6bbeb439 Merge branch 'feature/editline-pr' of https://github.com/dtzWill/nix 2018-11-13 20:42:13 +01:00
Eelco Dolstra
a0ef21262f
Restore parent mount namespace before executing a child process
This ensures that they can't write to /nix/store. Fixes #2535.
2018-11-13 16:15:30 +01:00
Eelco Dolstra
7751616645
nix-prefetch-url: Stop progress bar before printing results 2018-11-09 10:34:12 +01:00
Eelco Dolstra
220c79ec22
Merge branch 'feature/prefetch-progress' of https://github.com/dtzWill/nix 2018-11-09 10:29:59 +01:00
Eelco Dolstra
812e39313c
Enable sandboxing by default
Closes #179.
2018-11-07 17:08:28 +01:00
Eelco Dolstra
6323b0729a
Disable the S3 content-encoding patch
Since we're not using multi-part uploads at the moment, we can drop
this patch.
2018-11-01 15:18:38 +01:00
Jan Path
d1b049c4ea Fix typo in comments 2018-10-31 20:50:18 +01:00
Eelco Dolstra
fb35aaa422
Merge pull request #2501 from mayflower/xdg_config_dirs
config: use all of XDG_CONFIG_DIRS
2018-10-31 12:37:23 +01:00
Linus Heckemann
f3b8173a93 config: use all of XDG_CONFIG_DIRS
Previously, config would only be read from XDG_CONFIG_HOME. This change
allows reading config from additional directories, which enables e.g.
per-project binary caches or chroot stores with the help of direnv.
2018-10-31 09:24:43 +01:00
Eelco Dolstra
38d76d6d78
Remove redundant message 2018-10-30 19:28:42 +01:00
Eelco Dolstra
9f99d62480
S3BinaryCacheStore: Allow disabling multipart uploads
The use of TransferManager has several issues, including that it
doesn't allow setting a Content-Encoding without a patch, and it
doesn't handle exceptions in worker threads (causing termination on
memory allocation failure).

Fixes #2493.
2018-10-30 14:25:00 +01:00
Eelco Dolstra
0163e8928c
Fix broken uploadProgressCallback closure
Since the callback is global we can't refer to 'path' in it. This
could cause a segfault or printing of arbitrary data.
2018-10-30 11:45:31 +01:00
Eelco Dolstra
056c3fbbfc
Merge pull request #2494 from FPtje/fix-copy-from-old
copyStorePath: Fix hash errors when copying from older store
2018-10-29 20:45:25 +01:00
Falco Peijnenburg
49e272f647 copyStorePath: Fix hash errors when copying from older store
This commit partially reverts 48662d151b. When
copying from an older store (in my case a store running Nix 1.11.7), nix would
throw errors about there being no hash. This is fixed by recalculating the hash.
2018-10-29 20:24:37 +01:00
Will Dietz
9f998096d2 repl: complete if all matches share prefix 2018-10-29 08:45:04 -05:00
Will Dietz
3d974d31fa editline: wip 2018-10-29 08:44:58 -05:00
Daiderd Jordan
8e6bf49297
nix doctor: return nonzero exitcode if a check fails
This makes it easier to use this when testing the installer or when
running the checks with other automated tooling.
2018-10-29 12:09:22 +00:00
Eelco Dolstra
18b4c53f71
Restore old (dis)allowedRequisites behaviour for self-references
stdenv relies on this. So ignore self-references (but only in legacy non-structured attributes mode).
2018-10-27 15:41:53 +02:00
Eelco Dolstra
1427958b3c
Merge branch 'simplify-lexer' of https://github.com/layus/nix 2018-10-27 13:42:06 +02:00
Eelco Dolstra
0c61515be1
Fix signedness warning 2018-10-27 13:07:26 +02:00
Eelco Dolstra
63575ffa38
Merge branch 'nix-doctor' of https://github.com/LnL7/nix 2018-10-27 12:54:22 +02:00
Eelco Dolstra
2cf98218c8
Merge pull request #2488 from dtzWill/fix/struct-class-minor
remote-store.hh: ConnectionHandle is struct, minor fix warning
2018-10-27 12:50:16 +02:00
Guillaume Maudoux
6a5bf9b143 simplify handling of extra '}' 2018-10-27 00:14:51 +02:00
Will Dietz
9d24b5d56e nix-prefetch-url: progressbar 2018-10-26 13:19:10 -05:00
Will Dietz
3283c0dc45 remote-store.hh: ConnectionHandle is struct, minor fix warning 2018-10-26 13:15:48 -05:00
Will Dietz
27c2fcd4c0 src/nix/local.mk: fix typos in names of symlinks 2018-10-26 13:15:07 -05:00
Eelco Dolstra
f6a3dfe4e0
Merge all nix-* binaries into nix
These are all symlinks to 'nix' now, reducing the installed size by
about ~1.7 MiB.
2018-10-26 12:54:00 +02:00
Eelco Dolstra
c47e14ee45
Merge pull request #2483 from mayflower/tail-config
Promote verbose-build and log-lines
2018-10-23 16:59:52 +02:00
Linus Heckemann
a25abe823f Promote log-lines to a fully-qualified option
This allows commands like

nix build --log-lines 30 nixpkgs.hello

in order to obtain more information in case of a failure.
2018-10-23 16:54:24 +02:00
Eelco Dolstra
1a08ad75ea
Merge pull request #2479 from nlewo/graphml
Add --graphml option to the nix-store --query command
2018-10-23 15:56:05 +02:00
Eelco Dolstra
3cd15c5b1f
Per-output reference and closure size checks
In structured-attributes derivations, you can now specify per-output
checks such as:

  outputChecks."out" = {
    # The closure of 'out' must not be larger than 256 MiB.
    maxClosureSize = 256 * 1024 * 1024;

    # It must not refer to C compiler or to the 'dev' output.
    disallowedRequisites = [ stdenv.cc "dev" ];
  };

  outputChecks."dev" = {
    # The 'dev' output must not be larger than 128 KiB.
    maxSize = 128 * 1024;
  };

Also fixed a bug in allowedRequisites that caused it to ignore
self-references.
2018-10-23 01:29:16 +02:00
Antoine Eiche
d506342aa2 Remove the --xml query command option
The `--graphml` option can be used instead.
2018-10-20 09:48:53 +02:00
Antoine Eiche
73c2ae43f0 Add --graphml option to the nix-store --query command
This prints the references graph of the store paths in the graphML
format [1]. The graphML format is supported by several graph tools
such as the Python Networkx library or the Apache Thinkerpop project.

[1] http://graphml.graphdrawing.org
2018-10-20 09:48:34 +02:00
Eelco Dolstra
bd78544f66 Fix assertion failure in Store::queryPathInfo()
$ nix-store -qR /nix/store/fnord
  nix-store: src/libstore/store-api.cc:80: std::__cxx11::string nix::storePathToHash(const Path&): Assertion `base.size() >= storePathHashLen' failed.
  Aborted
2018-10-16 23:39:36 +02:00
Eelco Dolstra
79e358ce6d RemoteStore: Close connection if an exception occurs
Fixes #2075.
2018-10-16 23:36:15 +02:00
Eelco Dolstra
ba51100d64 Get rid of UDSRemoteStore::Connection
Since its superclass RemoteStore::Connection contains 'to' and 'from'
fields that refer to the file descriptor maintained in the subclass,
it was possible for the flush() call in Connection::~Connection() to
write to a closed file descriptor (or worse, a file descriptor now
referencing another file). So make sure that the file descriptor
survives 'to' and 'from'.
2018-10-16 22:58:01 +02:00
Eelco Dolstra
01bd66bf83
Merge pull request #2452 from ElvishJerricco/verify-sigs-overflow
Fix overflow when verifying signatures of content addressable paths
2018-10-08 12:03:03 +02:00
Eelco Dolstra
0fda9b22c7
Remove unnecessary typecast 2018-10-04 13:16:30 +02:00
Eelco Dolstra
1e7b8deea7
Check requiredSystemFeatures for local builds
For example, this prevents a "kvm" build on machines that don't have
KVM.

Fixes #2012.
2018-09-28 16:10:27 +02:00
Eelco Dolstra
7ae7a38c9a
Move structured attrs handling into a separate class
This is primarily because Derivation::{can,will}BuildLocally() depends
on attributes like preferLocalBuild and requiredSystemFeatures, but it
can't handle them properly because it doesn't have access to the
structured attributes.
2018-09-28 14:32:46 +02:00
Eelco Dolstra
99d4bb2d4c
Remove obsolete check on system features 2018-09-28 13:28:15 +02:00
Eelco Dolstra
c9ba33870e
Support special attributes in structured attributes derivations
E.g. __noChroot and allowedReferences now work correctly. We also now
check that the attribute type is correct. For instance, instead of

  allowedReferences = "out";

you have to write

  allowedReferences = [ "out" ];

Fixes #2453.
2018-09-28 12:43:01 +02:00
Eelco Dolstra
63786cbd3b
S3BinaryCacheStore: Don't create buckets
This meant that making a typo in an s3:// URI would cause a bucket to
be created. Also it didn't handle eventual consistency very well. Now
it's up to the user to create the bucket.
2018-09-27 21:34:41 +02:00
Eelco Dolstra
f74e0b4786
Merge pull request #2455 from grahamc/add-interactive-path
nix-shell: add bashInteractive to the start of the PATH, set SHELL
2018-09-27 21:15:50 +02:00
Graham Christensen
f11acbaf17
nix-shell: add bashInteractive to the start of the PATH, set SHELL
Tools which re-exec `$SHELL` or `$0` or `basename $SHELL` or even just
`bash` will otherwise  get the non-interactive bash, providing a
broken shell for the same reasons described in
https://github.com/NixOS/nixpkgs/issues/27493.

Extends c94f3d5575
2018-09-27 15:12:56 -04:00
Eelco Dolstra
7cba4214a8
Don't talk about a "current folder build output"
Presumably this refers to ./default.nix but the support for that in
'nix' is tenuous. Also folders are a Mac thing.
2018-09-27 21:01:19 +02:00
Daiderd Jordan
ed25753501
nix doctor: reimplement profile warning without gcroot check
Calculating roots seems significantly slower on darwin compared to
linux.  Checking for /profile/ links could show some false positives but
should still catch most issues.
2018-09-26 23:13:17 +02:00
Eelco Dolstra
9750430003
Ensure download thread liveness
* Don't wait forever for the client to remove data from the
  buffer. This does mean that the buffer can grow without bounds
  (e.g. when downloading is faster than writing to disk), but meh.

* Don't hold the state lock while calling the sink. The sink could
  take any amount of time to process the data (in particular when it's
  actually a coroutine), so we don't want to block the download
  thread.
2018-09-26 21:47:34 +02:00
Eelco Dolstra
98b2cc2e6e
Untabify 2018-09-26 21:39:06 +02:00
Eelco Dolstra
ed78582847
sinkToSource(): Start the coroutine lazily
In particular this causes copyStorePath() from HttpBinaryCacheStore to
only start a download if needed. E.g. if the destination LocalStore
goes to sleep waiting for the path lock and another process creates
the path, then LocalStore::addToStore() will never read from the
source so we don't have to do the download.
2018-09-26 21:19:34 +02:00
Eelco Dolstra
44e86304b6
Make NAR header check more robust
Changes

  std::bad_alloc

into

  bad archive: input doesn't look like a Nix archive
2018-09-26 12:03:58 +02:00
Will Fancher
0ae8d4033d Fix overflow when verifying signatures of content addressable paths 2018-09-25 21:19:24 -04:00
Will Dietz
8df367a92d nix-daemon: allow setting builders to "" by any user (untrusted)
(cherry picked from commit a94a2eb1cb1c81e90a7529be5fecac27899a3442)
2018-09-25 03:22:08 -05:00
Eelco Dolstra
42e4ea2b6d
Merge pull request #2312 from dtzWill/fix/matched-names-should-still-print-version
search: include version for matches too!
2018-09-24 13:03:28 +02:00
Eelco Dolstra
52f6d541b9
nix-env: Fix segfault if -f argument is not a directory or a Nix expression
Fixes #2425.
2018-09-17 16:36:30 +02:00
Eelco Dolstra
901dfc7978
Fix build failure if parallel xz is not available 2018-09-17 16:36:11 +02:00
Eelco Dolstra
8327a7a8fa
Merge branch 'dirOf-relative' of https://github.com/lheckemann/nix 2018-09-13 14:33:12 +02:00
Ding Xiang Fei
d6ac762bf7 auto args on repl 2018-09-10 15:43:17 +08:00
Eelco Dolstra
bba3f0a308
If fallback is enabled, temporarily disable substituters after a failure
Otherwise, we just keep asking the substituter for other .narinfo
files, which can take a very long time due to retries/timeouts.
2018-09-07 17:08:43 +02:00
Eelco Dolstra
33c3f91885
Handle queryPathInfo() failure from substituters when fallback is enabled
Fixes #1990.
2018-09-07 16:35:48 +02:00
Eelco Dolstra
91405986f4
Convert NIX_COUNT_CALLS to JSON too 2018-09-05 21:57:54 +02:00
Eelco Dolstra
0a2545f95c
Log stats to stderr
We shouldn't pollute stdout.
2018-09-05 21:35:58 +02:00
Eelco Dolstra
c4ba5f4be9
Merge branch 'improve-stats' of https://github.com/cleverca22/nix 2018-09-05 21:28:05 +02:00
Eelco Dolstra
2d91012754
fetchurl: Respect unpack
Fixes #2393.
2018-09-05 21:22:37 +02:00
Matthew Bauer
74f6d8767d Get effective user in Nix commands
‘geteuid’ gives us the user that the command is being run as,
including in setuid modes. By using geteuid to determind id, we can
avoid the ‘sudo -i’ hack when upgrading Nix. So now, upgrading Nix on
macOS is as simple as:

$ sudo nix-channel --update
$ sudo nix-env -u

$ sudo launchctl stop org.nixos.nix-daemon
$ sudo launchctl start org.nixos.nix-daemon
or
$ sudo systemctl restart nix-daemon
2018-09-04 19:32:39 -05:00
Will Dietz
28418af920 download: fix size reported to progress bar 2018-09-03 14:12:54 -05:00
Michael Bishop
4b034f390c remove the old text format output 2018-09-02 18:25:23 -03:00
Daiderd Jordan
80a4b44d3d
nix doctor: only perform path/profile checks with a daemon/local store
Not all store types LegacySSHStore support these operations and it
doesn't really make sense to check those.
2018-09-02 13:41:55 +02:00
Daiderd Jordan
bfdca55868
nix doctor: add check for profile roots
In most cases profiles that are in PATH should have a gcroot.
2018-09-02 12:56:28 +02:00
Daiderd Jordan
0f18dc5479
nix doctor: add warning for multiple versions
It's pretty easy to unintentionally install a second version of nix
into the user profile when using a daemon install.  In this case it
looks like nix was upgraded while the nix-daemon is probably still
unning an older version.
2018-09-02 12:56:28 +02:00
Daiderd Jordan
246acf93f2
nix doctor: handle serve protocol
The serve protocol used by LegacySSHStore has a different major and
shouldn't be compared to PROTOCOL_VERSION.
2018-09-02 12:54:42 +02:00
Daiderd Jordan
7314dc7f07
nix doctor: add warning if client/daemon protocol mismatches
A protocol mismatch can sometimes cause problems when using specific
features with an older daemon. For example:

Nix 2.0 changed the way files are compied to the store.  The daemon is
backwards compatible and can still handle older clients, however a 1.11
nix-daemon isn't forwards compatible.
2018-09-02 12:54:39 +02:00
Daiderd Jordan
070823baa4
Store: expose the protocol version used by a store 2018-09-02 12:54:29 +02:00
Daiderd Jordan
c9a08540c3
nix doctor: add command
Inspired by the homebrew command, shows a combination of debugging
information and warnings with potential issues with a nix installation.
2018-09-02 12:54:20 +02:00
Michael Bishop
2fd1008c70 add JSON to NIX_SHOW_STATS 2018-09-01 20:05:06 -03:00
Michael Bishop
c29e5fbb13 improve the stats when profiling 2018-09-01 17:11:56 -03:00
Graham Christensen
87702532d2
nix-build: Print stats even in failing builds 2018-08-31 21:03:32 -04:00
Graham Christensen
b7bb627f67
nix-build: print stats on sucessful builds 2018-08-31 20:57:07 -04:00
Eelco Dolstra
475a0a54a9
fetchGit/fetchMercurial: Don't absolutize paths
This is already done by coerceToString(), provided that the argument
is a path (e.g. 'fetchGit ./bla'). It fixes the handling of URLs like
git@github.com:owner/repo.git. It breaks 'fetchGit "./bla"', but that
was never intended to work anyway and is inconsistent with other
builtin functions (e.g. 'readFile "./bla"' fails).
2018-09-01 00:19:49 +02:00
Eelco Dolstra
b7409c5754
nix path-info: Remove trailing spaces
Fixes #2390.
2018-09-01 00:04:57 +02:00
Eelco Dolstra
d1f36e8787
Really fix the build 2018-08-31 19:20:08 +02:00
Eelco Dolstra
ef09da58f2
nix path-info: Hopefully fix macOS build failure
https://hydra.nixos.org/build/80480356
2018-08-31 17:30:45 +02:00
Eelco Dolstra
c0c31b58a4
Add localhost alias for ::1 to the sandbox 2018-08-31 11:12:51 +02:00
Michael Bishop
0767e402f1 fix error: unknown serve command 9 2018-08-30 20:10:08 -03:00
Eelco Dolstra
39f1722f36
nix upgrade-nix: Handle .nix-profile being a link to .../profiles/per-user/...
Also some cosmetic improvements.
2018-08-30 21:18:56 +02:00
Eelco Dolstra
f08b14c9d0
nix upgrade-nix: Improve error message if 'nix' is not in a profile
E.g.

  $ nix upgrade-nix
  error: directory '/home/eelco/Dev/nix/inst/bin' does not appear to be part of a Nix profile

instead of

  $ nix upgrade-nix
  error: '/home/eelco/Dev/nix/inst' is not a symlink
2018-08-30 17:14:17 +02:00
Eelco Dolstra
ddeda0b62e
Merge branch 'nix-upgrade-profile' of https://github.com/LnL7/nix 2018-08-30 17:10:59 +02:00
Eelco Dolstra
264e66f696
Add simple test for nix upgrade-nix 2018-08-30 17:10:28 +02:00
Eelco Dolstra
99828245f8
printSize() fixes
Fix a 32-bit overflow that resulted in negative numbers being printed;
use fmt() instead of boost::format(); change -H to -h for consistency
with 'ls' and 'du'; make the columns narrower (since they can't be
bigger than 1024.0).
2018-08-30 16:22:04 +02:00
Benjamin Hipple
c908df881f
Avoid overflow and use boost::format
If the user has an object greater than 1024 yottabytes, it'll just display it as
N yottabytes instead of overflowing.

Swaps to use boost::format strings instead of std::setw and std::setprecision.
2018-08-30 16:22:04 +02:00
Benjamin Hipple
3407a5d936
Add human readable closure sizes to nix path-info
Unfortunately, -h is already taken as a short option by --help, so we have to
use a different letter or the capitalized version.

Resolves #2363
2018-08-30 16:22:04 +02:00
Eelco Dolstra
5e83b0227f
nix: Remove the -h flag 2018-08-30 16:14:19 +02:00
Eelco Dolstra
1286d86b95
Merge pull request #2336 from ivan/repl-trailing-spaces
repl: don't add trailing spaces to history lines
2018-08-29 18:13:29 +02:00
aszlig
0ad643ed5c
libexpr: Use int64_t for NixInt
Using a 64bit integer on 32bit systems will come with a bit of a
performance overhead, but given that Nix doesn't use a lot of integers
compared to other types, I think the overhead is negligible also
considering that 32bit systems are in decline.

The biggest advantage however is that when we use a consistent integer
size across all platforms it's less likely that we miss things that we
break due to that. One example would be:

https://github.com/NixOS/nixpkgs/pull/44233

On Hydra it will evaluate, because the evaluator runs on a 64bit
machine, but when evaluating the same on a 32bit machine it will fail,
so using 64bit integers should make that consistent.

While the change of the type in value.hh is rather easy to do, we have a
few more options available for doing the conversion in the lexer:

  * Via an #ifdef on the architecture and using strtol() or strtoll()
    accordingly depending on which architecture we are. For the #ifdef
    we would need another AX_COMPILE_CHECK_SIZEOF in configure.ac.
  * Using istringstream, which would involve copying the value.
  * As we're already using boost, lexical_cast might be a good idea.

Spoiler: I went for the latter, first of all because lexical_cast does
have an overload for const char* and second of all, because it doesn't
involve copying around the input string. Also, because istringstream
seems to come with a bigger overhead than boost::lexical_cast:

https://www.boost.org/doc/libs/release/doc/html/boost_lexical_cast/performance.html

The first method (still using strtol/strtoll) also wasn't something I
pursued further, because it is also locale-aware which I doubt is what
we want, given that the regex for int is [0-9]+.

Signed-off-by: aszlig <aszlig@nix.build>
Fixes: #2339
2018-08-29 01:05:52 +02:00
Daiderd Jordan
d85bb4814f
upgrade-nix: resolve profile symlinks
The profile present in PATH is not necessarily the actual profile
location. User profiles are generally added as $HOME/.nix-profile
in which case the indirect profile link needs to be resolved first.

/home/user/.nix-profile -> /nix/var/nix/profiles/per-user/user/profile
/nix/var/nix/profiles/per-user/user/profile -> profile-15-link
/nix/var/nix/profiles/per-user/user/profile-14-link -> /nix/store/hyi4kkjh3bwi2z3wfljrkfymz9904h62-user-environment
/nix/var/nix/profiles/per-user/user/profile-15-link -> /nix/store/6njpl3qvihz46vj911pwx7hfcvwhifl9-user-environment

To upgrade nix here we want /nix/var/nix/profiles/per-user/user/profile-16-link
instead of /home/user/.nix-profile-1-link. The latter is not a gcroot
and would be garbage collected, resulting in a broken profile.

Fixes #2175
2018-08-25 20:50:44 +02:00
Daiderd Jordan
414397759a
upgrade-nix: add --dry-run 2018-08-25 20:25:43 +02:00
Eelco Dolstra
c651b7bdc9
Revert "Fix parser/lexer generation with parallel make"
This reverts commit d277442df5.

Make sucks.
2018-08-23 00:23:35 +02:00
Eelco Dolstra
1b01954a3d
Merge pull request #2368 from dezgeg/drop-dead-code
Drop dead code
2018-08-22 21:32:49 +02:00
Eelco Dolstra
ebe3d2d370 Improve 'coroutine has finished' error message 2018-08-21 15:22:04 +02:00
Eelco Dolstra
6317c65937 Fix warnings in compression.cc 2018-08-21 15:20:23 +02:00
Eelco Dolstra
17a92dfb7d Fix another 'coroutine has finished' during decompression
https://hydra.nixos.org/build/79867739
2018-08-21 15:19:20 +02:00
Tuomas Tynkkynen
2894197de7 Drop all references to NIX_INDENT_MAKE
Dead code since 6669a3b477
2018-08-20 02:01:59 +03:00
Eelco Dolstra
cc7b4386b1
nix run: Restore CPU affinity
Fixes #2359.
2018-08-19 12:05:08 +02:00
Eelco Dolstra
9b1bdf2db8
FIx floating point evaluation
Fixes #2361.
2018-08-19 11:59:49 +02:00
Graham Christensen
02098d2073 fetchGit: use a better caching scheme
The current usage technically works by putting multiple different
repos in to the same git directory. However, it is very slow as
Git tries very hard to find common commits between the two
repositories. If the two repositories are large (like Nixpkgs and
another long-running project,) it is maddeningly slow.

This change busts the cache for existing deployments, but users
will be promptly repaid in per-repository performance.
2018-08-17 11:27:34 -04:00
Eelco Dolstra
d277442df5
Fix parser/lexer generation with parallel make
Fun fact: rules with multiple targets don't work properly with 'make
-j'. For example, a rule like

  a b: c
    touch a b

is equivalent to

  a: c
    touch a b

  b: c
    touch a b

so with 'make -j', the 'touch' command will be run twice. See
e.g. https://stackoverflow.com/questions/2973445/gnu-makefile-rule-generating-a-few-targets-from-a-single-source-file.
2018-08-17 12:59:23 +02:00
Symphorien Gibol
5b19a6663b ignore when listxattr fails with ENODATA
This happens on CIFS and means the remote filesystem has no extended
attributes.
2018-08-13 21:11:36 +02:00
Linus Heckemann
d7402c9cd5 dirOf: allow use on non-absolute paths 2018-08-13 11:27:35 +02:00
Eelco Dolstra
f72c907ad8
Slightly questionable workaround for #2342 2018-08-10 11:34:41 +02:00
Eelco Dolstra
ada4e90267
S3BinaryCacheStore: Share TransferManager
TransferManager allocates a lot of memory (50 MiB by default), and it
might leak but I'm not sure about that. In any case it was causing
OOMs in hydra-queue-runner. So allocate only one TransferManager per
S3BinaryCacheStore.

Hopefully fixes https://github.com/NixOS/hydra/issues/586.
2018-08-09 20:48:45 +02:00
Eelco Dolstra
c87f4b9324
nix run: Respect propagated-user-env-packages
Also, add $path/bin to $PATH even if it doesn't exist. This makes
'man' work properly (since it looks for ../share/man relative to $PATH
entries).
2018-08-09 13:01:03 +02:00
Eelco Dolstra
a0b971dd9c
S3BinaryCacheStore: Don't use the transfer status callback
This callback is executed on a different thread, so exceptions thrown
from the callback are not caught:

  Aug 08 16:25:48 chef hydra-queue-runner[11967]: terminate called after throwing an instance of 'nix::Error'
  Aug 08 16:25:48 chef hydra-queue-runner[11967]:   what():  AWS error: failed to upload 's3://nix-cache/19dbddlfb0vp68g68y19p9fswrgl0bg7.ls'

Therefore, just check the transfer status after it completes. Also
include the S3 error message in the exception.
2018-08-08 21:39:11 +02:00
Bob van der Linden
58a85fa462 mention nix-store --query --roots when a path cannot be deleted 2018-08-08 21:21:21 +02:00
Ivan Kozik
ec49ea28dc repl: don't add trailing spaces to history lines 2018-08-07 03:59:34 +00:00
Eelco Dolstra
103ad1598c
Merge pull request #2334 from dtzWill/fix/revert-progress-bar-refresh-for-now
Revert "progress-bar: re-draw last update if nothing new for 1sec."
2018-08-06 19:01:08 +02:00
Eelco Dolstra
ed6c646f44
Doh 2018-08-06 17:27:08 +02:00
Eelco Dolstra
7de3e00ad9
Fix setting Content-Encoding in S3 uploads
Fixes https://github.com/NixOS/nix/issues/2333 and https://github.com/NixOS/nixpkgs/issues/44337.
2018-08-06 17:20:23 +02:00
Eelco Dolstra
d3761f5f8b
Fix Brotli decompression in 'nix log'
This didn't work anymore since decompression was only done in the
non-coroutine case.

Decompressors are now sinks, just like compressors.

Also fixed a bug in bzip2 API handling (we have to handle BZ_RUN_OK
rather than BZ_OK), which we didn't notice because there was a missing
'throw':

  if (ret != BZ_OK)
      CompressionError("error while compressing bzip2 file");
2018-08-06 15:40:29 +02:00
Eelco Dolstra
fa4def3d46
Require libbrotli 2018-08-06 14:06:54 +02:00
Eelco Dolstra
4361a4331f
Fix reporting of HTTP body size when a result callback is used 2018-08-06 11:31:14 +02:00
Eelco Dolstra
2825e05d21
Make adding paths via nix-store --serve run in constant memory
It adds a new operation, cmdAddToStoreNar, that does the same thing as
the corresponding nix-daemon operation, i.e. call addToStore(). This
replaces cmdImportPaths, which has the major issue that it sends the
NAR first and the store path second, thus requiring us to store the
incoming NAR either in memory or on disk until we decide what to do
with it.

For example, this reduces the memory usage of

  $ nix copy --to 'ssh://localhost?remote-store=/tmp/nix' /nix/store/95cwv4q54dc6giaqv6q6p4r02ia2km35-blender-2.79

from 267 MiB to 12 MiB.

Probably fixes #1988.
2018-08-03 21:21:16 +02:00
Eelco Dolstra
eeebe4cdc5
cmdDumpStorePath: Support chroot stores 2018-08-03 21:20:38 +02:00
Eelco Dolstra
848a9375c3
Support escaping in store URIs 2018-08-03 21:20:38 +02:00
Eelco Dolstra
4e7d5f660c
SSHMaster: Bypass SSH when connecting to localhost
This is primarily useful for testing since it removes the need to have
SSH working.
2018-08-03 21:20:38 +02:00
Eelco Dolstra
e268bbc054
LegacySSHStore: Add remote-store option
This is primarily useful for testing, e.g.

  $ nix copy --to 'ssh://localhost?remote-store=/tmp/nix' ...
2018-08-03 21:20:38 +02:00
Eelco Dolstra
bc65e02d96
Merge pull request #2326 from aszlig/fix-symlink-leak
Fix symlink leak in restricted eval mode
2018-08-03 17:01:34 +02:00
Eelco Dolstra
122e1a61f8
Merge pull request #2323 from samueldr/feature/selective-impurity
Allows selectively adding environment variables to pure shells.
2018-08-03 16:58:18 +02:00
aszlig
43e28a1b75
Fix symlink leak in restricted eval mode
In EvalState::checkSourcePath, the path is checked against the list of
allowed paths first and later it's checked again *after* resolving
symlinks.

The resolving of the symlinks is done via canonPath, which also strips
out "../" and "./". However after the canonicalisation the error message
pointing out that the path is not allowed prints the symlink target in
the error message.

Even if we'd suppress the message, symlink targets could still be leaked
if the symlink target doesn't exist (in this case the error is thrown in
canonPath).

So instead, we now do canonPath() without symlink resolving first before
even checking against the list of allowed paths and then later do the
symlink resolving and checking the allowed paths again.

The first call to canonPath() should get rid of all the "../" and "./",
so in theory the only way to leak a symlink if the attacker is able to
put a symlink in one of the paths allowed by restricted evaluation mode.

For the latter I don't think this is part of the threat model, because
if the attacker can write to that path, the attack vector is even
larger.

Signed-off-by: aszlig <aszlig@nix.build>
2018-08-03 06:46:43 +02:00
Samuel Dionne-Riel
438e02529d Allows selectively adding environment variables to pure shells.
Includes documentation and test.
2018-08-02 08:21:30 -04:00
Graham Christensen
49a53c1d3f s3 binary cache: support specifying an endpoint
Works for uploading and not downloading.
2018-08-02 08:19:50 -04:00
Samuel Dionne-Riel
fb6907e3d6 Removes unused variable from nix-build/nix-shell
This particular `shell` variable wasn't used, since a new one was
declared in the only side of the `if` branch that used a `shell`
variable.

It could realistically confuse developers thinking it could use `$SHELL`
under some situations.
2018-08-01 18:30:03 -04:00
Eelco Dolstra
45bcf5416a
Merge branch 'prim_mapAttr-fix' of https://github.com/volth/nix 2018-07-31 20:05:07 +02:00
Eelco Dolstra
4b4adbc93d
Fix race downloading into a sink
This fixes 'error 10 while decompressing xz file'.

https://hydra.nixos.org/build/78308551
2018-07-31 17:26:04 +02:00
Eelco Dolstra
d9e8ab0ff2
Add a check for broken Boost versions
In some Boost versions, coroutines don't propagate exceptions
properly, causing Nix to fail with the exception 'coroutine has
finished'.
2018-07-31 16:35:05 +02:00
Will Dietz
0d72f4e0ca search: include version 2018-07-27 10:45:35 -05:00
Linus Heckemann
9ac1a79882 copyPathsToStore: honour keep-going 2018-07-24 17:05:06 +02:00
Eelco Dolstra
a7fb7d3cde
Merge pull request #2303 from volth/patch-4
parser.y: fix assoc of -> and < > <= >=
2018-07-23 11:38:15 +02:00
volth
deaa6e9a34
parser.y: right-associativity of -> 2018-07-23 07:28:48 +00:00
volth
85fe4a819c
parser.y: fix assoc of -> and < > <= >=
The parser allowed senseless `a > b > c` but disallowed `a -> b -> c` which seems valid
It might be a typo
2018-07-21 15:24:51 +00:00
volth
e2b114cfe1
prim_foldlStrict: call forceValue() before value is copied
forceValue() were called after a value is copied effectively forcing only one of the copies keeping another copy not evaluated.
This resulted in its evaluation of the same lazy value more than once (the number of hits is not big though)
2018-07-21 06:44:42 +00:00
Will Dietz
4495560d6d Revert "progress-bar: re-draw last update if nothing new for 1sec."
Not ready for this yet, causes the prompt to disappear in nix repl
and more generally can overwrite non-progress-bar messages.

This reverts commit 44de71a396.
2018-07-19 10:29:31 -05:00
Eelco Dolstra
1b34b69b45
nix-prefetch-url: Download file in constant memory
Before:

  $ command time nix-prefetch-url https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.6.tar.xz
  1.19user 1.02system 0:41.96elapsed 5%CPU (0avgtext+0avgdata 182720maxresident)k

After:

  1.38user 1.05system 0:39.73elapsed 6%CPU (0avgtext+0avgdata 16204maxresident)k

Note however that addToStore() can still take a lot of memory
(e.g. RemoteStore::addToStore() is constant space, but
LocalStore::addToStore() isn't; that's fixed by
c94b4fc7ee
though).

Fixes #1400.
2018-07-12 18:48:41 +02:00
Eelco Dolstra
c2de2ff385
Fix build on 32-bit systems and macOS
Apparently, on macOS, 'long' != 'int64_t'.

https://hydra.nixos.org/build/77100756
2018-07-11 21:12:18 +02:00
Eelco Dolstra
efd04888ca
Shup up a warning 2018-07-11 21:05:09 +02:00
Eelco Dolstra
875cd9da2b
Remove unused function printStats2()
Closes #2282.
2018-07-11 20:29:18 +02:00
volth
c03d73c1cf
prim_map, prim_genList: no need to force function argument which will be stored in mkApp 2018-07-07 23:30:17 +00:00
volth
627e28ba33 prim_mapAttrs: `f' must be evaluated lazily to avoid infinite recursion 2018-07-06 21:52:54 +00:00
Eelco Dolstra
aa64e95bc8
Merge pull request #2250 from dtzWill/feature/repl-progress-bar-builds
repl: use `nix build` for building instead of `nix-store -r`
2018-07-06 21:36:37 +02:00
volth
1515c65616 prim_concatMap: no need to force value 2018-07-05 15:33:33 +00:00
volth
e6bf1a79d7 prim_mapAttrs: must be lazy to avoid infinite recursion 2018-07-05 15:33:12 +00:00
volth
841747b0e6
prim_concatMap: allocate intermediate list on stack 2018-07-05 12:37:37 +00:00
volth
ee218f99ca
primops.cc: fix comment 2018-07-05 11:58:15 +00:00
volth
403a76a18f lib.concatMap and lib.mapAttrs to be builtins 2018-07-05 02:54:09 +00:00
Eelco Dolstra
e388739098
nix-daemon: Respect --store
For example, this allows you to do run nix-daemon as a non-privileged
user:

  eelco$ NIX_STATE_DIR=~/my-nix/nix/var nix-daemon --store ~/my-nix/

The NIX_STATE_DIR is still needed because settings.nixDaemonSocketFile
is not derived from settings.storeUri (and we can't derive it from the
store's state directory because we don't want to open the store in the
parent process).
2018-07-03 19:52:32 +02:00
Eelco Dolstra
ea3c9dab5f
Include cpptoml for build simplicity 2018-07-03 18:39:36 +02:00
Eelco Dolstra
3b1f54cf06
Add a fromTOML primop
This is primarily useful for processing Cargo.lock files.
2018-07-03 18:39:36 +02:00