Commit graph

8135 commits

Author SHA1 Message Date
Eelco Dolstra
e136d57f26
Implement BinaryCacheStore::queryPathFromHashPart() 2022-10-18 17:48:09 +02:00
Eelco Dolstra
61f89e954a
Add command 'nix store path-from-hash-part'
This exposes the Store::queryPathFromHashPart() interface in the CLI.
2022-10-18 16:51:12 +02:00
Nathan Henrie
af9c9504ca
Fix typo -- dashes not underscores 2022-10-17 14:15:32 -06:00
Andrew Brooks
a259084c50 Fix #7146
When fetching a non-local git repo by ref (and no rev), don't consider unrelated
cached revs for the same repository.
2022-10-14 18:04:47 -05:00
Eelco Dolstra
3093bd3a85
Merge pull request #7168 from NixOS/rosetta-test
Improve Rosetta detection
2022-10-14 17:35:57 +02:00
Robert Hensing
ab4eb39386 libmain: Make the entire stack overflow handler pluggable 2022-10-14 12:53:46 +02:00
Robert Hensing
0d75675787 libmain: Add extraStackOverflowHandler 2022-10-14 12:53:07 +02:00
Eelco Dolstra
285277a61a Remove useless debug statements
We haven't parsed the '-v' command line flags yet when this code executes,
so we can't actually get debug output here.
2022-10-14 00:35:33 -07:00
Eelco Dolstra
ddd5503950 Use /usr/bin/true 2022-10-14 00:34:31 -07:00
Eelco Dolstra
0359d6d123 Fix error display if execve() in the builder fails
After we've send "\2\n" to the parent, we can't send a serialized
exception anymore. It will show up garbled like

  $ nix-build --store /tmp/nix --expr 'derivation { name = "foo"; system = "x86_64-linux"; builder = "/foo/bar"; }'
  this derivation will be built:
    /nix/store/xmdip0z5x1zqpp6gnxld3vqng7zbpapp-foo.drv
  building '/nix/store/xmdip0z5x1zqpp6gnxld3vqng7zbpapp-foo.drv'...

  ErrorErrorEexecuting '/foo/bar': No such file or directory
  error: builder for '/nix/store/xmdip0z5x1zqpp6gnxld3vqng7zbpapp-foo.drv' failed with exit code 1
2022-10-13 21:35:16 +02:00
Eelco Dolstra
96eb5ef156 Improve Rosetta detection
Turns out that one of those *.plist files can exist even if Rosetta is
not installed. So let's just try to run an x86_64-darwin binary
directly.
2022-10-13 11:46:16 -07:00
Eelco Dolstra
59a304a9a8 Fix clang warnings 2022-10-13 11:25:49 -07:00
Eelco Dolstra
27ed3d0458
Merge pull request #7140 from crawford/chroot-store
Make warning about chroot store location more accurate
2022-10-13 16:35:31 +02:00
Steam Deck User
a86916eb72 Make warning about chroot store location more accurate
While trying to use an alternate directory for my Nix installation, I
noticed that nix's output didn't reflect the updated state
directory. This patch corrects that and now prints the warning before
attempting to create the directory (if the directory creation fails,
it wouldn't have been obvious why nix was attempting to create the
directory in the first place).

With this patch, I now get the following warning:

    warning: '/home/deck/.var/app/org.nixos.nix/var/nix' does not
    exist, so Nix will use '/home/deck/.local/share/nix/root' as a
    chroot store
2022-10-12 12:12:12 -07:00
Eelco Dolstra
eba610956b Move some options into a misc category
This unclutters the per-command options a bit by moving out some
global options.
2022-10-12 15:09:17 +02:00
Eelco Dolstra
ae2d330455 Merge branch 'print-common-flags' of github.com:Hoverbear/nix 2022-10-12 13:43:11 +02:00
Eelco Dolstra
f3193edd87
Merge pull request #7149 from amjoseph-nixpkgs/pr/intersectAttrs/values
parseDrvName: remove doc/impl discrepancy, add test covering the gap
2022-10-12 10:00:39 +02:00
Adam Joseph
7ef71cd21f src/libexpr/primops.cc: parseDrvName: make documentation follow implementation
The documentation for `parseDrvName` does not agree with the implementation when
the derivation name contains a dash which is followed by something that is
neither a letter nor a digit.  This commit corrects the documentation to agree
with the implementation.
2022-10-08 17:21:34 -07:00
Ana Hobden
069409d167 Print common flags in --help 2022-10-07 09:07:22 -07:00
Valentin Gagarin
927234cfb2
Merge pull request #6870 from amjoseph-nixpkgs/pr/doc/explain-local-remote-binary-substituter 2022-10-05 09:01:42 +02:00
Jörg Thalheim
d9abce4ad4 libfetchers: avoid api.github.com ratelimit if no github token is set
If we don't have any github token, we won't be able to fetch private
repos, but we are also more likely to run into API limits since
we don't have a token. To mitigate this only ever use the github api
if we actually have a token.
2022-10-01 10:38:09 +02:00
Eelco Dolstra
89ca75c9f9
Merge pull request #7080 from squalus/nar-close-file
archive: check close errors when extracting nars
2022-09-30 12:58:28 +02:00
Linus Heckemann
19f3ecd830 nix-build: remove unused --add-root arg
Fixes #1982
2022-09-27 00:07:42 +02:00
Eelco Dolstra
6806f20157
Merge pull request #7089 from NixOS/mention-accept-flake-config-in-warning
Mention `--accept-flake-config` in the related warning
2022-09-26 11:13:02 +02:00
Adam Joseph
dc205c75a2 src/libexpr/primops.cc: correct definition for intersectAttrs
The current definition of `intersectAttrs` is incorrect:

> Return a set consisting of the attributes in the set e2 that also exist in the
> set e1.

Recall that (Nix manual, section 5.1):

> An attribute set is a collection of name-value-pairs (called attributes)

According to the existing description of `intersectAttrs`, the following should
evaluate to the empty set, since no key-value *pair* (i.e. attribute) exists in
both sets:

```
builtins.intersectAttrs { x=3; } {x="foo";}
```

And yet:

```
nix-repl> builtins.intersectAttrs { x=3; } {x="foo";}
{ x = "foo"; }
```

Clearly the intent here was for the *names* of the resulting attribute set to be
the intersection of the *names* of the two arguments, and for the values of the
resulting attribute set to be the values from the second argument.

This commit corrects the definition, making it match the implementation and intent.
2022-09-24 23:51:09 -07:00
Théophane Hufschmitt
371013c08d
Merge pull request #7010 from edolstra/ignore-shell
nix develop: Ignore stdenv's $SHELL
2022-09-24 12:55:53 +02:00
Théophane Hufschmitt
db29ddd113
Merge pull request #7078 from obsidiansystems/trustworthy-signature
"valid signature" -> "signature by a trusted key"
2022-09-24 12:52:35 +02:00
Théophane Hufschmitt
74276cb354
Merge pull request #7079 from matthewbauer/allow-untrusted-settings
Allow pass max-silent-time and build-poll-interval to daemon untrusted
2022-09-24 12:47:49 +02:00
Théophane Hufschmitt
1a5d094be7
Mention --accept-flake-config in the related warning
Make sure that people who run Nix in non-interactive mode (and so don't have the possibility to interactively accept the individual flake configuration settings) are aware of this flag.

Fix #7086
2022-09-24 12:11:26 +02:00
Eelco Dolstra
c13007f012
Merge pull request #7059 from NixOS/remove-useless-ca-file-message
Remove a useless debug message in filetransfer.cc
2022-09-24 10:39:23 +02:00
John Ericson
60e23c8bae
Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Rune K. Svendsen <runesvend@gmail.com>
2022-09-23 13:57:57 -04:00
squalus
223f8dace0 archive: check close errors when extracting nars 2022-09-22 12:50:32 -07:00
Matthew Bauer
6e049ae607 Allow pass max-silent-time and build-poll-interval to daemon untrusted
These settings seem harmless, they control the same polling
functionality that timeout does, but with different behavior. Should
be safe for untrusted users to pass in.
2022-09-22 13:59:16 -05:00
John Ericson
a2a8cb10ac Dodge "trusted" vs "trustworthy" by being explicit
Hopefully this is best!
2022-09-22 14:37:52 -04:00
John Ericson
752f967c0f "valid signature" -> "trustworthy signature"
I just had a colleague get confused by the previous phrase for good
reason. "valid" sounds like an *objective* criterion, e.g. and *invalid
signature* would be one that would be trusted by no one, e.g. because it
misformatted or something.

What is actually going is that there might be a signature which is
perfectly valid to *someone else*, but not to the user, because they
don't trust the corresponding public key. This is a *subjective*
criterion, because it depends on the arbitrary and personal choice of
which public keys to trust.

I therefore think "trustworthy" is a better adjective to use. Whether
something is worthy of trust is clearly subjective, and then "trust"
within that word nicely evokes `trusted-public-keys` and friends.
2022-09-22 10:49:31 -04:00
Théophane Hufschmitt
f704c2720f
Merge pull request #7070 from alyssais/repl-history
nix repl: warn if creating dir for history fails
2022-09-21 13:50:49 +02:00
Alyssa Ross
d234d01f01
nix repl: warn if creating dir for history fails
The history is not critical to the functionality of nix repl, so it's
enough to warn here, rather than refuse to start if the directory Nix
thinks the history should live in can't be created.
2022-09-20 11:01:40 +00:00
squalus
1b595026e1 Improve durability of schema version file writes
- call close explicitly in writeFile to prevent the close exception
  from being ignored
- fsync after writing schema file to flush data to disk
- fsync schema file parent to flush metadata to disk

https://github.com/NixOS/nix/issues/7064
2022-09-19 20:13:30 -07:00
Théophane Hufschmitt
0f977bf91e
Remove a useless debug message in filetransfer.cc
Remove the `verify TLS: Nix CA file = 'blah'` message that Nix used to print when fetching anything as it's both useless (`libcurl` prints the same info in its logs) and misleading (gives the impression that a new TLS connection is being established which might not be the case because of multiplexing. See #7011 )
2022-09-19 08:42:43 +02:00
Adam Joseph
fb985f855c fetchurl.nix: change other use of __impure 2022-09-16 01:54:24 -07:00
Adam Joseph
673fd21b7c
Update src/libexpr/fetchurl.nix
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-09-16 08:51:14 +00:00
Adam Joseph
b3550d9179 libexpr/fetchurl.nix: allow __impure fetch
This commit adds an optional `__impure` parameter to fetchurl.nix, which allows
the caller to use `libfetcher`'s fetcher in an impure derivation.  This allows
nixpkgs' patch-normalizing fetcher (fetchpatch) to be rewritten to use nix's
internal fetchurl, thereby eliminating the awkward "you can't use fetchpatch
here" banners scattered all over the place.

See also: https://github.com/NixOS/nixpkgs/pull/188587
2022-09-16 00:48:01 -07:00
Eelco Dolstra
d365cced4f
Trim option descriptions
This removes unintended blank lines in Markdown when the description
is a multiline string literal.
2022-09-13 16:58:32 +02:00
Eelco Dolstra
c6ff33ff5c
RunPager: Stop the progress bar
In particular, the progress bar was interfering with 'less' rendering
in '--help' (e.g. run 'nix --help' and hit '/' to search).
2022-09-13 15:29:13 +02:00
Théophane Hufschmitt
0f64bf445a
Merge pull request #6994 from agbrooks/master
Prevent tempdir from being GC-ed before addToStoreFromDump completes
2022-09-13 09:23:16 +02:00
Andrew Brooks
565d888e0f Address PR feedback on #6694 2022-09-12 11:33:23 -05:00
Valentin Gagarin
dceb4f9dbb
Merge pull request #6825 from rapenne-s/nix_shell_example
nix shell: document how to invoke multiple commands from the command …
2022-09-09 13:35:23 +02:00
Théophane Hufschmitt
c8e8eea95e
Merge pull request #7008 from jonringer/add-command-docs
Docs: Add nix develop --command entry
2022-09-08 13:34:40 +02:00
Andrew Brooks
84fe75a12a Keep created temp dirs inside store, but protect from GC
Implements the approach suggested by feedback on PR #6994, where
tempdir paths are created in the store (now with an exclusive lock).

As part of this work, the currently-broken and unused
`createTempDirInStore` function is updated to create an exclusive lock
on the temp directory in the store.

The GC now makes a non-blocking attempt to lock any store directories
that "look like" the temp directories created by this function, and if
it can't acquire one, ignores the directory.
2022-09-06 17:48:00 -05:00
Eelco Dolstra
ece12a97d9 lockfile -> lock file for consistency 2022-09-06 19:20:31 +02:00
Eelco Dolstra
27be54ca53 nix develop: Ignore stdenv's $SHELL
Stdenv sets this to a bash that doesn't have readline/completion
support, so running 'nix (develop|shell)' inside a 'nix develop' gives
you a crippled shell. So let's just ignore the derivation's $SHELL.

This could break interactive use of build phases that use $SHELL, but
they appear to be fairly rare.
2022-09-06 18:27:39 +02:00
Eelco Dolstra
9600109879
Merge pull request #6995 from matthewbauer/disable-sa-restart-on-macos
Disable SA_RESTART for some signals on macOS
2022-09-06 17:36:31 +02:00
Jonathan Ringer
6ce2e96c88
Docs: Add nix develop --command entry
Add example of nix develop being used to execuate a series of script
commands. This is common when doing things like CI/CD, and should be
represented in the official documentation.

Also useful for people looking for the 'nix develop' equivalent of
'nix-shell --run'.

Related:
 - https://github.com/NixOS/nix/issues/6908
 - https://github.com/NixOS/nix/issues/6908#issuecomment-1229266853
2022-09-06 08:18:29 -07:00
Matthew Bauer
77d3d3d18d Remove some signals 2022-09-06 09:47:53 -05:00
Matthew Bauer
a47b5476e1 Add more signals 2022-09-03 16:06:33 -05:00
Matthew Bauer
102434e4cb Disable SA_RESTART for some signals on macOS
Disables the SA_RESTART behavior on macOS which causes:

> Restarting of pending calls is requested by setting the SA_RESTART bit
> in sa_flags. The affected system calls include read(2), write(2),
> sendto(2), recvfrom(2), sendmsg(2) and recvmsg(2) on a communications
> channel or a slow device (such as a terminal, but not a regular file)
> and during a wait(2) or ioctl(2).

From: https://man.openbsd.org/sigaction#SA_RESTART

This being set on macOS caused a bug where read() calls to the daemon
socket were blocking after a SIGINT was received. As a result,
checkInterrupt was never reached even though the signal was received
by the signal handler thread.

On Linux, SA_RESTART is disabled by default. This probably effects
other BSDs but I don’t have the ability to test it there right now.
2022-09-03 00:27:16 -05:00
Andrew Brooks
1f041ac54f Prevent tempdir from being GC-ed before addToStoreFromDump has renamed it
This fixes issue 6823 by placing the tempdir used in LocalStore::addToStoreFromDump
outside the Nix store, where automatic GC is no longer a concern.
2022-09-02 18:32:35 -05:00
Matthew Bauer
4894e567fb Don’t readDerivation if impure derivations feature is disabled
readDerivation is pretty slow, and while it may not be significant for
some use cases, on things like ghc-nix where we have thousands of
derivations is really slows things down.

So, this just doesn’t do the impure derivation check if the impure
derivation experimental feature is disabled. Perhaps we could cache
the result of isPure() and keep the check, but this is a quick fix to
for the slowdown introduced with impure derivations features in 2.8.0.
2022-09-02 11:46:34 -05:00
Adam Joseph
1ab913467e linkify mention of other options 2022-09-01 18:03:35 -07:00
Adam Joseph
59dc8346ca move substituter signature-checking conditions to configuration file documentation 2022-09-01 17:51:56 -07:00
Eelco Dolstra
a8b3d777fb Revert "Merge pull request #6621 from Kha/nested-follows"
This reverts commit c530cda345, reversing
changes made to 4adcdff5c1.
2022-09-01 15:26:19 +02:00
Théophane Hufschmitt
c530cda345
Merge pull request #6621 from Kha/nested-follows
Fix nested flake input overrides
2022-09-01 12:04:00 +02:00
Eelco Dolstra
1f2b12b30e
Merge pull request #6968 from fricklerhandwerk/man-no-weblinks
do not render relative links in help pages
2022-08-29 08:46:35 +02:00
Sebastian Ullrich
6f65c11780 Fix overlapping flake follows 2022-08-28 11:50:25 +02:00
Sebastian Ullrich
2b9d381301 Fix nested flake input overrides 2022-08-28 11:47:25 +02:00
Valentin Gagarin
e7dcacbd7c do not render relative links in help pages
this simplifies the setup a lot, and avoids weird looking `./file.md`
links showing up.

it also does not show regular URLs any more. currently the command
reference only has few of them, and not showing them in the offline
documentation is hopefully not a big deal.

instead of building more special-case solutions, clumsily preprocessing
the input, or issuing verbal rules on dealing with URLs, should better
be solved sustainably by not rendering relative links in `lowdown`:

https://github.com/kristapsdz/lowdown/issues/105
2022-08-26 17:10:41 +02:00
Eelco Dolstra
7918adbb62
Merge pull request #6954 from winterqt/darwin-sandbox-trustd
fix(libstore): allow access to trustd on macOS
2022-08-26 11:45:00 +02:00
Eelco Dolstra
bb411e4ae1 Fix progress bar flicker with -L
This was caused by -L calling setLogFormat() again, which caused the
creation of a new progress bar without destroying the old one. So we
had two progress bars clobbering each other.

We should change 'logger' to be a smart pointer, but I'll do that in a
future PR.

Fixes #6931.
2022-08-24 22:36:40 +02:00
Eelco Dolstra
a17ce0a8a9 Fix evaluation cache
98e361ad4c introduced a regression where
previously stored attributes were replaced by placeholders. As a
result, a command like 'nix build nixpkgs#hello' had to be executed at
least twice to get caching.

This code does not seem necessary for suggestions to work.
2022-08-24 21:19:43 +02:00
Winter
8e5659423e fix(libstore): allow access to trustd on macOS 2022-08-24 13:09:44 -04:00
Eelco Dolstra
04e74f7c8b
Merge pull request #6612 from NixOS/parallel-nix-copy
Make nix copy parallel again
2022-08-24 15:31:42 +02:00
Eelco Dolstra
56d97d4b4d Remove redundant Finally 2022-08-24 14:49:58 +02:00
Eelco Dolstra
8d906b1f3b Fix macOS build 2022-08-24 14:11:03 +02:00
Rickard Nilsson
c2d7456926 Fix a misplaced parenthese in serve protocol check
This issue made it impossible for clients using a serve protocol of
version <= 2.3 to use the `cmdBuildDerivation` command of servers using
a protocol of version >= 2.6. The faulty version check makes the server
send back build outputs that the client is not expecting.
2022-08-24 01:54:43 +02:00
Eelco Dolstra
4a0c4ca186
Merge pull request #6944 from edolstra/bash-vars
nix develop: Ignore some more bash special variables
2022-08-23 20:15:18 +02:00
Eelco Dolstra
0e54fab0dd
Merge pull request #6909 from ncfavier/json-paths
JSON: print paths as strings without copying them to the store
2022-08-23 15:19:33 +02:00
Eelco Dolstra
db026103b1 nix develop: Ignore some more bash special variables
Fixes #6940.
2022-08-23 14:57:08 +02:00
Eelco Dolstra
db71f5646f
Merge pull request #6887 from winterqt/specify-bashinteractive-outputs
nix-shell: specify which outputs from bashInteractive to build
2022-08-23 14:44:23 +02:00
Naïm Favier
ff0b5a778c
Revert to copyStore = true for nix-instantiate and nix-env 2022-08-23 14:40:27 +02:00
Eelco Dolstra
f0358ed465 Fix a hang in nix-copy-ssh.sh
This hang for some reason didn't trigger in the Nix build, but did
running 'make installcheck' interactively. What happened:

* Store::addMultipleToStore() calls a SinkToSource object to copy a
  path, which in turn calls LegacySSHStore::narFromPath(), which
  acquires a connection.

* The SinkToSource object is not destroyed after the last bytes has
  been read, so the coroutine's stack is still alive and its
  destructors are not run. So the connection is not released.

* Then when the next path is copied, because max-connections = 1,
  LegacySSHStore::narFromPath() hangs forever waiting for a connection
  to be released.

The fix is to make sure that the source object is destroyed when we're
done with it.
2022-08-23 14:19:53 +02:00
Eelco Dolstra
f865048332 Indentation 2022-08-22 15:32:53 +02:00
Eelco Dolstra
4c96761c2b Merge remote-tracking branch 'origin/master' into parallel-nix-copy 2022-08-22 15:29:10 +02:00
Eelco Dolstra
7c3ab57515
Merge pull request #6937 from edolstra/fix-repl
nix repl: Stop the progress bar
2022-08-22 15:10:01 +02:00
Naïm Favier
4c2ff4a0f4
JSON: add missing newlines
after `nix eval --json` and `nix-instantiate --eval --json`.
2022-08-22 15:07:52 +02:00
Naïm Favier
062e4fcdde
JSON: print paths as strings without copying them to the store
Makes `printValueAsJSON` not copy paths to the store for `nix eval
--json`, `nix-instantiate --eval --json` and `nix-env --json`.

Fixes https://github.com/NixOS/nix/issues/5612
2022-08-22 15:01:35 +02:00
Eelco Dolstra
8d84634e26
Merge pull request #6926 from rapenne-s/download_limit
add a nix.conf option to set a download speed limit
2022-08-22 14:57:19 +02:00
Eelco Dolstra
590660c1c6
Merge pull request #6932 from jack1142/patch-1
Fix default profile path for root in nix profile documentation
2022-08-22 14:55:04 +02:00
Eelco Dolstra
0d2163c6dc nix repl: Stop the progress bar
The repl was broken since c3769c6846. In
general, the progress bar is incompatible with the repl.
2022-08-22 14:27:36 +02:00
Eelco Dolstra
9ddef9f0be
Merge pull request #6923 from ncfavier/nix-env-incomplete-json
Avoid producing incomplete JSON
2022-08-22 14:15:31 +02:00
Eelco Dolstra
c21b1a7e67
Spelling 2022-08-22 14:14:14 +02:00
Solène Rapenne
caad87e6db
Better documentation wording
Co-authored-by: Anderson Torres <torres.anderson.85@protonmail.com>
2022-08-20 18:21:36 +02:00
Jakub Kuczys
7d800909e9
Fix default profile path for root in nix profile documentation 2022-08-20 03:48:42 +02:00
Solene Rapenne
0d2bf7acf9 add a nix.conf option to set a download speed limit 2022-08-19 12:40:22 +02:00
pennae
7d934f7880 don't read outputs into memory for output rewriting
RewritingSink can handle being fed input where a reference crosses a
chunk boundary. we don't need to load the whole source into memory, and
in fact *not* loading the whole source lets nix build FODs that do not
fit into memory (eg fetchurl'ing data files larger than system memory).
2022-08-19 11:26:26 +02:00
Naïm Favier
7535ee345d
nix-env: don't output incomplete JSON 2022-08-19 01:00:10 +02:00
Naïm Favier
8188b1d0ab
json: write null on abnormal placeholder destruction
Avoids leaving dangling attributes like

    {
        "foo":
    }

in case of exceptions.
2022-08-19 01:00:10 +02:00
Eelco Dolstra
c80a74b7d5 Don't pass --force to 'git add'
Fixes #5810.
2022-08-17 16:59:02 +02:00
Eelco Dolstra
53e7b7e8ac Remove warnLargeDump()
This message was unhelpful (#1184) and probably misleading since
memory is O(1) in most cases now.
2022-08-17 11:32:01 +02:00
Eelco Dolstra
c3769c6846 ProgressBar: Delay before showing a new activity
Some activities are numerous but usually very short (e.g. copying a
source file to the store) which would cause a lot of flickering. So
only show activities that have been running for at least 10 ms.
2022-08-17 11:31:44 +02:00
Eelco Dolstra
e62160579f nix flake metadata: Don't show "Inputs" if there are no inputs 2022-08-17 11:31:34 +02:00
Eelco Dolstra
3d4489b623 Show when we're evaluating a flake 2022-08-17 11:31:27 +02:00
David Arnold
b58785ab83
docfix: bundlers 2022-08-10 18:49:29 -05:00
Théophane Hufschmitt
8f3fdef1e0
Merge pull request #6850 from NinjaTrappeur/nin/build-check
Fix Nix build --check flag
2022-08-10 17:44:06 +02:00
Winter
5f37c5191a nix-shell: specify which outputs from bashInteractive to build 2022-08-09 22:54:22 -04:00
Théophane Hufschmitt
5192bb093a Merge remote-tracking branch 'upstream/master' into fix-mv-in-different-filesystems 2022-08-08 15:42:56 +02:00
Théophane Hufschmitt
f4a8426098 Remove the explicit c++fs linkage on darwin
Doesn't seem needed on a recent-enough clang anymore (and even seems to
break stuff)
2022-08-08 14:34:22 +02:00
Dave Nicponski
cb6794a0d9 Do not spam logs if the owned-homedir check results in a noop 2022-08-07 10:13:11 -04:00
Erik Arvstedt
4c8441be0a docs/flake-update: fix example 2022-08-04 09:45:30 +02:00
Erik Arvstedt
53833dfb40 libexpr/flake: remove FIXME
Line 593 checks that all overrides (i.e. all elements of
`lockFlags.inputOverrides`) are members of `overridesUsed`.
2022-08-04 09:45:29 +02:00
Erik Arvstedt
c9f446ede1 flakeref: fix comment 2022-08-04 09:45:28 +02:00
Eelco Dolstra
ccbd906c86 Fix NIX_COUNT_CALLS=1
Also, make the JSON writer support std::string_view.

Fixes #6857.
2022-08-03 17:46:51 +02:00
Théophane Hufschmitt
1ba5b3e001 Make moveFile more atomic
Rather than directly copying the source to its dest, copy it first to a
temporary location, and eventually move that temporary.
That way, the move is at least atomic from the point-of-view of the destination
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
90f9680733 Only use renameFile where needed
In most places the fallback to copying isn’t needed and can actually be
bad, so we’d rather not transparently fallback
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
d71d9e9fbf moveFile -> renameFile
`move` tends to have this `mv` connotation of “I will copy it for you if
needs be”
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
a4f0fd633c Link against c++fs on darwin
Required by the old clang version
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
c5db1821a9 Re-implement the recursive directory copy
The recursive copy from the stl doesn’t exactly do what we need because
1. It doesn’t delete things as we go
2. It doesn’t keep the mtime, which change the nars

So re-implement it ourselves. A bit dull, but that way we have what we want
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
6f89fb6008 rename: Fallback to a copy if the filesystems mismatch
In `nix::rename`, if the call to `rename` fails with `EXDEV` (failure
because the source and the destination are in a different filesystems)
switch to copying and removing the source.

To avoid having to re-implement the copy manually, I switched the
function to use the c++17 `filesystem` library (which has a `copy`
function that should do what we want).

Fix #6262
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
c2de0a232c Create a wrapper around stdlib’s rename
Directly takes some c++ strings, and gently throws an exception on error
(rather than having to inline this logic everywhere)
2022-08-03 10:27:25 +02:00
Théophane Hufschmitt
8119390abc Move some fs-related functions to their own file
Unclutter `util.cc` a bit
2022-08-03 10:27:25 +02:00
Félix Baylac-Jacqué
1467a98d4c
derivation-goal.cc: remove bmCheck custom return branch on buildDone
Once a derivation goal has been completed, we check whether or not
this goal was meant to be repeated to check its output.

An early return branch was preventing the worker to reach that repeat
code branch, hence breaking the --check command (#2619).

It seems like this early return branch is an artifact of a passed
refactoring. As far as I can tell, buildDone's main branch also
cleanup the tmp directory before returning.
2022-08-01 11:39:19 +02:00
Théophane Hufschmitt
2805439335
Merge pull request #6814 from amjoseph-nixpkgs/pr/sandbox-error-messages
local-derivation-goal.cc: improve error messages when sandboxing fails
2022-07-22 13:27:52 +02:00
Solène Rapenne
92bae33ca5
nix shell: example shouldn't use an absolute path for the shell
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-07-21 14:25:07 +02:00
Solène Rapenne
64404220f5 nix shell: document how to invoke multiple commands from the command line 2022-07-20 14:54:46 +02:00
Théophane Hufschmitt
7ed91d6c6a
Merge branch 'master' into parallel-nix-copy 2022-07-20 10:05:34 +02:00
Alex Wied
722de8ddcc libstore/globals.cc: Move cgroup detection to libutil 2022-07-19 16:25:53 -04:00
Alex Wied
1af5d798a4 libstore/globals.cc: Automatically set cores based on cgroup CPU limit
By default, Nix sets the "cores" setting to the number of CPUs which are
physically present on the machine. If cgroups are used to limit the CPU
and memory consumption of a large Nix build, the OOM killer may be
invoked.

For example, consider a GitLab CI pipeline which builds a large software
package. The GitLab runner spawns a container whose CPU is limited to 4
cores and whose memory is limited to 16 GiB. If the underlying machine
has 64 cores, Nix will invoke the build with -j64. In many cases, that
level of parallelism will invoke the OOM killer and the build will
completely fail.

This change sets the default value of "cores" to be
ceil(cpu_quota / cpu_period), with a fallback to
std:🧵:hardware_concurrency() if cgroups v2 is not detected.
2022-07-19 16:03:58 -04:00
Théophane Hufschmitt
56f6f3725f Don't ultimately trust the signed paths
Like the old implem did (and like you'd want it to be anyways)
2022-07-19 19:46:00 +02:00
Adam Joseph
36e1383b6b local-derivation-goal.cc: save global errno to the stack before performing tests which might clobber it 2022-07-19 03:53:20 -07:00
Adam Joseph
a9e75eca00 error.hh: add additional constructor with explicit errno argument 2022-07-19 03:49:33 -07:00
Adam Joseph
99fcc91f67 as requested by @thufschmitt https://github.com/NixOS/nix/pull/6814#discussion_r924275777 2022-07-19 03:33:12 -07:00
Adam Joseph
5f51539f88 change warn() to notice() 2022-07-19 03:30:52 -07:00
Adam Joseph
c8c6203c2c local-derivation-goal.cc: detect unprivileged_userns_clone failure mode
The workaround for "Some distros patch Linux" mentioned in
local-derivation-goal.cc will not help in the `--option
sandbox-fallback false` case.  To provide the user more helpful
guidance on how to get the sandbox working, let's check to see if the
`/proc` node created by the aforementioned patch is present and
configured in a way that will cause us problems.  If so, give the user
a suggestion for how to troubleshoot the problem.
2022-07-17 01:27:22 -07:00
Adam Joseph
6fc56318bf local-derivation-goal.cc: add comment re: CLONE_NEWUSER
local-derivation-goal.cc contains a comment stating that "Some distros
patch Linux to not allow unprivileged user namespaces."  Let's give a
pointer to a common version of this patch for those who want more
details about this failure mode.
2022-07-17 01:23:32 -07:00
Adam Joseph
8d35f387dc local-derivation-goal.cc: warn if failing and /proc/self/ns/user missing
This commit causes nix to `warn()` if sandbox setup has failed and
`/proc/self/ns/user` does not exist.  This is usually a sign that the
kernel was compiled without `CONFIG_USER_NS=y`, which is required for
sandboxing.
2022-07-16 19:37:27 -07:00
Adam Joseph
90830b1074 local-derivation-goal.cc: warn if failing due to max_user_namespaces==0
This commit uses `warn()` to notify the user if sandbox setup fails
with errno==EPERM and /proc/sys/user/max_user_namespaces is missing or
zero, since that is at least part of the reason why sandbox setup
failed.

Note that `echo -n 0 > /proc/sys/user/max_user_namespaces` or
equivalent at boot time has been the recommended mitigation for
several Linux LPE vulnerabilities over the past few years.  Many users
have applied this mitigation and then forgotten that they have done
so.
2022-07-16 19:30:53 -07:00
Adam Joseph
8ea3a911aa local-derivation-goal.cc: improve error messages when sandboxing fails
The failure modes for nix's sandboxing setup are pretty complicated.
When nix is unable to set up the sandbox, let's provide more detail
about what went wrong.  Specifically:

* Make sure the error message includes the word "sandbox" so the user
  knows that the failure was related to sandboxing.

* If `--option sandbox-fallback false` was provided, and removing it
  would have allowed further attempts to make progress, let the user
  know.
2022-07-16 14:56:24 -07:00
Alex Wied
b88fb50e21 fix(libstore): allow Nix to access all Rosetta 2 paths on MacOS
Fixes: #5884
2022-07-15 12:10:56 -07:00
John Ericson
12461e246b Leverage existing docs for new store-path^outputs syntax 2022-07-15 14:04:22 +00:00
John Ericson
0e4ec98ae8
Fix typo in docs
Thanks!

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-07-15 09:49:23 -04:00
John Ericson
279ecf7cde Remove computed-derivations experimental feature
We don't need it yet.
2022-07-15 13:42:17 +00:00
Eelco Dolstra
59764eb842
Merge pull request #6810 from jfly/jfly/do-not-assume-savedvars-exist
nix develop: do not assume that saved vars are set
2022-07-15 13:59:25 +02:00
Eelco Dolstra
3bcd7a5474 Disable auto-chroot if $NIX_STATE_DIR is set
Issue #6732.
2022-07-15 12:32:29 +02:00
Jeremy Fleischman
04386f7d69
nix develop: do not assume that saved vars are set
This fixes https://github.com/NixOS/nix/issues/6809
2022-07-14 23:25:39 -07:00
John Ericson
8735f55dec Fix bug, test more, document more 2022-07-14 20:23:43 -04:00
John Ericson
f3262bc216 Combine InstallableStorePath with InstallableIndexedStorePath
No behavior should be changed, the `isDerivation` logic is moved from
the methods to the constructor.
2022-07-14 19:37:55 -04:00
John Ericson
6cafe308c9 Merge remote-tracking branch 'upstream/master' into indexed-store-path-outputs 2022-07-14 16:15:37 -04:00
Eelco Dolstra
ff49c75502 Disable auto-chroot if $NIX_STORE_DIR is set
Fixes #6732.
2022-07-14 17:47:09 +02:00
Eelco Dolstra
12df8885cc Simplify the check for overrides on non-existent inputs 2022-07-13 13:40:40 +02:00
Ikko Ashimine
694a9dc282
Fix typo in flake.cc
non-existant -> non-existent
2022-07-13 01:10:32 +09:00
Théophane Hufschmitt
2dbd5ed0b4
Merge pull request #6663 from Ma27/follows-invalid-input
flakes: throw an error if `follows`-declaration for an input is invalid
2022-07-12 16:44:22 +02:00