Commit graph

14549 commits

Author SHA1 Message Date
Ben Burdette
c941803861 spacing 2022-04-29 11:27:38 -06:00
Ben Burdette
172a83d22a line endings 2022-04-29 11:24:54 -06:00
Ben Burdette
ca6cba8b81 fix 'suggestions' error 2022-04-29 10:51:10 -06:00
Ben Burdette
2a5632c70d incorporate PosIdx changes, symbol changes. 2022-04-29 10:02:17 -06:00
Guillaume Maudoux
9ff892aad4 Add release notes for error traces revamp 2022-04-29 11:24:48 +02:00
Guillaume Maudoux
143b73f52d tests: Distinguish crashes from expected failures 2022-04-29 11:14:08 +02:00
Guillaume Maudoux
e93b59fbc5 Merge remote-tracking branch 'origin/master' into coerce-string 2022-04-29 00:12:25 +02:00
Ben Burdette
6e19947993 Merge branch 'master' into debug-merge-master 2022-04-28 12:32:57 -06:00
Eelco Dolstra
4bb111c8d4
Merge pull request #6461 from edolstra/libcxx
Fix libcxx build
2022-04-28 15:11:49 +02:00
Eelco Dolstra
70a30dbc00 Fix libcxx build
Fixes #6458.
2022-04-28 14:37:05 +02:00
Eelco Dolstra
cb5361628d Add libcxxStdenv devshell 2022-04-28 14:36:48 +02:00
Eelco Dolstra
d77d813017 Shut up clang warning 2022-04-28 14:24:17 +02:00
Eelco Dolstra
660835da69
Merge pull request #6459 from edolstra/fix-out-paths
Fix passing $OUT_PATHS to the post-build hook
2022-04-28 14:05:29 +02:00
Eelco Dolstra
4a9623b129 Fix passing $OUT_PATHS to the post-build hook
Fixes #6446.
2022-04-28 13:36:01 +02:00
Guillaume Maudoux
f6baa4d188 fixup! fix errors case and wording 2022-04-28 13:18:19 +02:00
Guillaume Maudoux
402ee8ab64 No point in passing string_views by reference 2022-04-28 13:02:39 +02:00
Guillaume Maudoux
5ef88457b8 Better document error location indent 2022-04-28 13:00:24 +02:00
Guillaume Maudoux
acf990c9ea fix errors case and wording 2022-04-28 12:54:14 +02:00
Adam Joseph
49119072e7 local-derivation-goal.cc: seccomp filters for MIPS secondary arch/abi
A mips64el Linux MIPS kernel can execute userspace code using any of
three ABIs:

  mips64el-linux-*abin64
  mips64el-linux-*abin32
    mipsel-linux-*

The first of these is the native 64-bit ABI, and the only ABI with
64-bit pointers; this is sometimes called "n64".  The last of these is
the old legacy 32-bit ABI, whose binaries can execute natively on
32-bit MIPS hardware; this is sometimes called "o32".

The second ABI, "n32" is essentially the 64-bit ABI with 32-bit
pointers and address space.  Hardware 64-bit integer/floating
arithmetic is still allowed, as well as the much larger mips64
register set and more-efficient calling convention.

Let's enable seccomp filters for all of these.  Likewise for big
endian (mips64-linux-*).
2022-04-27 21:39:26 -07:00
Eelco Dolstra
35393dc2c6
Merge pull request #6444 from midchildan/feat/zsh-runhelp
feat: add integration with zsh's run-help
2022-04-26 21:26:33 +02:00
midchildan
68d51ae012
refactor: don't hardcode nix subcommands in run-help-nix 2022-04-27 01:56:10 +09:00
Eelco Dolstra
717298c749 Bump eval cache schema version 2022-04-26 17:17:51 +02:00
Eelco Dolstra
13d8400ac5 Remove obsolete FIXME 2022-04-26 17:17:27 +02:00
Eelco Dolstra
6e0a2b971b Add a test for outputsToInstall 2022-04-26 17:17:27 +02:00
Eelco Dolstra
1ddabe1a01 nix: Respect meta.outputsToInstall, and use all outputs by default
'nix profile install' will now install all outputs listed in the
package's meta.outputsToInstall attribute, or all outputs if that
attribute doesn't exist. This makes it behave consistently with
nix-env. Fixes #6385.

Furthermore, for consistency, all other 'nix' commands do this as
well. E.g. 'nix build' will build and symlink the outputs in
meta.outputsToInstall, defaulting to all outputs. Previously, it only
built/symlinked the first output. Note that this means that selecting
a specific output using attrpath selection (e.g. 'nix build
nixpkgs#libxml2.dev') no longer works. A subsequent PR will add a way
to specify the desired outputs explicitly.
2022-04-26 17:17:27 +02:00
Eelco Dolstra
a81622c21d
Merge pull request #6447 from edolstra/eval-cache-symbols
EvalCache: Use Symbol in more places
2022-04-26 15:41:40 +02:00
Théophane Hufschmitt
fb5f13fb65
Merge pull request #6440 from Ma27/fix-nix-with-latest-git
libfetchers/git: hardcode `--git-dir`
2022-04-26 14:50:47 +02:00
Eelco Dolstra
b12c33510c EvalCache AttrKey: Use Symbol instead of std::string 2022-04-26 14:16:20 +02:00
Eelco Dolstra
474695975d EvalCache: Revert to using symbols in getAttr() 2022-04-26 14:01:42 +02:00
Eelco Dolstra
fab731a9d4 Don't pass Symbol by reference
Since Symbol is just an integer, passing it by const reference is
never advantageous.
2022-04-26 13:25:17 +02:00
Eelco Dolstra
29e52194a8 Merge branch 'symbolidx-symbol' of https://github.com/pennae/nix 2022-04-26 13:25:10 +02:00
pennae
d6d6bbd9ef
Merge branch 'master' into lto 2022-04-25 14:02:37 +00:00
pennae
a385e51a08 rename SymbolIdx -> Symbol, Symbol -> SymbolStr
after #6218 `Symbol` no longer confers a uniqueness invariant on the
string it wraps, it is now possible to create multiple symbols that
compare equal but whose string contents have different addresses. this
guarantee is now only provided by `SymbolIdx`, leaving `Symbol` only as
a string wrapper that knows about the intricacies of how symbols need to
be formatted for output.

this change renames `SymbolIdx` to `Symbol` to restore the previous
semantics of `Symbol` to that name. we also keep the wrapper type and
rename it to `SymbolStr` instead of returning plain strings from lookups
into the symbol table because symbols are formatted for output in many
places. theoretically we do not need `SymbolStr`, only a function that
formats a string for output as a symbol, but having to wrap every symbol
that appears in a message into eg `formatSymbol()` is error-prone and
inconvient.
2022-04-25 15:37:01 +02:00
midchildan
0f7c7ab97b
fix: typo make-content-addressable -> make-content-addressed 2022-04-25 02:34:59 +09:00
midchildan
c67d8876c3
feat: add integration with zsh's run-help 2022-04-25 02:21:41 +09:00
Maximilian Bosch
d1f5356311
libfetchers/git: fix for nixos-rebuild
The `--git-dir=` must be `.` in some cases (for cached repos that are
"bare" repos in `~/.cache/nix/gitv3`). With this fix we can add
`--git-dir` to each `git`-invokation needed for `nixos-rebuild`.
2022-04-24 18:14:24 +02:00
Maximilian Bosch
0256e5578e
libfetchers/git: hardcode --git-dir
To demonstrate the problem:

* You need a `git` at 2.33.3 in your $PATH
* An expression like this in a git repository:

  ``` nix
  {
    outputs = { self, nixpkgs }: {
      packages.foo.x86_64-linux = with nixpkgs.legacyPackages.x86_64-linux;
        runCommand "snens" { } ''
          echo ${(builtins.fetchGit ./.).lastModifiedDate} > $out
        '';
    };
  }
  ```

Now, when instantiating the package via `builtins.getFlake`, it fails on
Nix 2.7 like this:

    $ nix-instantiate -E '(builtins.getFlake "'"$(pwd)"'").packages.foo.x86_64-linux'
    fatal: unsafe repository ('/nix/store/a7j3125km4h8l0p71q6ssfkxamfh5d61-source' is owned by someone else)
    To add an exception for this directory, call:

    	git config --global --add safe.directory /nix/store/a7j3125km4h8l0p71q6ssfkxamfh5d61-source
    error: program 'git' failed with exit code 128
    (use '--show-trace' to show detailed location information)

This breaks e.g. `nixops`-deployments using flakes with similar
expressions as shown above.

The cause for this is that `git(1)` tries to find the highest
`.git`-directory in the directory tree and if it finds a such a
directory, but with another owning user (root vs. the user who evaluates
the expression), it fails as above. This was changed recently to fix
CVE-2022-24765[1].

By explicitly specifying `--git-dir`, Git assumes to be in the top-level
directory and doesn't attempt to look for a `.git`-directory in the
parent directories and thus the code-path leading to said error is never
reached.

[1] https://lore.kernel.org/git/xmqqv8veb5i6.fsf@gitster.g/
2022-04-23 23:20:17 +02:00
Théophane Hufschmitt
7f814d6d9a
Merge pull request #6259 from Artturin/nixbuildaddprintstorepaths
nix build: add --print-out-paths flag
2022-04-22 14:21:55 +02:00
Théophane Hufschmitt
197aa2fa96 Merge remote-tracking branch 'origin/master' into nixbuildaddprintstorepaths 2022-04-22 13:20:39 +02:00
Théophane Hufschmitt
92de62fe6f
Merge pull request #6437 from NixOS/fix-darwin-build
Fix the darwin build
2022-04-22 13:19:53 +02:00
Théophane Hufschmitt
be28603dca Merge remote-tracking branch 'origin/master' into nixbuildaddprintstorepaths 2022-04-22 11:11:01 +02:00
Théophane Hufschmitt
7b889f31ea Fix the darwin build
Looks like the auto-merge is indeed quite broken and merges even when the CI fails
2022-04-22 10:56:56 +02:00
Théophane Hufschmitt
c4ffc8e2f8
Merge pull request #6218 from pennae/pos-symbol-tables
reduce the size of Attr from 3 pointers to 2 on 64 bit machines
2022-04-22 10:28:06 +02:00
Théophane Hufschmitt
484badfa09 Add some tests for ChunkedVector 2022-04-22 10:03:44 +02:00
Théophane Hufschmitt
7ca6fbc8ca Move ChunkedVector to its own header 2022-04-22 10:01:02 +02:00
Théophane Hufschmitt
35ca5fdf91
Merge pull request #6436 from flox/tofile_allow
fix: builtins.toFile adds path to allowedPaths
2022-04-22 08:50:54 +02:00
Tom Bereknyei
f25112d383 fix: builtins.toFile adds path to allowedPaths
The produced path is then allowed be imported or utilized elsewhere:
```
assert (43 == import (builtins.toFile "source" "43")); "good"
```

This will still fail on write-only stores.
2022-04-21 16:41:37 -04:00
pennae
8adaa6acb5 remove pos<T>
it's no longer needed now that positions aren't really pointers any
more.
2022-04-21 21:56:34 +02:00
pennae
8168a4cf4a shrink Attr by 8 bytes on 64bit machines
with position and symbol tables in place we can now shrink Attr by a full
pointer with some simple field reordering. since Attr is a very hot struct this
has substantial impact on memory use, decreasing GC allocations and heap size by
10-15% each. we also get a ~15% performance improvement due to reduced GC
loading.

pure parsing has taken a hit over the branch base because positions are now
slightly more expensive to create, but overall we get a noticeable improvement.

before (on memory-friendliness):

  Benchmark 1: nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.960 s ±  0.028 s    [User: 5.832 s, System: 0.897 s]
    Range (min … max):    6.886 s …  7.005 s    20 runs

  Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     328.1 ms ±   1.7 ms    [User: 295.8 ms, System: 32.2 ms]
    Range (min … max):   324.9 ms … 331.2 ms    20 runs

  Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.688 s ±  0.029 s    [User: 2.365 s, System: 0.238 s]
    Range (min … max):    2.642 s …  2.742 s    20 runs

after:

  Benchmark 1: nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      6.902 s ±  0.039 s    [User: 5.844 s, System: 0.783 s]
    Range (min … max):    6.820 s …  6.956 s    20 runs

  Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     330.7 ms ±   2.2 ms    [User: 300.6 ms, System: 30.0 ms]
    Range (min … max):   327.5 ms … 334.5 ms    20 runs

  Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.330 s ±  0.027 s    [User: 2.040 s, System: 0.234 s]
    Range (min … max):    2.272 s …  2.383 s    20 runs
2022-04-21 21:56:34 +02:00
pennae
8775be3393 store Symbols in a table as well, like positions
this slightly increases the amount of memory used for any given symbol, but this
increase is more than made up for if the symbol is referenced more than once in
the EvalState that holds it. on average every symbol should be referenced at
least twice (once to introduce a binding, once to use it), so we expect no
increase in memory on average.

symbol tables are limited to 2³² entries like position tables, and similar
arguments apply to why overflow is not likely: 2³² symbols would require as many
string instances (at 24 bytes each) and map entries (at 24 bytes or more each,
assuming that the map holds on average at most one item per bucket as the docs
say). a full symbol table would require at least 192GB of memory just for
symbols, which is well out of reach. (an ofborg eval of nixpks today creates
less than a million symbols!)
2022-04-21 21:56:31 +02:00