Commit graph

1733 commits

Author SHA1 Message Date
Robert Hensing
3511430902
Merge pull request #9673 from pennae/drv-parse-opts
optimize derivation parsing
2023-12-31 13:49:03 +01:00
Rebecca Turner
7434caca05
Fix segfault on infinite recursion in some cases
This fixes a segfault on infinite function call recursion (rather than
infinite thunk recursion) by tracking the function call depth in
`EvalState`.

Additionally, to avoid printing extremely long stack traces, stack
frames are now deduplicated, with a `(19997 duplicate traces omitted)`
message. This should only really be triggered in infinite recursion
scenarios.

Before:

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)'
    Segmentation fault: 11

After:

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)'
    error: stack overflow

           at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)' --show-trace
    error:
           … from call site
             at «string»:1:1:
                1| (x: x x) (x: x x)
                 | ^

           … while calling anonymous lambda
             at «string»:1:2:
                1| (x: x x) (x: x x)
                 |  ^

           … from call site
             at «string»:1:5:
                1| (x: x x) (x: x x)
                 |     ^

           … while calling anonymous lambda
             at «string»:1:11:
                1| (x: x x) (x: x x)
                 |           ^

           … from call site
             at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^

           (19997 duplicate traces omitted)

           error: stack overflow
           at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^
2023-12-29 22:16:44 -08:00
pennae
2cfc4ace35 malloc/memset even less
more buffers that can be uninitialized and on the stack. small
difference, but still worth doing.

before:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.963 s ±  0.011 s    [User: 5.330 s, System: 1.421 s]
  Range (min … max):    6.943 s …  6.974 s    10 runs

after:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.952 s ±  0.015 s    [User: 5.294 s, System: 1.452 s]
  Range (min … max):    6.926 s …  6.974 s    10 runs
2023-12-30 00:45:11 +01:00
Robert Hensing
ee439734e9
Merge pull request #9582 from pennae/misc-opts
a packet of small optimizations
2023-12-22 17:00:59 +01:00
John Ericson
d65d56fa77
Merge pull request #9655 from iFreilicht/move-flake-url-name-util
Move url-name utility to libexpr/flake
2023-12-22 10:26:42 -05:00
Robert Hensing
d2016c6b59
Merge pull request #9621 from blaggacao/fix/too-restrictive-branch-regex-master
fix: valid branch name
2023-12-22 16:02:25 +01:00
Felix Uhl
26d7b0c793 Move url-name utility to libexpr/flake 2023-12-22 09:33:02 +01:00
Robert Hensing
4f47152209
libutil/url-parts.hh: Fix regex
Regex syntax is awful.
2023-12-21 23:11:25 +01:00
Robert Hensing
8c4ea12f11
libutil/url-parts.hh: comment 2023-12-21 21:03:06 +01:00
Eelco Dolstra
d77a39a314 Fix indent 2023-12-21 16:22:34 +01:00
Eelco Dolstra
9d9d9ff0de Merge remote-tracking branch 'origin/master' into profile-names-instead-of-index 2023-12-21 16:21:26 +01:00
pennae
f9aee2f2c4 don't malloc/memset posix accessor buffer
it's relatively small and fits on the stack nicely, and we don't need it
initialized either.
2023-12-19 19:32:16 +01:00
pennae
cc4038d541 use std::tie() for macro-generated operators
as written the comparisons generate copies, even though it looks as
though they shouldn't.

before:

  Time (mean ± σ):      4.396 s ±  0.002 s    [User: 3.894 s, System: 0.501 s]
  Range (min … max):    4.393 s …  4.399 s    10 runs

after:

  Time (mean ± σ):      4.260 s ±  0.003 s    [User: 3.754 s, System: 0.505 s]
  Range (min … max):    4.257 s …  4.266 s    10 runs
2023-12-19 19:32:16 +01:00
John Ericson
dfc876531f Organize content addressing, use SourceAccessor with Store::addToStore
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:41:54 -05:00
David Arnold
c05d4fadd5
fix: valid branch name 2023-12-16 23:14:33 +01:00
Rebecca Turner
5cb98095ba
Remove some blank lines from stack traces
This keeps hint messages, source location information, and source code
snippets grouped together, while making stack traces shorter (so that
more stack frames can be viewed on the same terminal).

Before:

    error:
           … while evaluating the attribute 'body'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:

                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:

                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed

           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|

After:

    error:
           … while evaluating the attribute 'body'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:
                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:
                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed
           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|
2023-12-15 23:57:26 -08:00
Robert Hensing
0b87ba50c0 Revert "Add nix::isASCII*, locale-independent"
This reverts commit 79eb2920bb.

Not used at this time.
2023-12-12 17:46:34 +01:00
Robert Hensing
4eaeda6604 isValidSchemeName: Use regex
As requested by Eelco Dolstra. I think it used to be simpler.
2023-12-12 17:46:34 +01:00
Robert Hensing
2e451a663e schemeRegex -> schemeNameRegex
Scheme could be understood to include the typical `:` separator.
2023-12-12 17:25:20 +01:00
Robert Hensing
d3a85b6834 isValidSchemeName: Add function 2023-12-11 12:12:43 +01:00
Robert Hensing
79eb2920bb Add nix::isASCII*, locale-independent 2023-12-11 12:12:43 +01:00
John Ericson
ce4ca574d2 Clarify SourceAccessor methods should never implicitly follow symlinks
The code has already been fixed (yay!) so what is left of this commit is
just updating the API docs.

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2023-12-08 20:00:47 -05:00
Robert Hensing
d4f6b1d38b
Merge pull request #9497 from edolstra/move-access-control
Move restricted/pure-eval access control out of the evaluator and into the accessor
2023-12-08 22:21:50 +01:00
Bryan Honof
bf00d5ecef
fix(libutil/tarfile): add option to libarchive so it behaves correctly with AppleDouble files
AppleDouble files were extracted differently on macOS machines than on other
UNIX's.
Setting `archive_read_set_format_option(this->archive, NULL ,"mac-ext",NULL)`
fixes this problem, since it just ignores the AppleDouble file and treats it as
a normal one.
This was a problem since it caused source archives to be different between macOS
and Linux.

Ref: nixos/nix#9290
2023-12-07 11:35:15 +01:00
Peter Kolloch
fc6f29053a Renamed HashFormat::Base32 to HashFormat::Nix32
...and also adjusted parsing accordingly.

Also added CLI completion for HashFormats.

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch
837b889c41 Further HashType renaming + using mkHashAlgoOptFlag for new conversion
https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch
5334c9c792 HashType: Rename to HashAlgorithm
To be consistent with CLI, nix API
and many other references.

As part of this, we also converted it to a scoped enum.

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch
0c2d5f7673 nix hash convert: s/--type/--algo/ + more functional tests
https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch
e7abf60a0c hash.cc/hash.h: Minor C++ improvements 2023-12-06 23:41:07 +01:00
Théophane Hufschmitt
7fff625e39
Improve the error message for “multicommands” commands (#9510)
* Factor out the default `MultiCommand` behavior

All the `MultiCommand`s had (nearly) the same behavior when called
without a subcommand.
Factor out this behavior into the `NixMultiCommand` class.

* Display the list of available subcommands when none is specified

Whenever a user runs a command that excepts a subcommand, add the list
of available subcommands to the error message.

* Print the multi-command lists as Markdown lists

This takes more screen real estate, but is also much more readable than
a comma-separated list
2023-12-06 13:13:45 +00:00
Eelco Dolstra
57246c4c38 PosixSourceAccessor: Cache lstat() calls
Since we're doing a lot of them in assertNoSymlinks().
2023-12-06 13:55:54 +01:00
Eelco Dolstra
504e4fc457 CanonPath: Support std::hash 2023-12-06 13:45:59 +01:00
Eelco Dolstra
83c067c0fa PosixSourceAccessor: Don't follow any symlinks
All path components must not be symlinks now (so the user needs to
call `resolveSymlinks()` when needed).
2023-12-05 23:02:59 +01:00
Valentin Gagarin
3c310bde2e reword description for the fetch-tree experimental feature
without knowing a lot of context, it's not clear who "we" are in that
text. I'm also strongly opposed to adding procedural notes into
a reference manual; it just won't age well.

this change leaves a factual description of the experimental feature and
its purpose.
2023-12-04 16:42:32 +01:00
John Ericson
91b6833686 Move tests to separate directories, and document
Today, with the tests inside a `tests` intermingled with the
corresponding library's source code, we have a few problems:

- We have to be careful that wildcards don't end up with tests being
  built as part of Nix proper, or test headers being installed as part
  of Nix proper.

- Tests in libraries but not executables is not right:

  - It means each executable runs the previous unit tests again, because
    it needs the libraries.

  - It doesn't work right on Windows, which doesn't want you to load a
    DLL just for the side global variable . It could be made to work
    with the dlopen equivalent, but that's gross!

This reorg solves these problems.

There is a remaining problem which is that sibbling headers (like
`hash.hh` the test header vs `hash.hh` the main `libnixutil` header) end
up shadowing each other. This PR doesn't solve that. That is left as
future work for a future PR.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-01 10:48:58 -05:00
John Ericson
82359eba6b
Merge pull request #9233 from bouk/bouk/apply-config-inner
config: add included files into parsedContents before applying
2023-12-01 08:23:32 -05:00
Valentin Gagarin
4781e7fa70 Document each store type on its own page
This makes for more useful manual table of contents, that displays the
information at a glance.

The `nix help-stores` command is kept as-is, even though it will show up
in the manual with the same information as these pages due to the way it
is written as a "`--help`-style" command. Deciding what to do with that
command is left for a later PR.

This change also lists all store types at the top of the respective overview page.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems
2023-12-01 01:27:52 +01:00
Valentin Gagarin
39de819eda rename debugging helper environment variable 2023-11-30 23:23:32 +01:00
Alex Ameen
02bd821f2e
fix: nlohmann::adl_serializer for std::optional (#9147)
This allows templates such as `NLOHMANN_DEFINE_TYPE_*` templates and other generators with things like `std::vector<std::optional<T>>`.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2023-11-30 01:26:39 +00:00
John Ericson
949f5841f8 Add the MountedSSHStore experimental feature
It will be implemented in the subsequent commits of this PR.
2023-11-21 13:16:46 -05:00
Eelco Dolstra
a0162d5732 Improve SourceAccessor path display
Backported from lazy-trees. This allows SourceAccessors to show the
origin of the accessor. E.g. we now get

  copying '«git+https://github.com/blender/blender.git?ref=refs/heads/main&rev=4edc1389337dd3679ff66969c332d2aff52e1992»/' to the store

instead of

  copying '/' to the store
2023-11-20 18:54:36 +01:00
Eelco Dolstra
f450c8773c
Merge pull request #9408 from roblabla/fix-bad-format-string
Fix bad_format_string error when builder stdout contains %
2023-11-20 17:22:40 +01:00
roblabla
e2b6821ca0 Fix bad_format_string error when builder stdout contains % 2023-11-20 15:41:38 +01:00
John Ericson
4ab27e5595 Merge remote-tracking branch 'upstream/master' into libgit2 2023-11-20 08:30:26 -05:00
Robert Hensing
251fb23aea Shebang parser: add virtual destructor
Fixes:

    warning: destructor called on non-final 'nix::ParseUnquoted' that has virtual functions but non-virtual destructor [-Wdelete-non-abstract-non-virtual-dtor]
2023-11-19 01:48:08 +01:00
John Ericson
293ae59257 Fix make check
After 9c7749e135, `libutil-tests_RUN`
doesn't exist. It needs to become `libutil-tests-exe_RUN`.
2023-11-17 11:26:45 -05:00
John Ericson
6c8f4ef350 Allow installing unit tests
Closes #9343

See that issue for motivation.

Installing these is disabled by default, but we enable it (and the
additional output we want isntall these too so as not to clutter the
existing ones) to use in cross builds and dev shells.
2023-11-16 09:55:42 -05:00
Eelco Dolstra
2890999911 Show Git fetch progress 2023-11-15 13:57:20 +01:00
John Ericson
9c7749e135 Fix makefile bug confusing libnixutil-test exe vs lib
The `-exe` variant is the program, the unsuffixed variant is the
library.

The corrected usage matches `libnixstore-test`.
2023-11-14 11:45:52 -05:00
Eelco Dolstra
2964a9f562 Fix relative submodule handling
Tested on

  nix flake prefetch 'git+https://github.com/blender/blender.git?rev=4ed8a360e956daf2591add4d3c9ec0719e2628fe&submodules=1'
2023-11-14 16:00:21 +01:00