max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2024-11-24 23:06:16 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
17654 commits 7 branches 0 tags 64 MiB
Commit graph

2962 commits

Author SHA1 Message Date
John Ericson
d902481a36 Better org 2024-07-02 09:23:25 -04:00
John Ericson
a81e319528 Deduplicating 2024-07-02 09:23:24 -04:00
John Ericson
4fa8068b78 Mesonify other external API 2024-07-02 09:23:24 -04:00
John Ericson
31257009e1 Meson build for libexpr and libflake 2024-07-02 09:23:24 -04:00
Eelco Dolstra
10c9764c27
Merge pull request #10992 from hercules-ci/issue-10947-dont-cache-disallowed-ifd 
Fix #10947; don't cache disallowed IFD
 2024-07-01 11:20:27 +02:00
Robert Hensing
b2c7f09b0a Fix underflow in Printer::printList 
Analogous to 9b88bf8adf / three commits back
 2024-06-29 14:10:58 +02:00
Robert Hensing
bfc5416240 Refactor: rename ValuePrinter::totalListItemsPrinted 2024-06-29 14:02:28 +02:00
Robert Hensing
ce1dc87711 Refactor: rename ValuePrinter::totalAttrsPrinted 
Make it more distinct from the attrs printed of any specific
attrset.
 2024-06-29 14:01:15 +02:00
Robert Hensing
9b88bf8adf Fix underflow in Printer::printAttrs 
The code that counts the number of elided attrs incorrectly used the
per-printer "global" attribute counter instead of a counter that
was relevant only to the current attribute set.

This bug flew under the radar because often the attribute sets aren't
nested, not big enough, or we wouldn't pay attention to the numbers.
I've noticed the issue because the difference underflowed.

Although this behavior is tested by the functional test
lang/eval-fail-bad-string-interpolation-4.nix, the underflow slipped
through review. A simpler reproducer would be as follows, but I
haven't added it to the test suite to keep it simple and marginally
faster.

```
$ nix run nix/2.23.1 -- eval --expr '"" + (let v = { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; b = 1; c = 1; }; }; in builtins.deepSeq v v)'
error:
       … while evaluating a path segment
         at «string»:1:6:
            1| "" + (let v = { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; b = 1; c = 1; }; }; in builtins.deepSeq v v)
             |      ^

       error: cannot coerce a set to a string: { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; «4294967289 attributes elided» }; }
```
 2024-06-29 13:53:05 +02:00
Robert Hensing
fd94b74ee5 Fix #10947; don't cache disallowed IFD 2024-06-29 13:19:04 +02:00
Winter
149d8eb8aa Stop vendoring toml11 
We don't apply any patches to it, and vendoring it locks users into
bugs (it hasn't been updated since its introduction in late 2021).

Closes https://git.lix.systems/lix-project/lix/issues/164

Change-Id: Ied071c841fc30b0dfb575151afd1e7f66970fdb9
(cherry picked from commit 80405d06264f0de1c16ee2646388ab501df20628)
 2024-06-26 22:27:13 -04:00
John Ericson
f002f85861 Avoid libmain header in libexpr 
We just don't need it!
 2024-06-26 22:26:45 -04:00
John Ericson
0084a486cc Split out a new libnixflake 
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2024-06-26 19:56:21 -04:00
John Ericson
52730d38e2 Factor out flake:... lookup path from evaluator 
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2024-06-26 19:56:21 -04:00
Robert Hensing
7df9d6da65 Improve error messages for invalid derivation names 2024-06-25 19:41:29 +02:00
Robert Hensing
c66f1e7660
Merge pull request #10913 from NixOS/no-global-eval-settings-in-libexpr 
No global eval settings in `libnixexpr`
 2024-06-24 18:52:19 +02:00
John Ericson
fda4c78921
Merge pull request #10951 from obsidiansystems/load-just-one-config 
Small global config refactors
 2024-06-24 12:38:04 -04:00
John Ericson
52bfccf8d8 No global eval settings in libnixexpr 
Progress on #5638

There is still a global eval settings, but it pushed down into
`libnixcmd`, which is a lot less bad a place for this sort of thing.
 2024-06-24 12:15:16 -04:00
John Ericson
1620ad4587 Split out GlobalConfig into its own header 
This makes it easier to understand the reach of global variables /
global state in the config system.
 2024-06-24 11:36:21 -04:00
John Ericson
b51e161af5 Cleanup ContentAddressMethod to match docs 
The old `std::variant` is bad because we aren't adding a new case to
`FileIngestionMethod` so much as we are defining a separate concept ---
store object content addressing rather than file system object content
addressing. As such, it is more correct to just create a fresh
enumeration.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2024-06-24 10:24:06 -04:00
John Ericson
64e599ebe1 Rename Recursive -> NixArchive 
For enums:

- `FileIngestionMethod`

- `FileSerialisationMethod`
 2024-06-24 10:24:06 -04:00
Robert Hensing
6f64154eea
Merge pull request #10884 from tomberek/tomberek.warn_structuredAttrs_advanced 
fix: warn and document when advanced attributes will have no impact d…
 2024-06-24 07:56:26 +02:00
John Ericson
5b53d8fec3 Factor out GC initialization code 
This is not really part of the evaluator: it is just an integration
between Boehm GC and Boost coroutines usable for any purpose. The
evaluator (merely) optionally uses it.
 2024-06-12 16:00:03 -04:00
John Ericson
7c2981fc55 Fix FreeBSD build 
This restores some CPP'd code that was added in
c18911602e and accidentally lost in
2477e4e3b8.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
 2024-06-12 15:59:54 -04:00
Tom Bereknyei
4809e59b7e fix: warn and document when advanced attributes will have no impact due to __structuredAttrs 2024-06-10 09:31:21 -04:00
Robert Hensing
f91f34aa65 bdwgc 8.2.4 has sp_corrector 
> Support client-defined stack pointer adjustment before thread stack push

-- https://github.com/ivmai/bdwgc/releases/tag/v8.2.4

This fixes an inaccuracy in cc6f315252,
in the update to Nixpkgs 24.05 https://github.com/NixOS/nix/pull/10835

After this fixup, the build log won't ask for an upgrade, and we'll be
able to collect when a coroutine exists, e.g. during filterSource.
 2024-06-10 12:57:13 +02:00
Robert Hensing
0a09597790 Typo 2024-06-10 12:46:04 +02:00
Robert Hensing
4755e133c4 Fix warning 2024-06-10 12:46:04 +02:00
Robert Hensing
0ab9369572
Merge pull request #10854 from DeterminateSystems/thread-safe-RegexCache 
Make RegexCache thread-safe
 2024-06-08 18:07:35 +02:00
Robert Hensing
e1b3716d50
Merge pull request #10853 from DeterminateSystems/thread-safe-srcToStore 
Make EvalState::srcToStore thread-safe
 2024-06-06 19:31:46 +02:00
John Ericson
1a32490aca
Merge pull request #10835 from hercules-ci/update-nixpkgs-24.05 
Update to Nixpkgs 24.05
 2024-06-05 17:56:16 -04:00
Eelco Dolstra
3e72ed9743 Typo 2024-06-05 16:19:01 +02:00
Eelco Dolstra
d2eeabf3e6 PackageInfo::queryDrvPath(): Don't dereference an empty optional 
Fixes a regression introduced in f923ed6b6a.

https://hydra.nixos.org/build/262267313
 2024-06-05 16:17:24 +02:00
Jörg Thalheim
49c6f34911
docs: fixup description of builtins.importNative (#10810) 
There was an argument missing and the fact that the imported function is
called.
 2024-06-04 19:55:05 +00:00
Eelco Dolstra
fbbca59453 Make RegexCache thread-safe 2024-06-04 16:59:30 +02:00
Eelco Dolstra
7d295c594e Make EvalState::srcToStore thread-safe 2024-06-04 16:56:06 +02:00
John Ericson
4e0d058fc3 eval.cc: Fix for Windows 2024-06-04 10:18:22 +02:00
Robert Hensing
cc6f315252 nix: Disable GC during coroutine when bdwgc < 8.4 
This re-enables support for older bwdgc versions without complicating
the code too much.
Coroutines generally only interfere with GC during source filtering,
so it's not too bad of a regression on older bdwgc.
This seems preferable over conditional compilation to enable the patch
etc; we've already spent a lot of complexity budget on this GC-coroutine
interaction...
 2024-06-03 16:37:39 +02:00
Robert Hensing
2477e4e3b8 libexpr: Use GC_set_sp_corrector instead of patch 
Manually tested by printing to stderr in both branches (sp in os
stack, or not), and triggering a GC in a filterSource function,
e.g.:

    let
      generateTree = n: if n == 0 then "ha" else { left = generateTree (n - 1); right = generateTree (n - 1); };
    in
      builtins.deepSeq (generateTree 18) ...

Note that the darwin still uses the strategy of disabling GC, despite
having an implementation that compiles. The proper solution will be
enabled and tested later.
 2024-06-03 16:37:39 +02:00
Robert Hensing
70b1036224 builtins.warn: Use new EvalBaseError + "evaluation warning" 2024-06-03 16:24:21 +02:00
Robert Hensing
831d96d8d7 builtins.warn: Do not throw EvalError 2024-06-03 16:24:21 +02:00
Robert Hensing
c07500e14d refactor: Extract EvalState::{runDebugRepl,canDebug} 2024-06-03 16:24:21 +02:00
Robert Hensing
da82d67022 builtins.warn: Require string argument 
... so that we may perhaps later extend the interface.
Note that Nixpkgs' lib.warn already requires a string coercible
argument, so this is reasonable. Also note that string coercible
values aren't all strings, but in practice, for warn, they are.
 2024-06-03 16:24:21 +02:00
Robert Hensing
923cbea2af builtins.warn: Use logWarning 
Constructing ErrorInfo is a little awkward for now, but this does
produce a richer log entry.
 2024-06-03 16:24:21 +02:00
Robert Hensing
2d4c9d8f4a Add builtins.warn 2024-06-03 16:24:21 +02:00
Eelco Dolstra
ac3e5d22e3
Merge pull request #10028 from DavHau/fetchTree-shallow-default 
fetchTree: shallow git fetching by default
 2024-06-03 16:02:34 +02:00
Eelco Dolstra
ecfad6a828
Merge pull request #10564 from edolstra/remove-forceErrors 
AttrCursor: Remove forceErrors
 2024-06-03 15:30:01 +02:00
Jade Lovelace
473d2d56fc Remove 100s of CPU time (10%) from build times (1465s -> 1302s) 
Result's from Mic92's framework 13th Gen Intel Core i7-1360P:

Before: 3595.92s user 183.01s system 1360% cpu 4:37.74 total
After: 3486.07s user 168.93s system 1354% cpu 4:29.79 total

I saw that boost/lexical_cast was costing about 100s in CPU time on our
compiles. We can fix this trivially by doing explicit template
instantiation in exactly one place and eliminating all other includes of
it, which is a code improvement anyway by hiding the boost.

Before:
```
lix/lix2 » ClangBuildAnalyzer --analyze buildtimeold.bin
Analyzing build trace from 'buildtimeold.bin'...
**** Time summary:
Compilation (551 times):
  Parsing (frontend):         1465.3 s
  Codegen & opts (backend):   1110.9 s

<snip>

**** Expensive headers:
178153 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 3426 ms), included via:
  40x: command.hh
  5x: command-installable-value.hh
  3x: installable-flake.hh
  2x: <direct include>
  2x: installable-attr-path.hh

176217 ms: ../src/libutil/error.hh (included 246 times, avg 716 ms), included via:
  36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh
  12x: globals.hh config.hh experimental-features.hh
  11x: file-system.hh file-descriptor.hh
  6x: serialise.hh strings.hh
  6x: <direct include>
  6x: archive.hh serialise.hh strings.hh
  ...

173243 ms: ../src/libstore/store-api.hh (included 152 times, avg 1139 ms), included via:
  55x: <direct include>
  39x: command.hh installable-value.hh installables.hh
  7x: libexpr.hh
  4x: local-store.hh
  4x: command-installable-value.hh installable-value.hh installables.hh
  3x: binary-cache-store.hh
  ...

170482 ms: ../src/libutil/serialise.hh (included 201 times, avg 848 ms), included via:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh
  14x: store-api.hh nar-info.hh hash.hh
  11x: <direct include>
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh
  6x: fetchers.hh hash.hh
  ...

169397 ms: ../src/libcmd/installables.hh (included 53 times, avg 3196 ms), included via:
  40x: command.hh installable-value.hh
  5x: command-installable-value.hh installable-value.hh
  3x: installable-flake.hh installable-value.hh
  2x: <direct include>
  1x: installable-derived-path.hh
  1x: installable-value.hh
  ...

159740 ms: ../src/libutil/strings.hh (included 221 times, avg 722 ms), included via:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh
  19x: <direct include>
  14x: store-api.hh nar-info.hh hash.hh serialise.hh
  11x: serialise.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh
  ...

156796 ms: ../src/libcmd/command.hh (included 51 times, avg 3074 ms), included via:
  42x: <direct include>
  7x: command-installable-value.hh
  2x: installable-attr-path.hh

150392 ms: ../src/libutil/types.hh (included 251 times, avg 599 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh
  11x: file-system.hh
  10x: globals.hh
  6x: fetchers.hh
  6x: serialise.hh strings.hh error.hh
  5x: archive.hh
  ...

133101 ms: /nix/store/644b90j1vms44nr18yw3520pzkrg4dd1-boost-1.81.0-dev/include/boost/lexical_cast.hpp (included 226 times, avg 588 ms), included via
:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh strings.hh
  19x: file-system.hh
  11x: store-api.hh nar-info.hh hash.hh serialise.hh strings.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  6x: eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  ...

132887 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 507 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh
  16x: gtest.h
  11x: file-system.hh types.hh ref.hh
  10x: globals.hh types.hh ref.hh
  10x: json.hpp
  6x: serialise.hh
  ...

  done in 0.6s.
```

After:
```
lix/lix2 » maintainers/buildtime_report.sh build
Processing all files and saving to '/home/jade/lix/lix2/maintainers/../buildtime.bin'...
  done in 0.6s. Run 'ClangBuildAnalyzer --analyze /home/jade/lix/lix2/maintainers/../buildtime.bin' to analyze it.
Analyzing build trace from '/home/jade/lix/lix2/maintainers/../buildtime.bin'...
**** Time summary:
Compilation (551 times):
  Parsing (frontend):         1302.1 s
  Codegen & opts (backend):    956.3 s

<snip>

**** Expensive headers:
178145 ms: ../src/libutil/error.hh (included 246 times, avg 724 ms), included via:
  36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh
  12x: globals.hh config.hh experimental-features.hh
  11x: file-system.hh file-descriptor.hh
  6x: <direct include>
  6x: serialise.hh strings.hh
  6x: fetchers.hh hash.hh serialise.hh strings.hh
  ...

154043 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 2962 ms), included via:
  40x: command.hh
  5x: command-installable-value.hh
  3x: installable-flake.hh
  2x: <direct include>
  2x: installable-attr-path.hh

153593 ms: ../src/libstore/store-api.hh (included 152 times, avg 1010 ms), included via:
  55x: <direct include>
  39x: command.hh installable-value.hh installables.hh
  7x: libexpr.hh
  4x: local-store.hh
  4x: command-installable-value.hh installable-value.hh installables.hh
  3x: binary-cache-store.hh
  ...

149948 ms: ../src/libutil/types.hh (included 251 times, avg 597 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh
  11x: file-system.hh
  10x: globals.hh
  6x: fetchers.hh
  6x: serialise.hh strings.hh error.hh
  5x: archive.hh
  ...

144560 ms: ../src/libcmd/installables.hh (included 53 times, avg 2727 ms), included via:
  40x: command.hh installable-value.hh
  5x: command-installable-value.hh installable-value.hh
  3x: installable-flake.hh installable-value.hh
  2x: <direct include>
  1x: installable-value.hh
  1x: installable-derived-path.hh
  ...

136585 ms: ../src/libcmd/command.hh (included 51 times, avg 2678 ms), included via:
  42x: <direct include>
  7x: command-installable-value.hh
  2x: installable-attr-path.hh

133394 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 509 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh
  16x: gtest.h
  11x: file-system.hh types.hh ref.hh
  10x: globals.hh types.hh ref.hh
  10x: json.hpp
  6x: serialise.hh
  ...

89315 ms: ../src/libstore/derived-path.hh (included 178 times, avg 501 ms), included via:
  37x: command.hh installable-value.hh installables.hh
  25x: store-api.hh realisation.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh context.hh
  6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh
  6x: libexpr.hh value.hh context.hh
  6x: shared.hh
  ...

87347 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/ostream (included 273 times, avg 319 ms), included via:
  35x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh memory unique_ptr.h
  12x: regex sstream istream
  10x: file-system.hh types.hh ref.hh memory unique_ptr.h
  10x: gtest.h memory unique_ptr.h
  10x: globals.hh types.hh ref.hh memory unique_ptr.h
  6x: fetchers.hh types.hh ref.hh memory unique_ptr.h
  ...

85249 ms: ../src/libutil/config.hh (included 213 times, avg 400 ms), included via:
  37x: command.hh installable-value.hh installables.hh derived-path.hh
  20x: globals.hh
  20x: logging.hh
  16x: store-api.hh logging.hh
  6x: <direct include>
  6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh derived-path.hh
  ...

  done in 0.5s.
```

Adapated from 18aa3e1d57
 2024-05-31 13:00:09 +02:00
Jörg Thalheim
5786e1ae7c
docs: mention importNative/exec in allow-unsafe-native-code-during-evaluation (#10803) 
* docs: mention importNative/exec in allow-unsafe-native-code-during-evaluation

Both of these still needs their own actual documentation, but they are
at least now mentioned that they exist and what they're enabled by.

Co-authored-by: Qyriad <qyriad@qyriad.me>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
 2024-05-29 07:50:51 +00:00
Eelco Dolstra
2c88930ef2 AttrCursor: Remove forceErrors 
Instead, force evaluation of the original value only if we need to
show the exception to the user.
 2024-05-24 16:34:57 +02:00