max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2025-01-31 15:36:47 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
17474 commits 7 branches 0 tags 64 MiB
Commit graph

1952 commits

Author SHA1 Message Date
Eelco Dolstra
fd9e49480a PosixSourceAccessor: Use SharedSync 2024-06-06 10:44:53 +02:00
Eelco Dolstra
2f39caf180 Sync: Add support for shared locks 2024-06-06 10:44:09 +02:00
eldritch horrors
e291087747 libutil: guard Finally against invalid exception throws 
throwing exceptions is fine, but throwing exceptions during exception
handling is hard enough to do correctly that we should just forbid it
entirely out of an overabundance of caution. in cases where terminate
is the correct answer the users of Finally must call it manually now.

Source: 6c777476c9
 2024-06-05 22:45:37 +02:00
Eelco Dolstra
ef140c25d1
Merge pull request #10852 from siddhantk232/add-interrupts 
add call to `checkInterrupt` in a bunch of places
 2024-06-04 17:21:03 +02:00
siddhantCodes
8f1a26667e add call to checkInterrupt in a bunch of places 
This brings back the old behaviour. We check for interrupts in places
that may iterate over wide directories.
 2024-06-04 19:35:40 +05:30
John Ericson
06be6812a6 Create and install a nix-util.pc 
Before, `-lnixutil` was just stuck in `nix-store.pc`, but that doesn't
seem so nice.

This prepares us to distribute `libnixutil` in a separate package if we
want, but it should be a good change either way. I suspect it wasn't
done before because libutil was an extra unstable interface, but I don't
think we need worry about that. *All* the C++ is less stable than the C
(or that's the goal at least).

For what it's worth, Lix also created this pkg-config file *en passant*
during their rename:
c97e17144e (diff-3c4f60cc44a0e35444c7f45331cfa50f76637118)
 2024-06-03 14:14:40 -04:00
Eelco Dolstra
da92ad7dd2
Merge pull request #10592 from hercules-ci/builtins-warn 
Add `builtins.warn`
 2024-06-03 17:16:32 +02:00
Robert Hensing
70b1036224 builtins.warn: Use new EvalBaseError + "evaluation warning" 2024-06-03 16:24:21 +02:00
Eelco Dolstra
54a9fbe5d6 Merge remote-tracking branch 'origin/master' into large-path-warning 2024-06-03 16:17:52 +02:00
Eelco Dolstra
eb0d46fab6
Merge pull request #9897 from bryango/fix-submodule-subdir 
libutil/url: fix git+file:./ parse error
 2024-06-03 16:04:41 +02:00
Eelco Dolstra
7f5b57d18f Merge remote-tracking branch 'origin/master' into large-path-warning 2024-06-03 15:32:27 +02:00
John Ericson
cfc18a7739 Modernize Hash ordering with C++20 <=> 
Progress on #10832

This doesn't switch to auto-deriving the fields, but by defining `<=>`
we allow deriving `<=>` in downstream types where `Hash` is used.
 2024-06-03 09:24:33 -04:00
John Ericson
84c65135a5 ValidPathInfo JSON format should use null not omit field 
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2024-06-03 08:21:22 -04:00
Jade Lovelace
473d2d56fc Remove 100s of CPU time (10%) from build times (1465s -> 1302s) 
Result's from Mic92's framework 13th Gen Intel Core i7-1360P:

Before: 3595.92s user 183.01s system 1360% cpu 4:37.74 total
After: 3486.07s user 168.93s system 1354% cpu 4:29.79 total

I saw that boost/lexical_cast was costing about 100s in CPU time on our
compiles. We can fix this trivially by doing explicit template
instantiation in exactly one place and eliminating all other includes of
it, which is a code improvement anyway by hiding the boost.

Before:
```
lix/lix2 » ClangBuildAnalyzer --analyze buildtimeold.bin
Analyzing build trace from 'buildtimeold.bin'...
**** Time summary:
Compilation (551 times):
  Parsing (frontend):         1465.3 s
  Codegen & opts (backend):   1110.9 s

<snip>

**** Expensive headers:
178153 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 3426 ms), included via:
  40x: command.hh
  5x: command-installable-value.hh
  3x: installable-flake.hh
  2x: <direct include>
  2x: installable-attr-path.hh

176217 ms: ../src/libutil/error.hh (included 246 times, avg 716 ms), included via:
  36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh
  12x: globals.hh config.hh experimental-features.hh
  11x: file-system.hh file-descriptor.hh
  6x: serialise.hh strings.hh
  6x: <direct include>
  6x: archive.hh serialise.hh strings.hh
  ...

173243 ms: ../src/libstore/store-api.hh (included 152 times, avg 1139 ms), included via:
  55x: <direct include>
  39x: command.hh installable-value.hh installables.hh
  7x: libexpr.hh
  4x: local-store.hh
  4x: command-installable-value.hh installable-value.hh installables.hh
  3x: binary-cache-store.hh
  ...

170482 ms: ../src/libutil/serialise.hh (included 201 times, avg 848 ms), included via:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh
  14x: store-api.hh nar-info.hh hash.hh
  11x: <direct include>
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh
  6x: fetchers.hh hash.hh
  ...

169397 ms: ../src/libcmd/installables.hh (included 53 times, avg 3196 ms), included via:
  40x: command.hh installable-value.hh
  5x: command-installable-value.hh installable-value.hh
  3x: installable-flake.hh installable-value.hh
  2x: <direct include>
  1x: installable-derived-path.hh
  1x: installable-value.hh
  ...

159740 ms: ../src/libutil/strings.hh (included 221 times, avg 722 ms), included via:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh
  19x: <direct include>
  14x: store-api.hh nar-info.hh hash.hh serialise.hh
  11x: serialise.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh
  ...

156796 ms: ../src/libcmd/command.hh (included 51 times, avg 3074 ms), included via:
  42x: <direct include>
  7x: command-installable-value.hh
  2x: installable-attr-path.hh

150392 ms: ../src/libutil/types.hh (included 251 times, avg 599 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh
  11x: file-system.hh
  10x: globals.hh
  6x: fetchers.hh
  6x: serialise.hh strings.hh error.hh
  5x: archive.hh
  ...

133101 ms: /nix/store/644b90j1vms44nr18yw3520pzkrg4dd1-boost-1.81.0-dev/include/boost/lexical_cast.hpp (included 226 times, avg 588 ms), included via
:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh strings.hh
  19x: file-system.hh
  11x: store-api.hh nar-info.hh hash.hh serialise.hh strings.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  6x: eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  ...

132887 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 507 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh
  16x: gtest.h
  11x: file-system.hh types.hh ref.hh
  10x: globals.hh types.hh ref.hh
  10x: json.hpp
  6x: serialise.hh
  ...

  done in 0.6s.
```

After:
```
lix/lix2 » maintainers/buildtime_report.sh build
Processing all files and saving to '/home/jade/lix/lix2/maintainers/../buildtime.bin'...
  done in 0.6s. Run 'ClangBuildAnalyzer --analyze /home/jade/lix/lix2/maintainers/../buildtime.bin' to analyze it.
Analyzing build trace from '/home/jade/lix/lix2/maintainers/../buildtime.bin'...
**** Time summary:
Compilation (551 times):
  Parsing (frontend):         1302.1 s
  Codegen & opts (backend):    956.3 s

<snip>

**** Expensive headers:
178145 ms: ../src/libutil/error.hh (included 246 times, avg 724 ms), included via:
  36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh
  12x: globals.hh config.hh experimental-features.hh
  11x: file-system.hh file-descriptor.hh
  6x: <direct include>
  6x: serialise.hh strings.hh
  6x: fetchers.hh hash.hh serialise.hh strings.hh
  ...

154043 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 2962 ms), included via:
  40x: command.hh
  5x: command-installable-value.hh
  3x: installable-flake.hh
  2x: <direct include>
  2x: installable-attr-path.hh

153593 ms: ../src/libstore/store-api.hh (included 152 times, avg 1010 ms), included via:
  55x: <direct include>
  39x: command.hh installable-value.hh installables.hh
  7x: libexpr.hh
  4x: local-store.hh
  4x: command-installable-value.hh installable-value.hh installables.hh
  3x: binary-cache-store.hh
  ...

149948 ms: ../src/libutil/types.hh (included 251 times, avg 597 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh
  11x: file-system.hh
  10x: globals.hh
  6x: fetchers.hh
  6x: serialise.hh strings.hh error.hh
  5x: archive.hh
  ...

144560 ms: ../src/libcmd/installables.hh (included 53 times, avg 2727 ms), included via:
  40x: command.hh installable-value.hh
  5x: command-installable-value.hh installable-value.hh
  3x: installable-flake.hh installable-value.hh
  2x: <direct include>
  1x: installable-value.hh
  1x: installable-derived-path.hh
  ...

136585 ms: ../src/libcmd/command.hh (included 51 times, avg 2678 ms), included via:
  42x: <direct include>
  7x: command-installable-value.hh
  2x: installable-attr-path.hh

133394 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 509 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh
  16x: gtest.h
  11x: file-system.hh types.hh ref.hh
  10x: globals.hh types.hh ref.hh
  10x: json.hpp
  6x: serialise.hh
  ...

89315 ms: ../src/libstore/derived-path.hh (included 178 times, avg 501 ms), included via:
  37x: command.hh installable-value.hh installables.hh
  25x: store-api.hh realisation.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh context.hh
  6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh
  6x: libexpr.hh value.hh context.hh
  6x: shared.hh
  ...

87347 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/ostream (included 273 times, avg 319 ms), included via:
  35x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh memory unique_ptr.h
  12x: regex sstream istream
  10x: file-system.hh types.hh ref.hh memory unique_ptr.h
  10x: gtest.h memory unique_ptr.h
  10x: globals.hh types.hh ref.hh memory unique_ptr.h
  6x: fetchers.hh types.hh ref.hh memory unique_ptr.h
  ...

85249 ms: ../src/libutil/config.hh (included 213 times, avg 400 ms), included via:
  37x: command.hh installable-value.hh installables.hh derived-path.hh
  20x: globals.hh
  20x: logging.hh
  16x: store-api.hh logging.hh
  6x: <direct include>
  6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh derived-path.hh
  ...

  done in 0.5s.
```

Adapated from 18aa3e1d57
 2024-05-31 13:00:09 +02:00
John Ericson
bcdee80a0d More work on the scheduler for windows 
- Get a rump derivation goal: hook instance will come later, local
  derivation goal will come after that.

- Start cleaning up the channel / waiting code with an abstraction.
 2024-05-28 11:39:49 -04:00
John Ericson
1d5d748fe4 Fix format 
39b2a399ad passed CI but was landed after
the formatting change in 1d6c2316a9.
 2024-05-27 22:32:52 -04:00
John Ericson
d0c7da131f
Merge pull request #10678 from nix-windows/windows-substitution-goal 
Start building the scheduler for Windows
 2024-05-27 17:47:29 -04:00
John Ericson
aa4a2927a7
Merge pull request #10778 from Mic92/safer-tab-completion 
repl: do not crash when tab-completing import errors
 2024-05-27 09:54:29 -04:00
Jörg Thalheim
f97da4b11c libutil/source-accessor: custom error if source does not exist 
This allows better error handling by catching this error in particular.
 2024-05-25 23:06:57 +02:00
eldritch horrors
2de4589e46 libstore: remove unused copyPath function 
Source: 47523944c5

Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
 2024-05-25 22:28:41 +02:00
John Ericson
5cfa75ea16
Merge pull request #10737 from poweredbypie/mingw-stackSize 
Implement `setStackSize` on Windows
 2024-05-25 09:56:02 -04:00
PoweredByPie
5f68e6d69f Get max stack size in setStackSize to match Linux 2024-05-23 03:54:35 -07:00
John Ericson
4a19f4a866
Merge pull request #9839 from obsidiansystems/more-machine-cleanup 
Create `StoreReference` and use it in `Machine`
 2024-05-22 17:01:57 -04:00
John Ericson
dc7615dbbb tryUnshareFilesystem: Ignore ENOSYS too 
Fixes #10747
 2024-05-22 16:07:38 -04:00
John Ericson
d5fdfdc592 unshareFilesystem: Do not assume caller 2024-05-22 16:07:34 -04:00
John Ericson
b59a7a14c4 Add StoreReference::render 
This will be needed for the next step.

Also allows us to write round trip tests.
 2024-05-22 09:20:15 -04:00
John Ericson
1d6c2316a9 Slightly change formatting style 
For long expressions, one argument or parameter per line is just easier.
 2024-05-22 09:20:15 -04:00
Robert Hensing
e4be8abe42
Merge pull request #10701 from cole-h/nix-command-warn-unknown-settings 
Warn on unknown settings when the first positional is an argument
 2024-05-20 15:15:13 +02:00
John Ericson
d48bbda2e7
Update the updateWindowSize documentation 2024-05-20 08:34:49 -04:00
PoweredByPie
6a3f906382 Implement setStackSize for Windows 2024-05-18 16:19:01 -07:00
PoweredByPie
e42d00c961 Change rlim_t to size_t in setStackSize in preparation of Windows impl 2024-05-18 16:18:50 -07:00
PoweredByPie
53f0c44d6c Implement updateWindowSize for Windows 2024-05-18 16:14:20 -07:00
Qyriad
52a16b7e59 add and fix -Wdeprecated-copy 
*so* many warnings, from only two definitions

Change-Id: If2561cd500c05a1e33cce984faf9f3e42a8a95ac
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
 2024-05-17 19:25:35 +02:00
Robert Hensing
303268bb71
Merge pull request #10479 from obsidiansystems/ca-fso-docs 
Document file system object content addressing
 2024-05-15 22:52:53 +02:00
John Ericson
043135a848 Document file system object content addressing 
In addition:

- Take the opportunity to add a bunch more missing hyperlinks, too.

- Remove some glossary entries that are now subsumed by dedicated pages.
  We used to not be able to do this without breaking link fragments, but
  now we can, so pick up where we left off.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2024-05-15 16:28:48 -04:00
Cole Helbling
39a269657e libutil/args: warn on unknown settings after parsing all flags 2024-05-15 12:25:03 -07:00
siddhantCodes
39e8aad446 Merge branch 'master' of github.com:NixOS/nix 2024-05-13 18:43:12 +05:30
siddhantCodes
62e1ea2f4b use path for from arg in nix::copyFile 2024-05-13 16:10:21 +05:30
Eelco Dolstra
5534682166 Update src/libutil/util.hh 
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2024-05-13 12:14:43 +02:00
Eelco Dolstra
cf3b044b7e Make large path warnings human-readable 2024-05-13 11:52:38 +02:00
Eelco Dolstra
5314430437 Move printSize() into libutil 
Also always include the unit (i.e. "MiB" instead of "M").
 2024-05-13 11:52:38 +02:00
Eelco Dolstra
dbe1b51580 Add setting to warn about copying/hashing large paths 
This is useful for diagnosing whether an evaluation is copying large
paths to the store. Example:

   $ nix build .#packages.x86_64-linux.default --large-path-warning-threshold 1000000
   warning: copied large path '/home/eelco/Dev/nix-master/' to the store (6271792 bytes)
   warning: copied large path '«github:NixOS/nixpkgs/b550fe4b4776908ac2a861124307045f8e717c8e?narHash=sha256-7kkJQd4rZ%2BvFrzWu8sTRtta5D1kBG0LSRYAfhtmMlSo%3D»/' to the store (155263768 bytes)
   warning: copied large path '«github:libgit2/libgit2/45fd9ed7ae1a9b74b957ef4f337bc3c8b3df01b5?narHash=sha256-oX4Z3S9WtJlwvj0uH9HlYcWv%2Bx1hqp8mhXl7HsLu2f0%3D»/' to the store (22175416 bytes)
   warning: copied large path '/nix/store/z985088mcd6w23qwdlirsinnyzayagki-source' to the store (5885872 bytes)
 2024-05-13 11:52:27 +02:00
siddhantCodes
ccf94545db rename copy -> copyFile and remove old copyFile 
the old `copyFile` was just a wrapper that was calling the `copy`
function. This wrapper function is removed and the `copy` function is
renamed to `copyFile`.
 2024-05-12 19:20:17 +05:30
siddhantCodes
d3b7367c80 inline usage of nix::getFileType and remove it 2024-05-12 18:58:05 +05:30
siddhantCodes
4537663740 inline the usage of nix::renameFile 
use `std::filesystem::rename` everywhere and remove `nix::renameFile`
 2024-05-12 18:40:16 +05:30
siddhantCodes
1db7d1b840 inline the usage of nix::readDirectory 
`nix::readDirectory` is removed. `std::filesystem::directory_iterator`
is used directly in places that used this util.
 2024-05-12 17:42:18 +05:30
John Ericson
39b2a399ad Start building the scheduler for Windows 
Building derivations is a lot harder, but the downloading goals is
portable enough.

The "common channel" code is due to Volth. I wonder if there is a way we
can factor it out into separate functions / files to avoid some
within-function CPP.

Co-authored-by: volth <volth@volth.com>
 2024-05-10 20:23:59 -04:00
John Ericson
e0ff8da9d5 Build the local store on Windows 
Fixes #10558

Co-Authored-By: Eugene Butler <eugene@eugene4.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
 2024-05-10 13:05:23 -04:00
Eelco Dolstra
77a406a5a6 Fix warning 2024-05-08 21:16:53 +02:00
Eelco Dolstra
79c7d6205c Support unit prefixes in configuration settings 
E.g. you can now say `--min-free 1G`.
 2024-05-08 21:11:09 +02:00