max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2024-09-20 16:38:05 +03:00
Code Issues Projects Releases Packages Wiki Activity Actions
17543 commits 7 branches 0 tags 62 MiB
Commit graph

1974 commits

Author SHA1 Message Date
Eelco Dolstra
903acc7c0f
Merge pull request #10873 from siddhantk232/rm-createdirs 
use `std::filesystem::create_directories` for createDirs
 2024-06-24 14:54:37 +02:00
John Ericson
df068734ac
Merge pull request #10769 from poweredbypie/mingw-spawn 
Implement runProgram for Windows
 2024-06-23 14:12:36 -04:00
siddhantCodes
85b7989764 fix: handle errors in nix::createDirs 
the `std::filesystem::create_directories` can fail due to insufficient
permissions. We convert this error into a `SysError` and catch it
wherever required.
 2024-06-20 19:53:25 +05:30
PoweredByPie
8b81d083a7 Remove lookupPathForProgram and implement initial runProgram test 
Apparently, CreateProcessW already searches path, so manual path search
isn't really necessary.
 2024-06-18 01:01:52 -07:00
PoweredByPie
fcb92b4fa4 Fix DWORD vs. int comparison warning 2024-06-17 22:14:38 -07:00
PoweredByPie
4f6e3b9402 Implement tests for lookupPathForProgram and fix bugs caught by tests 2024-06-17 18:46:08 -07:00
PoweredByPie
4662e7d856 Implement windowsEscape 2024-06-17 14:57:57 -07:00
Tom Bereknyei
706edf26eb build: meson for libfetchers 2024-06-17 17:25:56 -04:00
PoweredByPie
b11cf8166f Format runProgram declaration 2024-06-17 13:12:28 -07:00
PoweredByPie
a58ca342ca Initial runProgram implementation for Windows 
This is incomplete; proper shell escaping needs to be done
 2024-06-17 11:13:22 -07:00
John Ericson
a1bb668ccb Merge remote-tracking branch 'upstream/master' into rm-createdirs 2024-06-17 12:57:54 -04:00
Eelco Dolstra
48d38b32d2
Merge pull request #10918 from andir/restrict-tarfile-formats 
Restrict supported tarball formats to actual Tarballs
 2024-06-17 13:20:23 +02:00
PoweredByPie
b0cfac8f93 Fix compile error on windows 2024-06-17 00:03:50 -07:00
Andreas Rammhold
5a9e1c0d20 Restrict supported tarball formats to actual Tarballs 
The documentation is clear about the supported formats (with at least
`builtins.fetchTarball`). The way the code was written previously it
supported all the formats that libarchive supported. That is a
surprisingly large amount of formats that are likely not on the radar
of the Nix developers and users. Before people end up relying on
this (or if they do) it is better to break it now before it becomes a
widespread "feature".

Zip file support has been retained as (at least to my knowledge)
historically that has been used to fetch nixpkgs in some shell
expressions *many* years back.

Fixes https://github.com/NixOS/nix/issues/10917
 2024-06-15 14:28:20 +02:00
John Ericson
ea8e49bea5 Force the cpuid option for libutil rather than relying on detection 
This is more robust, and match's Nixpkgs policy to force enable flags
statically by default (a common distro thing).
 2024-06-14 10:25:14 -04:00
John Ericson
2f5fdab06c
Merge pull request #10905 from obsidiansystems/platform-namespace 
Put some file descriptor functions in unix and windows namespaces
 2024-06-14 08:45:31 -04:00
John Ericson
56f0b5304f Document the nix-util Meson build system more extensively 
I hope this will make it easier to maintain, and also make it easier for
others to assist with porting the rest of the build system to Meson.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2024-06-13 11:39:59 -04:00
John Ericson
ff87c1a318 Put some file descriptor functions in unix and windows namespaces 
It is misleading when platform-specific functions are in the overall
`nix` namespace. More namespaces also makes for nicer doxygen.
 2024-06-13 11:18:59 -04:00
Robert Hensing
afdd12be5e
Merge pull request #10900 from obsidiansystems/hash-order 
hash: Compare hash algo second for back compat
 2024-06-13 01:35:31 +02:00
John Ericson
25a9894943 hash: Compare hash algo second for back compat 
Previously (in cfc18a7739), we forgot to
compare the algo at all. This means we keep the same ordering as before
by making the stuff we always have compared take priority.
 2024-06-12 18:35:47 -04:00
John Ericson
28d2af4ea6 Build nix-util with Meson 
The idea is two-fold:

- Replace autotools with Meson

- Build each library in its own derivation

The interaction of these two features is that Meson's "subprojects"
feature (https://mesonbuild.com/Subprojects) allows us to have single
dev shell for building all libraries still, while also building things
separately. This allows us to break up the build without a huge
productivity lost.

I tested the Linux native build, and NetBSD and Windows cross builds.

Also do some clean ups of the Flake in the process of supporting new
jobs.

Special thanks to everyone that has worked on a Meson port so far,
@p01arst0rm and @Qyriad in particular.

Co-Authored-By: p01arst0rm <polar@ever3st.com>
Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Artemis Tosini <me@artem.ist>
Co-Authored-By: Felix Uhl <felix.uhl@outlook.com>
Co-Authored-By: Jade Lovelace <lix@jade.fyi>
Co-Authored-By: Lunaphied <lunaphied@lunaphied.me>
Co-Authored-By: Maximilian Bosch <maximilian@mbosch.me>
Co-Authored-By: Pierre Bourdon <delroth@gmail.com>
Co-Authored-By: Qyriad <qyriad@qyriad.me>
Co-Authored-By: Rebecca Turner <rbt@sent.as>
Co-Authored-By: Winter <winter@winter.cafe>
Co-Authored-By: eldritch horrors <pennae@lix.systems>
Co-Authored-By: jade <lix@jade.fyi>
Co-Authored-By: julia <midnight@trainwit.ch>
Co-Authored-By: rebecca “wiggles” turner <rbt@sent.as>
Co-Authored-By: wiggles dog <rbt@sent.as>
Co-Authored-By: fricklerhandwerk <valentin@fricklerhandwerk.de>
Co-authored-By: Eli Schwartz <eschwartz93@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2024-06-12 18:31:02 -04:00
siddhantCodes
25b0242ca6 std::filesystem::create_directories for createDirs 
The implementation of `nix::createDirs` allows it to be a simple wrapper
around `std::filesystem::create_directories` as its return value is not
used anywhere.
 2024-06-09 19:49:39 +05:30
Eelco Dolstra
fd9e49480a PosixSourceAccessor: Use SharedSync 2024-06-06 10:44:53 +02:00
Eelco Dolstra
2f39caf180 Sync: Add support for shared locks 2024-06-06 10:44:09 +02:00
eldritch horrors
e291087747 libutil: guard Finally against invalid exception throws 
throwing exceptions is fine, but throwing exceptions during exception
handling is hard enough to do correctly that we should just forbid it
entirely out of an overabundance of caution. in cases where terminate
is the correct answer the users of Finally must call it manually now.

Source: 6c777476c9
 2024-06-05 22:45:37 +02:00
Eelco Dolstra
ef140c25d1
Merge pull request #10852 from siddhantk232/add-interrupts 
add call to `checkInterrupt` in a bunch of places
 2024-06-04 17:21:03 +02:00
siddhantCodes
8f1a26667e add call to checkInterrupt in a bunch of places 
This brings back the old behaviour. We check for interrupts in places
that may iterate over wide directories.
 2024-06-04 19:35:40 +05:30
John Ericson
06be6812a6 Create and install a nix-util.pc 
Before, `-lnixutil` was just stuck in `nix-store.pc`, but that doesn't
seem so nice.

This prepares us to distribute `libnixutil` in a separate package if we
want, but it should be a good change either way. I suspect it wasn't
done before because libutil was an extra unstable interface, but I don't
think we need worry about that. *All* the C++ is less stable than the C
(or that's the goal at least).

For what it's worth, Lix also created this pkg-config file *en passant*
during their rename:
c97e17144e (diff-3c4f60cc44a0e35444c7f45331cfa50f76637118)
 2024-06-03 14:14:40 -04:00
Eelco Dolstra
da92ad7dd2
Merge pull request #10592 from hercules-ci/builtins-warn 
Add `builtins.warn`
 2024-06-03 17:16:32 +02:00
Robert Hensing
70b1036224 builtins.warn: Use new EvalBaseError + "evaluation warning" 2024-06-03 16:24:21 +02:00
Eelco Dolstra
54a9fbe5d6 Merge remote-tracking branch 'origin/master' into large-path-warning 2024-06-03 16:17:52 +02:00
Eelco Dolstra
eb0d46fab6
Merge pull request #9897 from bryango/fix-submodule-subdir 
libutil/url: fix git+file:./ parse error
 2024-06-03 16:04:41 +02:00
Eelco Dolstra
7f5b57d18f Merge remote-tracking branch 'origin/master' into large-path-warning 2024-06-03 15:32:27 +02:00
John Ericson
cfc18a7739 Modernize Hash ordering with C++20 <=> 
Progress on #10832

This doesn't switch to auto-deriving the fields, but by defining `<=>`
we allow deriving `<=>` in downstream types where `Hash` is used.
 2024-06-03 09:24:33 -04:00
John Ericson
84c65135a5 ValidPathInfo JSON format should use null not omit field 
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2024-06-03 08:21:22 -04:00
Jade Lovelace
473d2d56fc Remove 100s of CPU time (10%) from build times (1465s -> 1302s) 
Result's from Mic92's framework 13th Gen Intel Core i7-1360P:

Before: 3595.92s user 183.01s system 1360% cpu 4:37.74 total
After: 3486.07s user 168.93s system 1354% cpu 4:29.79 total

I saw that boost/lexical_cast was costing about 100s in CPU time on our
compiles. We can fix this trivially by doing explicit template
instantiation in exactly one place and eliminating all other includes of
it, which is a code improvement anyway by hiding the boost.

Before:
```
lix/lix2 » ClangBuildAnalyzer --analyze buildtimeold.bin
Analyzing build trace from 'buildtimeold.bin'...
**** Time summary:
Compilation (551 times):
  Parsing (frontend):         1465.3 s
  Codegen & opts (backend):   1110.9 s

<snip>

**** Expensive headers:
178153 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 3426 ms), included via:
  40x: command.hh
  5x: command-installable-value.hh
  3x: installable-flake.hh
  2x: <direct include>
  2x: installable-attr-path.hh

176217 ms: ../src/libutil/error.hh (included 246 times, avg 716 ms), included via:
  36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh
  12x: globals.hh config.hh experimental-features.hh
  11x: file-system.hh file-descriptor.hh
  6x: serialise.hh strings.hh
  6x: <direct include>
  6x: archive.hh serialise.hh strings.hh
  ...

173243 ms: ../src/libstore/store-api.hh (included 152 times, avg 1139 ms), included via:
  55x: <direct include>
  39x: command.hh installable-value.hh installables.hh
  7x: libexpr.hh
  4x: local-store.hh
  4x: command-installable-value.hh installable-value.hh installables.hh
  3x: binary-cache-store.hh
  ...

170482 ms: ../src/libutil/serialise.hh (included 201 times, avg 848 ms), included via:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh
  14x: store-api.hh nar-info.hh hash.hh
  11x: <direct include>
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh
  6x: fetchers.hh hash.hh
  ...

169397 ms: ../src/libcmd/installables.hh (included 53 times, avg 3196 ms), included via:
  40x: command.hh installable-value.hh
  5x: command-installable-value.hh installable-value.hh
  3x: installable-flake.hh installable-value.hh
  2x: <direct include>
  1x: installable-derived-path.hh
  1x: installable-value.hh
  ...

159740 ms: ../src/libutil/strings.hh (included 221 times, avg 722 ms), included via:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh
  19x: <direct include>
  14x: store-api.hh nar-info.hh hash.hh serialise.hh
  11x: serialise.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh
  ...

156796 ms: ../src/libcmd/command.hh (included 51 times, avg 3074 ms), included via:
  42x: <direct include>
  7x: command-installable-value.hh
  2x: installable-attr-path.hh

150392 ms: ../src/libutil/types.hh (included 251 times, avg 599 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh
  11x: file-system.hh
  10x: globals.hh
  6x: fetchers.hh
  6x: serialise.hh strings.hh error.hh
  5x: archive.hh
  ...

133101 ms: /nix/store/644b90j1vms44nr18yw3520pzkrg4dd1-boost-1.81.0-dev/include/boost/lexical_cast.hpp (included 226 times, avg 588 ms), included via
:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh strings.hh
  19x: file-system.hh
  11x: store-api.hh nar-info.hh hash.hh serialise.hh strings.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  6x: eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  ...

132887 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 507 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh
  16x: gtest.h
  11x: file-system.hh types.hh ref.hh
  10x: globals.hh types.hh ref.hh
  10x: json.hpp
  6x: serialise.hh
  ...

  done in 0.6s.
```

After:
```
lix/lix2 » maintainers/buildtime_report.sh build
Processing all files and saving to '/home/jade/lix/lix2/maintainers/../buildtime.bin'...
  done in 0.6s. Run 'ClangBuildAnalyzer --analyze /home/jade/lix/lix2/maintainers/../buildtime.bin' to analyze it.
Analyzing build trace from '/home/jade/lix/lix2/maintainers/../buildtime.bin'...
**** Time summary:
Compilation (551 times):
  Parsing (frontend):         1302.1 s
  Codegen & opts (backend):    956.3 s

<snip>

**** Expensive headers:
178145 ms: ../src/libutil/error.hh (included 246 times, avg 724 ms), included via:
  36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh
  12x: globals.hh config.hh experimental-features.hh
  11x: file-system.hh file-descriptor.hh
  6x: <direct include>
  6x: serialise.hh strings.hh
  6x: fetchers.hh hash.hh serialise.hh strings.hh
  ...

154043 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 2962 ms), included via:
  40x: command.hh
  5x: command-installable-value.hh
  3x: installable-flake.hh
  2x: <direct include>
  2x: installable-attr-path.hh

153593 ms: ../src/libstore/store-api.hh (included 152 times, avg 1010 ms), included via:
  55x: <direct include>
  39x: command.hh installable-value.hh installables.hh
  7x: libexpr.hh
  4x: local-store.hh
  4x: command-installable-value.hh installable-value.hh installables.hh
  3x: binary-cache-store.hh
  ...

149948 ms: ../src/libutil/types.hh (included 251 times, avg 597 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh
  11x: file-system.hh
  10x: globals.hh
  6x: fetchers.hh
  6x: serialise.hh strings.hh error.hh
  5x: archive.hh
  ...

144560 ms: ../src/libcmd/installables.hh (included 53 times, avg 2727 ms), included via:
  40x: command.hh installable-value.hh
  5x: command-installable-value.hh installable-value.hh
  3x: installable-flake.hh installable-value.hh
  2x: <direct include>
  1x: installable-value.hh
  1x: installable-derived-path.hh
  ...

136585 ms: ../src/libcmd/command.hh (included 51 times, avg 2678 ms), included via:
  42x: <direct include>
  7x: command-installable-value.hh
  2x: installable-attr-path.hh

133394 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 509 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh
  16x: gtest.h
  11x: file-system.hh types.hh ref.hh
  10x: globals.hh types.hh ref.hh
  10x: json.hpp
  6x: serialise.hh
  ...

89315 ms: ../src/libstore/derived-path.hh (included 178 times, avg 501 ms), included via:
  37x: command.hh installable-value.hh installables.hh
  25x: store-api.hh realisation.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh context.hh
  6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh
  6x: libexpr.hh value.hh context.hh
  6x: shared.hh
  ...

87347 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/ostream (included 273 times, avg 319 ms), included via:
  35x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh memory unique_ptr.h
  12x: regex sstream istream
  10x: file-system.hh types.hh ref.hh memory unique_ptr.h
  10x: gtest.h memory unique_ptr.h
  10x: globals.hh types.hh ref.hh memory unique_ptr.h
  6x: fetchers.hh types.hh ref.hh memory unique_ptr.h
  ...

85249 ms: ../src/libutil/config.hh (included 213 times, avg 400 ms), included via:
  37x: command.hh installable-value.hh installables.hh derived-path.hh
  20x: globals.hh
  20x: logging.hh
  16x: store-api.hh logging.hh
  6x: <direct include>
  6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh derived-path.hh
  ...

  done in 0.5s.
```

Adapated from 18aa3e1d57
 2024-05-31 13:00:09 +02:00
John Ericson
bcdee80a0d More work on the scheduler for windows 
- Get a rump derivation goal: hook instance will come later, local
  derivation goal will come after that.

- Start cleaning up the channel / waiting code with an abstraction.
 2024-05-28 11:39:49 -04:00
John Ericson
1d5d748fe4 Fix format 
39b2a399ad passed CI but was landed after
the formatting change in 1d6c2316a9.
 2024-05-27 22:32:52 -04:00
John Ericson
d0c7da131f
Merge pull request #10678 from nix-windows/windows-substitution-goal 
Start building the scheduler for Windows
 2024-05-27 17:47:29 -04:00
John Ericson
aa4a2927a7
Merge pull request #10778 from Mic92/safer-tab-completion 
repl: do not crash when tab-completing import errors
 2024-05-27 09:54:29 -04:00
Jörg Thalheim
f97da4b11c libutil/source-accessor: custom error if source does not exist 
This allows better error handling by catching this error in particular.
 2024-05-25 23:06:57 +02:00
eldritch horrors
2de4589e46 libstore: remove unused copyPath function 
Source: 47523944c5

Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
 2024-05-25 22:28:41 +02:00
John Ericson
5cfa75ea16
Merge pull request #10737 from poweredbypie/mingw-stackSize 
Implement `setStackSize` on Windows
 2024-05-25 09:56:02 -04:00
PoweredByPie
5f68e6d69f Get max stack size in setStackSize to match Linux 2024-05-23 03:54:35 -07:00
John Ericson
4a19f4a866
Merge pull request #9839 from obsidiansystems/more-machine-cleanup 
Create `StoreReference` and use it in `Machine`
 2024-05-22 17:01:57 -04:00
John Ericson
dc7615dbbb tryUnshareFilesystem: Ignore ENOSYS too 
Fixes #10747
 2024-05-22 16:07:38 -04:00
John Ericson
d5fdfdc592 unshareFilesystem: Do not assume caller 2024-05-22 16:07:34 -04:00
John Ericson
b59a7a14c4 Add StoreReference::render 
This will be needed for the next step.

Also allows us to write round trip tests.
 2024-05-22 09:20:15 -04:00
John Ericson
1d6c2316a9 Slightly change formatting style 
For long expressions, one argument or parameter per line is just easier.
 2024-05-22 09:20:15 -04:00
Robert Hensing
e4be8abe42
Merge pull request #10701 from cole-h/nix-command-warn-unknown-settings 
Warn on unknown settings when the first positional is an argument
 2024-05-20 15:15:13 +02:00