max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2024-11-16 03:06:17 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
17671 commits 7 branches 0 tags 64 MiB
Commit graph

1937 commits

Author SHA1 Message Date
Jörg Thalheim
8cea1fbd97 src/nix/prefetch: fix prefetch containing current directory instead of tarball 
When --unpack was used the nix would add the current directory to the
nix store instead of the content of unpacked.
The reason for this is that std::distance already consumes the iterator.
To fix this we re-instantiate the directory iterator in case the
directory only contains a single entry.
 2024-07-05 19:18:49 +02:00
Eelco Dolstra
e1b6b3ce27
Merge pull request #11020 from DeterminateSystems/fix-tarball-caching 
Tarball fetcher: Fix fetchToStore() and eval caching
 2024-07-05 16:30:12 +02:00
kn
10ccdb7a41 Use proper struct sockpeercred for SO_PEERCRED for OpenBSD 
getsockopt(2) documents this;  ucred is wrong ("cr_" member prefix, no pid).
 2024-07-03 11:16:39 -04:00
John Ericson
a09360400b Ident some CPP in nix daemon 
Makes it easier for me to read.
 2024-07-03 11:15:56 -04:00
Eelco Dolstra
1ff186fc6e nix flake metadata: Show flake fingerprint 
This is useful for testing/debugging and maybe for sharing eval caches
(since it tells you what file in ~/.cache/nix/eval-cache-v5 to copy).
 2024-07-03 17:00:30 +02:00
John Ericson
0084a486cc Split out a new libnixflake 
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2024-06-26 19:56:21 -04:00
Robert Hensing
c66f1e7660
Merge pull request #10913 from NixOS/no-global-eval-settings-in-libexpr 
No global eval settings in `libnixexpr`
 2024-06-24 18:52:19 +02:00
John Ericson
fda4c78921
Merge pull request #10951 from obsidiansystems/load-just-one-config 
Small global config refactors
 2024-06-24 12:38:04 -04:00
John Ericson
52bfccf8d8 No global eval settings in libnixexpr 
Progress on #5638

There is still a global eval settings, but it pushed down into
`libnixcmd`, which is a lot less bad a place for this sort of thing.
 2024-06-24 12:15:16 -04:00
John Ericson
1620ad4587 Split out GlobalConfig into its own header 
This makes it easier to understand the reach of global variables /
global state in the config system.
 2024-06-24 11:36:21 -04:00
John Ericson
b51e161af5 Cleanup ContentAddressMethod to match docs 
The old `std::variant` is bad because we aren't adding a new case to
`FileIngestionMethod` so much as we are defining a separate concept ---
store object content addressing rather than file system object content
addressing. As such, it is more correct to just create a fresh
enumeration.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2024-06-24 10:24:06 -04:00
John Ericson
64e599ebe1 Rename Recursive -> NixArchive 
For enums:

- `FileIngestionMethod`

- `FileSerialisationMethod`
 2024-06-24 10:24:06 -04:00
John Ericson
ff87c1a318 Put some file descriptor functions in unix and windows namespaces 
It is misleading when platform-specific functions are in the overall
`nix` namespace. More namespaces also makes for nicer doxygen.
 2024-06-13 11:18:59 -04:00
Enno Richter
80ba7778e7 flake check: Recognize well known homeModule/homeModules attributes 2024-06-05 07:38:20 +02:00
Eelco Dolstra
ef140c25d1
Merge pull request #10852 from siddhantk232/add-interrupts 
add call to `checkInterrupt` in a bunch of places
 2024-06-04 17:21:03 +02:00
siddhantCodes
8f1a26667e add call to checkInterrupt in a bunch of places 
This brings back the old behaviour. We check for interrupts in places
that may iterate over wide directories.
 2024-06-04 19:35:40 +05:30
Eelco Dolstra
7f5b57d18f Merge remote-tracking branch 'origin/master' into large-path-warning 2024-06-03 15:32:27 +02:00
Eelco Dolstra
ecfad6a828
Merge pull request #10564 from edolstra/remove-forceErrors 
AttrCursor: Remove forceErrors
 2024-06-03 15:30:01 +02:00
Robert Hensing
84e116379c
Merge pull request #10812 from Mic92/build-perf 
Remove 100s of CPU time (10%) from build times (1465s -> 1302s)
 2024-05-31 13:28:24 +02:00
Jade Lovelace
473d2d56fc Remove 100s of CPU time (10%) from build times (1465s -> 1302s) 
Result's from Mic92's framework 13th Gen Intel Core i7-1360P:

Before: 3595.92s user 183.01s system 1360% cpu 4:37.74 total
After: 3486.07s user 168.93s system 1354% cpu 4:29.79 total

I saw that boost/lexical_cast was costing about 100s in CPU time on our
compiles. We can fix this trivially by doing explicit template
instantiation in exactly one place and eliminating all other includes of
it, which is a code improvement anyway by hiding the boost.

Before:
```
lix/lix2 » ClangBuildAnalyzer --analyze buildtimeold.bin
Analyzing build trace from 'buildtimeold.bin'...
**** Time summary:
Compilation (551 times):
  Parsing (frontend):         1465.3 s
  Codegen & opts (backend):   1110.9 s

<snip>

**** Expensive headers:
178153 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 3426 ms), included via:
  40x: command.hh
  5x: command-installable-value.hh
  3x: installable-flake.hh
  2x: <direct include>
  2x: installable-attr-path.hh

176217 ms: ../src/libutil/error.hh (included 246 times, avg 716 ms), included via:
  36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh
  12x: globals.hh config.hh experimental-features.hh
  11x: file-system.hh file-descriptor.hh
  6x: serialise.hh strings.hh
  6x: <direct include>
  6x: archive.hh serialise.hh strings.hh
  ...

173243 ms: ../src/libstore/store-api.hh (included 152 times, avg 1139 ms), included via:
  55x: <direct include>
  39x: command.hh installable-value.hh installables.hh
  7x: libexpr.hh
  4x: local-store.hh
  4x: command-installable-value.hh installable-value.hh installables.hh
  3x: binary-cache-store.hh
  ...

170482 ms: ../src/libutil/serialise.hh (included 201 times, avg 848 ms), included via:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh
  14x: store-api.hh nar-info.hh hash.hh
  11x: <direct include>
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh
  6x: fetchers.hh hash.hh
  ...

169397 ms: ../src/libcmd/installables.hh (included 53 times, avg 3196 ms), included via:
  40x: command.hh installable-value.hh
  5x: command-installable-value.hh installable-value.hh
  3x: installable-flake.hh installable-value.hh
  2x: <direct include>
  1x: installable-derived-path.hh
  1x: installable-value.hh
  ...

159740 ms: ../src/libutil/strings.hh (included 221 times, avg 722 ms), included via:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh
  19x: <direct include>
  14x: store-api.hh nar-info.hh hash.hh serialise.hh
  11x: serialise.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh
  ...

156796 ms: ../src/libcmd/command.hh (included 51 times, avg 3074 ms), included via:
  42x: <direct include>
  7x: command-installable-value.hh
  2x: installable-attr-path.hh

150392 ms: ../src/libutil/types.hh (included 251 times, avg 599 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh
  11x: file-system.hh
  10x: globals.hh
  6x: fetchers.hh
  6x: serialise.hh strings.hh error.hh
  5x: archive.hh
  ...

133101 ms: /nix/store/644b90j1vms44nr18yw3520pzkrg4dd1-boost-1.81.0-dev/include/boost/lexical_cast.hpp (included 226 times, avg 588 ms), included via
:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh strings.hh
  19x: file-system.hh
  11x: store-api.hh nar-info.hh hash.hh serialise.hh strings.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  6x: eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  ...

132887 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 507 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh
  16x: gtest.h
  11x: file-system.hh types.hh ref.hh
  10x: globals.hh types.hh ref.hh
  10x: json.hpp
  6x: serialise.hh
  ...

  done in 0.6s.
```

After:
```
lix/lix2 » maintainers/buildtime_report.sh build
Processing all files and saving to '/home/jade/lix/lix2/maintainers/../buildtime.bin'...
  done in 0.6s. Run 'ClangBuildAnalyzer --analyze /home/jade/lix/lix2/maintainers/../buildtime.bin' to analyze it.
Analyzing build trace from '/home/jade/lix/lix2/maintainers/../buildtime.bin'...
**** Time summary:
Compilation (551 times):
  Parsing (frontend):         1302.1 s
  Codegen & opts (backend):    956.3 s

<snip>

**** Expensive headers:
178145 ms: ../src/libutil/error.hh (included 246 times, avg 724 ms), included via:
  36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh
  12x: globals.hh config.hh experimental-features.hh
  11x: file-system.hh file-descriptor.hh
  6x: <direct include>
  6x: serialise.hh strings.hh
  6x: fetchers.hh hash.hh serialise.hh strings.hh
  ...

154043 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 2962 ms), included via:
  40x: command.hh
  5x: command-installable-value.hh
  3x: installable-flake.hh
  2x: <direct include>
  2x: installable-attr-path.hh

153593 ms: ../src/libstore/store-api.hh (included 152 times, avg 1010 ms), included via:
  55x: <direct include>
  39x: command.hh installable-value.hh installables.hh
  7x: libexpr.hh
  4x: local-store.hh
  4x: command-installable-value.hh installable-value.hh installables.hh
  3x: binary-cache-store.hh
  ...

149948 ms: ../src/libutil/types.hh (included 251 times, avg 597 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh
  11x: file-system.hh
  10x: globals.hh
  6x: fetchers.hh
  6x: serialise.hh strings.hh error.hh
  5x: archive.hh
  ...

144560 ms: ../src/libcmd/installables.hh (included 53 times, avg 2727 ms), included via:
  40x: command.hh installable-value.hh
  5x: command-installable-value.hh installable-value.hh
  3x: installable-flake.hh installable-value.hh
  2x: <direct include>
  1x: installable-value.hh
  1x: installable-derived-path.hh
  ...

136585 ms: ../src/libcmd/command.hh (included 51 times, avg 2678 ms), included via:
  42x: <direct include>
  7x: command-installable-value.hh
  2x: installable-attr-path.hh

133394 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 509 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh
  16x: gtest.h
  11x: file-system.hh types.hh ref.hh
  10x: globals.hh types.hh ref.hh
  10x: json.hpp
  6x: serialise.hh
  ...

89315 ms: ../src/libstore/derived-path.hh (included 178 times, avg 501 ms), included via:
  37x: command.hh installable-value.hh installables.hh
  25x: store-api.hh realisation.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh context.hh
  6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh
  6x: libexpr.hh value.hh context.hh
  6x: shared.hh
  ...

87347 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/ostream (included 273 times, avg 319 ms), included via:
  35x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh memory unique_ptr.h
  12x: regex sstream istream
  10x: file-system.hh types.hh ref.hh memory unique_ptr.h
  10x: gtest.h memory unique_ptr.h
  10x: globals.hh types.hh ref.hh memory unique_ptr.h
  6x: fetchers.hh types.hh ref.hh memory unique_ptr.h
  ...

85249 ms: ../src/libutil/config.hh (included 213 times, avg 400 ms), included via:
  37x: command.hh installable-value.hh installables.hh derived-path.hh
  20x: globals.hh
  20x: logging.hh
  16x: store-api.hh logging.hh
  6x: <direct include>
  6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh derived-path.hh
  ...

  done in 0.5s.
```

Adapated from 18aa3e1d57
 2024-05-31 13:00:09 +02:00
Robert Hensing
d93cc11491 Format 2024-05-30 19:41:58 +02:00
Robert Hensing
c692f6af13 nix env shell: Move from nix shell, add shorthand alias 2024-05-30 19:41:58 +02:00
Robert Hensing
98b85b2166 nix/main: Add AliasStatus::{Deprecated,AcceptedShorthand} 2024-05-30 19:41:58 +02:00
Martin Joerg
e7ea5591a2 fix typos 2024-05-27 15:56:52 +02:00
John Ericson
5cfa75ea16
Merge pull request #10737 from poweredbypie/mingw-stackSize 
Implement `setStackSize` on Windows
 2024-05-25 09:56:02 -04:00
Eelco Dolstra
eeb4c40867 Typo 2024-05-24 16:35:06 +02:00
Eelco Dolstra
2c88930ef2 AttrCursor: Remove forceErrors 
Instead, force evaluation of the original value only if we need to
show the exception to the user.
 2024-05-24 16:34:57 +02:00
PoweredByPie
0b7da099d1 Commit more stack size in some windows binaries 
This way we can commit the same amount of stack size (64 MB) without a conditional.
Includes nix, libnixexpr-tests, libnixfetchers-tests, libnixstore-tests, libnixutil-tests.
 2024-05-23 17:42:55 -07:00
John Ericson
f923ed6b6a Require drvPath attribute to end with .drv 
Fixes #4977
 2024-05-22 12:50:24 -04:00
Robert Hensing
20ed0c02b8
Merge pull request #10688 from hraban/fix/develop-aliases 
fix: don’t expand aliases in develop stdenv setup
 2024-05-20 15:59:21 +02:00
John Ericson
1c75af969a Document store object content addressing & improve JSON format 
The JSON format no longer uses the legacy ATerm `r:` prefixing nonsese,
but separate fields.

Progress on #9866

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2024-05-20 09:34:35 -04:00
PoweredByPie
a41f4223de Use setStackSize on Windows 2024-05-18 16:19:01 -07:00
Robert Hensing
303268bb71
Merge pull request #10479 from obsidiansystems/ca-fso-docs 
Document file system object content addressing
 2024-05-15 22:52:53 +02:00
John Ericson
043135a848 Document file system object content addressing 
In addition:

- Take the opportunity to add a bunch more missing hyperlinks, too.

- Remove some glossary entries that are now subsumed by dedicated pages.
  We used to not be able to do this without breaking link fragments, but
  now we can, so pick up where we left off.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2024-05-15 16:28:48 -04:00
Eelco Dolstra
39f7cbdc7c
Merge pull request #10691 from DeterminateSystems/commit-lock-file-summary 
Rename commit-lockfile-summary to commit-lock-file-summary for consistency
 2024-05-14 10:06:48 +02:00
Graham Christensen
e1e041ed8f Rename commit-lockfile-summary to commit-lock-file-summary for consistency 2024-05-13 09:23:59 -04:00
Eelco Dolstra
cf3b044b7e Make large path warnings human-readable 2024-05-13 11:52:38 +02:00
Eelco Dolstra
5314430437 Move printSize() into libutil 
Also always include the unit (i.e. "MiB" instead of "M").
 2024-05-13 11:52:38 +02:00
Hraban Luyat
6bf7edb18b fix: don’t expand aliases in develop stdenv setup 
This fixes https://github.com/NixOS/nixpkgs/pull/290775 by not expanding aliases
when sourcing the stdenv setup script. The way bash handles aliases is to expand
them when a function is defined, not when it is used. I.e.:

    $ alias echo="echo bar "
    $ echo foo
    bar foo
    $ xyzzy() { echo foo; }
    $ shopt -u expand_aliases
    $ xyzzy
    bar foo
    $ xyzzy2() { echo foo; }
    $ xyzzy2
    foo

The problem is that ~/.bashrc is sourced before the stdenv setup, and bashrc
commonly sets aliases for ‘cp’, ‘mv’ and ‘rm’ which you don’t want to take
effect in the stdenv derivation builders. The original commit introducing this
feature (5fd8cf7667) even mentioned this very
alias.

The only way to avoid this is to disable aliases entirely while sourcing the
stdenv setup, and reenable them afterwards.
 2024-05-12 21:57:55 -04:00
siddhantCodes
1db7d1b840 inline the usage of nix::readDirectory 
`nix::readDirectory` is removed. `std::filesystem::directory_iterator`
is used directly in places that used this util.
 2024-05-12 17:42:18 +05:30
siddhantCodes
ddea4c6deb rm isLink 
isLink util is removed in favour of std::filesystem::is_symlink
 2024-05-08 19:59:37 +05:30
Siddhant Kumar
fcbc36cf78
Use std::filesystem::path in more places (#10657) 
Progress on #9205

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>

* Get rid of `PathNG`, just use `std::filesystem::path`
 2024-05-07 22:28:50 +00:00
John Ericson
a3c573950b Replace our DirEntry with std::filesystem's 2024-05-07 16:21:02 -04:00
John Ericson
c371070580 Use std::filesystem functions in more places 
This makes for shorter and more portable code.

The only tricky part is catching exceptions: I just searched for near by
`catch (Error &)` or `catch (SysError &)` and adjusted them to `catch
(std::filesystem::filesystem_error &)` according to my human judgement.

Good for windows portability; will help @siddhantk232 with his GSOC
project.
 2024-05-07 00:16:54 -04:00
ramboman
b4950404ba
Honor the same set of proxy environment variables (#10611) 
Different parts of the project honor different sets of proxy environment
variables. With this commit all parts of the project will honor the same
set of proxy environment variables.

---------

Co-authored-by: Your Name <you@example.com>
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
 2024-05-06 15:39:22 -04:00
Eelco Dolstra
eab2919119 Use SourcePath in more places 
Now that SourcePath uses a SourceAccessor instead of an InputAccessor,
we can use it in function signatures instead of passing a
SourceAccessor and CanonPath separately.
 2024-05-06 19:05:42 +02:00
Eelco Dolstra
ba5929c7be Merge InputAccessor into SourceAccessor 
After the removal of the InputAccessor::fetchToStore() method, the
only remaining functionality in InputAccessor was `fingerprint` and
`getLastModified()`, and there is no reason to keep those in a
separate class.
 2024-05-03 12:14:01 +02:00
Eelco Dolstra
cf1adbdd64
Merge pull request #10572 from NorfairKing/no-more-prev 
Remove the 'prev' check entirely.
 2024-04-22 10:53:31 +02:00
Tom Sydney Kerckhove
ad65a50a94 Remove the 'prev' check 2024-04-20 14:46:23 +02:00
John Ericson
6fa3656a32 Make a few commands that were Unix-only no longer 
Also clean up some more linux-specific (`setPersonality`) code in
alignment with recent best practices.
 2024-04-17 16:20:56 -04:00