Commit graph

10887 commits

Author SHA1 Message Date
Robert Hensing
4c014e238b nix-main: Add openssl 2024-07-06 17:52:57 +02:00
Robert Hensing
4d0c55ae55 api docs: Use mkMesonDerivation 2024-07-06 17:52:57 +02:00
John Ericson
3acf3fc746 Package libnixmain and libnixcmd with Meson
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-05 16:40:55 -04:00
Robert Hensing
0b901e10ee
Merge pull request #11050 from hercules-ci/issue-10677
Explain when `man` is missing
2024-07-05 22:25:38 +02:00
John Ericson
ff9b6d0e1f
Merge pull request #11037 from fricklerhandwerk/document-config-parsing
use self-descriptive name for config file parser, document
2024-07-05 15:21:26 -04:00
Eelco Dolstra
d5461b9009
Merge pull request #11051 from Mic92/fix-prefetch
src/nix/prefetch: fix prefetch containing current directory instead o…
2024-07-05 20:33:05 +02:00
Robert Hensing
ddff76f667
Merge pull request #10973 from NixOS/meson-libexpr
Meson build for libexpr libflake, external C API, unit tests
2024-07-05 20:27:12 +02:00
Jörg Thalheim
05381c0b30
Update src/nix/prefetch.cc
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-07-05 19:45:03 +02:00
Jörg Thalheim
8cea1fbd97 src/nix/prefetch: fix prefetch containing current directory instead of tarball
When --unpack was used the nix would add the current directory to the
nix store instead of the content of unpacked.
The reason for this is that std::distance already consumes the iterator.
To fix this we re-instantiate the directory iterator in case the
directory only contains a single entry.
2024-07-05 19:18:49 +02:00
Robert Hensing
6ef00a503a Explain when man is missing
Have you seen this man?

Fixes #10677
2024-07-05 19:18:23 +02:00
Eelco Dolstra
8f280d72ff
Merge pull request #11019 from DeterminateSystems/fix-failed-to-open-archive
Tarball fetcher: Fix handling of cached tarballs
2024-07-05 17:10:02 +02:00
Robert Hensing
a476383f46
Merge pull request #11031 from emilazy/push-xsrvoyspsvqx
libstore: fix sandboxed builds on macOS
2024-07-05 17:08:39 +02:00
Robert Hensing
d63bd8295e assert: Report why values aren't equal 2024-07-05 16:43:48 +02:00
Eelco Dolstra
98bef7c38e
Merge pull request #11035 from siddhantk232/refactor
Factor duplicate code into util function `append`
2024-07-05 16:34:23 +02:00
Eelco Dolstra
61e1880847
Merge pull request #11041 from hercules-ci/trace-nix-env-attribute-names
getDerivations: add attributes to trace
2024-07-05 16:32:27 +02:00
Eelco Dolstra
e1b6b3ce27
Merge pull request #11020 from DeterminateSystems/fix-tarball-caching
Tarball fetcher: Fix fetchToStore() and eval caching
2024-07-05 16:30:12 +02:00
Eelco Dolstra
e7e070d36b Document 2024-07-05 16:29:16 +02:00
Robert Hensing
09763c7cad getDerivations: add attributes to trace
This improves the error message of nix-env -qa, among others, which
is crucial for understanding some ofborg eval error reports, such as
https://gist.github.com/GrahamcOfBorg/89101ca9c2c855d288178f1d3c78efef

After this change, it will report the same trace, but also start with

```
error:
       … while evaluating the attribute 'devShellTools'

       … while evaluating the attribute 'nixos'

       … while evaluating the attribute 'docker-tools-nix-shell'

       … while evaluating the attribute 'aarch64-darwin'

       … from call site
         at /home/user/h/nixpkgs/outpaths.nix:48:6:
           47|   tweak = lib.mapAttrs
           48|     (name: val:
             |      ^
           49|       if name == "recurseForDerivations" then true

<same>
```
2024-07-05 15:30:07 +02:00
John Ericson
e4056b9afd
Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-04 17:48:27 -04:00
Emily
af2e1142b1 libstore: fix sandboxed builds on macOS
The recent fix for CVE-2024-38531 broke the sandbox on macOS
completely. As it’s not practical to use `chroot(2)` on
macOS, the build takes place in the main filesystem tree, and the
world‐unreadable wrapper directory prevents the build from accessing
its `$TMPDIR` at all.

The macOS sandbox probably shouldn’t be treated as any kind of a
security boundary in its current state, but this specific vulnerability
wasn’t possible to exploit on macOS anyway, as creating `set{u,g}id`
binaries is blocked by sandbox policy.

Locking down the build sandbox further may be a good idea in future,
but it already has significant compatibility issues. For now, restore
the previous status quo on macOS.

Thanks to @alois31 for helping me come to a better understanding of
the vulnerability.

Fixes: 1d3696f0fb
Closes: #11002
2024-07-04 16:28:37 +01:00
Emily
76e4adfaac libstore: clean up the build directory properly
After the fix for CVE-2024-38531, this was only removing the nested
build directory, rather than the top‐level temporary directory.

Fixes: 1d3696f0fb
2024-07-04 16:22:02 +01:00
Valentin Gagarin
c66079f1e8 use self-descriptive name for config file parser, document
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-04 10:36:48 +02:00
siddhantCodes
976c05879f factor duplicate code into util function append 2024-07-04 11:09:23 +05:30
Eelco Dolstra
5b4102c3b2 Tarball fetcher: Include revCount/lastModified in the fingerprint
This can influence the evaluation result so they should be included in
the fingerprint.
2024-07-03 22:05:45 +02:00
kn
10ccdb7a41 Use proper struct sockpeercred for SO_PEERCRED for OpenBSD
getsockopt(2) documents this;  ucred is wrong ("cr_" member prefix, no pid).
2024-07-03 11:16:39 -04:00
John Ericson
a09360400b Ident some CPP in nix daemon
Makes it easier for me to read.
2024-07-03 11:15:56 -04:00
Eelco Dolstra
1ff186fc6e nix flake metadata: Show flake fingerprint
This is useful for testing/debugging and maybe for sharing eval caches
(since it tells you what file in ~/.cache/nix/eval-cache-v5 to copy).
2024-07-03 17:00:30 +02:00
Eelco Dolstra
9d95c228ee Tarball fetcher: Fix fetchToStore() and eval caching 2024-07-03 16:50:45 +02:00
John Ericson
30de61f16d
Merge pull request #11018 from siddhantk232/canonpath-fs-sink
Use `CanonPath` in `fs-sink.hh`
2024-07-03 10:36:18 -04:00
Eelco Dolstra
79ed3df8f8 Tarball fetcher: Fix handling of cached tarballs
Fixes a regression introduced in
5a9e1c0d20 where downloading a cached
file causes the error "Failed to open archive (Unrecognized archive
format)".
2024-07-03 14:14:20 +02:00
siddhantCodes
2cf24a2df0 fix tests and minor changes
- use the iterator in `CanonPath` to count `level`
- use the `CanonPath::basename` method
- use `CanonPath::root` instead of `CanonPath{""}`
- remove `Path` and `PathView`, use `std::filesystem::path` directly
2024-07-03 17:43:55 +05:30
Valentin Gagarin
a713476790
docs: split types from syntax (#11013)
move together all syntactic and semantic information into one
page, and add a page on data types, which in turn links to the syntax and
semantics.

also split out the note on scoping rules into its own page.

Co-authored-by: Ryan Hendrickson <ryan.hendrickson@alum.mit.edu>
2024-07-03 09:03:41 +02:00
John Ericson
451f8a8c19 Put back files for now
We'll revert this sometime later
2024-07-02 09:26:22 -04:00
John Ericson
11946817f0 fileset for store unit test data 2024-07-02 09:26:22 -04:00
John Ericson
4d6bc61b8d Fix things 2024-07-02 09:26:22 -04:00
John Ericson
11dab30be9 Update docs on the unit tests 2024-07-02 09:26:22 -04:00
John Ericson
224c6c3256 Fix test symlinks 2024-07-02 09:26:22 -04:00
John Ericson
496b4a9cd2 Move around unit test dirs to match new names 2024-07-02 09:26:22 -04:00
John Ericson
3ad39d2afb Fix library name 2024-07-02 09:26:22 -04:00
John Ericson
513f6b9718 meson: Prelink links to avoid missing C++ initializers
This is the same as what the old build system did in
7eca8a16ea, done for the same reasons.
2024-07-02 09:26:22 -04:00
John Ericson
912c517bc0 Fix build of unit tests 2024-07-02 09:26:22 -04:00
John Ericson
f7ce10dbc1 Fix static build 2024-07-02 09:26:22 -04:00
John Ericson
874ff000d4 Fix format 2024-07-02 09:26:22 -04:00
John Ericson
7312d13acc Keep another test dir 2024-07-02 09:26:22 -04:00
John Ericson
479befa76d More fixes 2024-07-02 09:26:22 -04:00
John Ericson
6a0582d9fd Rename file to avoid reserved name 2024-07-02 09:26:22 -04:00
John Ericson
2c184f694b Ensure we have data dir for libexpr unit tests 2024-07-02 09:26:22 -04:00
John Ericson
46ec69a483 Everything builds in the dev shell now 2024-07-02 09:26:22 -04:00
John Ericson
429d6ae2b5 Add missing package.nix 2024-07-02 09:26:22 -04:00
John Ericson
0b539dea4a Improve boost hacks 2024-07-02 09:26:22 -04:00
John Ericson
8399bd6b8f Dedup 2024-07-02 09:26:21 -04:00
John Ericson
8198888bc4 More dedup 2024-07-02 09:23:25 -04:00
John Ericson
d6f57f3260 More dedup 2024-07-02 09:23:25 -04:00
John Ericson
c88f83b471 More dedup 2024-07-02 09:23:25 -04:00
John Ericson
4609ab318c Fix internal API docs 2024-07-02 09:23:25 -04:00
John Ericson
d902481a36 Better org 2024-07-02 09:23:25 -04:00
John Ericson
a81e319528 Deduplicating 2024-07-02 09:23:24 -04:00
John Ericson
17a8c2bfce Unit tests and external libraries 2024-07-02 09:23:24 -04:00
John Ericson
4fa8068b78 Mesonify other external API 2024-07-02 09:23:24 -04:00
John Ericson
31257009e1 Meson build for libexpr and libflake 2024-07-02 09:23:24 -04:00
John Ericson
fbdc554908 Fix Nix shell for building Perl too 2024-07-02 09:23:24 -04:00
Jörg Thalheim
101915c9b7 enable -Werror=unused-result
Inspired by
010ff57ebb

From the original PR:

> We do not have any of these warnings appearing at the moment, but
> it seems like a good idea to enable [[nodiscard]] checking anyway.
> Once we start introducing more functions with must-use conditions we will
> need such checking, and the rust stdlib has proven them very useful.
2024-07-02 08:46:06 +02:00
Robert Hensing
f0329568b5 GitFileSystemObjectSink: catch an overflow 2024-07-01 17:29:07 +02:00
Robert Hensing
1fac22b16e GitFileSystemObjectSink: Add path context to some messages 2024-07-01 17:29:07 +02:00
Robert Hensing
992912f3b4 test-support: Add TracingFileSystemObjectSink 2024-07-01 17:29:07 +02:00
Eelco Dolstra
10c9764c27
Merge pull request #10992 from hercules-ci/issue-10947-dont-cache-disallowed-ifd
Fix #10947; don't cache disallowed IFD
2024-07-01 11:20:27 +02:00
Robert Hensing
93b50857ed packaging: Restore .version value altering behavior 2024-06-30 19:44:36 +02:00
Robert Hensing
7dd938b228 libutil/package.nix: Remove .version symlink replacement solution 2024-06-30 19:44:04 +02:00
siddhantCodes
72bb530141 use CanonPath in fs-sink and its derivatives 2024-06-30 19:03:15 +05:30
Robert Hensing
b2c7f09b0a Fix underflow in Printer::printList
Analogous to 9b88bf8adf / three commits back
2024-06-29 14:10:58 +02:00
Robert Hensing
bfc5416240 Refactor: rename ValuePrinter::totalListItemsPrinted 2024-06-29 14:02:28 +02:00
Robert Hensing
ce1dc87711 Refactor: rename ValuePrinter::totalAttrsPrinted
Make it more distinct from the attrs printed of any specific
attrset.
2024-06-29 14:01:15 +02:00
Robert Hensing
9b88bf8adf Fix underflow in Printer::printAttrs
The code that counts the number of elided attrs incorrectly used the
per-printer "global" attribute counter instead of a counter that
was relevant only to the current attribute set.

This bug flew under the radar because often the attribute sets aren't
nested, not big enough, or we wouldn't pay attention to the numbers.
I've noticed the issue because the difference underflowed.

Although this behavior is tested by the functional test
lang/eval-fail-bad-string-interpolation-4.nix, the underflow slipped
through review. A simpler reproducer would be as follows, but I
haven't added it to the test suite to keep it simple and marginally
faster.

```
$ nix run nix/2.23.1 -- eval --expr '"" + (let v = { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; b = 1; c = 1; }; }; in builtins.deepSeq v v)'
error:
       … while evaluating a path segment
         at «string»:1:6:
            1| "" + (let v = { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; b = 1; c = 1; }; }; in builtins.deepSeq v v)
             |      ^

       error: cannot coerce a set to a string: { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; «4294967289 attributes elided» }; }
```
2024-06-29 13:53:05 +02:00
Robert Hensing
fd94b74ee5 Fix #10947; don't cache disallowed IFD 2024-06-29 13:19:04 +02:00
Cole Helbling
d906d42330 Restore commit-lock-file-summary rename for consistency
It was originally renamed in https://github.com/NixOS/nix/pull/10691,
but https://github.com/NixOS/nix/pull/9063 accidentally removed the new
name and alias.
2024-06-28 14:53:40 -07:00
Winter
149d8eb8aa Stop vendoring toml11
We don't apply any patches to it, and vendoring it locks users into
bugs (it hasn't been updated since its introduction in late 2021).

Closes https://git.lix.systems/lix-project/lix/issues/164

Change-Id: Ied071c841fc30b0dfb575151afd1e7f66970fdb9
(cherry picked from commit 80405d06264f0de1c16ee2646388ab501df20628)
2024-06-26 22:27:13 -04:00
John Ericson
f002f85861 Avoid libmain header in libexpr
We just don't need it!
2024-06-26 22:26:45 -04:00
John Ericson
7181d1f4a1 Reformat
Factored out code is now elegible for formatting.
2024-06-26 19:56:21 -04:00
John Ericson
0084a486cc Split out a new libnixflake
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-26 19:56:21 -04:00
John Ericson
52730d38e2 Factor out flake:... lookup path from evaluator
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-26 19:56:21 -04:00
John Ericson
8a420162ab Merge branch 'master' into fix-sandbox-escape 2024-06-26 18:11:39 -04:00
Robert Hensing
85de5a60c7 Use lib instead of explicit fileset passing 2024-06-26 04:11:20 +02:00
Robert Hensing
6fe8fb967a libstore/worker.hh: Document Worker 2024-06-26 01:43:31 +02:00
Robert Hensing
fd0b376c79 libstore/worker.cc: Remove outdated comment
It was added above this conditional

    Worker::Worker(LocalStore & store)
        : store(store)
    {
        /* Debugging: prevent recursive workers. */
        if (working) abort();
        working = true;

However, `working` has since been removed.

Source: 7f8e805c8e/src/libstore/build.cc (L2617)
2024-06-26 01:43:31 +02:00
John Ericson
65d7c80365
Merge pull request #10955 from NixOS/meson-nix-util-c
Build nix-util-c with meson + unit tests
2024-06-25 19:06:06 -04:00
Robert Hensing
ac89828b5a Build nix-util-c with meson and unit test 2024-06-25 21:35:23 +02:00
Robert Hensing
7df9d6da65 Improve error messages for invalid derivation names 2024-06-25 19:41:29 +02:00
John Ericson
5f4f789144
Merge pull request #10954 from NixOS/ci-meson
ci.yml: Add meson_build
2024-06-25 09:02:33 -04:00
Robert Hensing
0674be8d49 nix-util: Fix build 2024-06-25 10:26:57 +02:00
Brian McKenna
5be44d235a Guard uses of lutimes, for portability 2024-06-24 17:35:34 -04:00
John Ericson
05580a373f Fix error in the no-GC build 2024-06-24 17:18:16 -04:00
Robert Hensing
c66f1e7660
Merge pull request #10913 from NixOS/no-global-eval-settings-in-libexpr
No global eval settings in `libnixexpr`
2024-06-24 18:52:19 +02:00
John Ericson
fda4c78921
Merge pull request #10951 from obsidiansystems/load-just-one-config
Small global config refactors
2024-06-24 12:38:04 -04:00
John Ericson
52bfccf8d8 No global eval settings in libnixexpr
Progress on #5638

There is still a global eval settings, but it pushed down into
`libnixcmd`, which is a lot less bad a place for this sort of thing.
2024-06-24 12:15:16 -04:00
John Ericson
cb0c868da4 Allow loading config files into other config objects
This gives us some hope of moving away from global variables.
2024-06-24 12:07:56 -04:00
John Ericson
b46e13840b Format config-global.{cc,hh}
Since the code is factored out, it is no longer avoding the formatter.
2024-06-24 12:07:56 -04:00
John Ericson
1620ad4587 Split out GlobalConfig into its own header
This makes it easier to understand the reach of global variables /
global state in the config system.
2024-06-24 11:36:21 -04:00
John Ericson
b51e161af5 Cleanup ContentAddressMethod to match docs
The old `std::variant` is bad because we aren't adding a new case to
`FileIngestionMethod` so much as we are defining a separate concept ---
store object content addressing rather than file system object content
addressing. As such, it is more correct to just create a fresh
enumeration.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-24 10:24:06 -04:00
John Ericson
64e599ebe1 Rename Recursive -> NixArchive
For enums:

- `FileIngestionMethod`

- `FileSerialisationMethod`
2024-06-24 10:24:06 -04:00
Eelco Dolstra
903acc7c0f
Merge pull request #10873 from siddhantk232/rm-createdirs
use `std::filesystem::create_directories` for createDirs
2024-06-24 14:54:37 +02:00
Eelco Dolstra
bc21c54565
Merge pull request #10943 from pineapplehunter/master
Accept response from gitlab api with more than one entry in json
2024-06-24 14:23:47 +02:00
Robert Hensing
6f64154eea
Merge pull request #10884 from tomberek/tomberek.warn_structuredAttrs_advanced
fix: warn and document when advanced attributes will have no impact d…
2024-06-24 07:56:26 +02:00
John Ericson
df068734ac
Merge pull request #10769 from poweredbypie/mingw-spawn
Implement runProgram for Windows
2024-06-23 14:12:36 -04:00
Shogo Takata
0468061dd2
accept response from gitlab with more than one entry 2024-06-23 00:52:19 +09:00
Eelco Dolstra
d54590fdf3 Fix --no-sandbox
When sandboxing is disabled, we cannot put $TMPDIR underneath an
inaccessible directory.
2024-06-21 17:06:19 +02:00
Eelco Dolstra
58b7b3fd15 Formatting 2024-06-21 17:06:19 +02:00
Eelco Dolstra
ede95b1fc1 Put the chroot inside a directory that isn't group/world-accessible
Previously, the .chroot directory had permission 750 or 755 (depending
on the uid-range system feature) and was owned by root/nixbld. This
makes it possible for any nixbld user (if uid-range is disabled) or
any user (if uid-range is enabled) to inspect the contents of the
chroot of an active build and maybe interfere with it (e.g. via /tmp
in the chroot, which has 1777 permission).

To prevent this, the root is now a subdirectory of .chroot, which has
permission 700 and is owned by root/root.
2024-06-21 17:06:19 +02:00
Théophane Hufschmitt
1d3696f0fb Run the builds in a daemon-controled directory
Instead of running the builds under
`$TMPDIR/{unique-build-directory-owned-by-the-build-user}`, run them
under `$TMPDIR/{unique-build-directory-owned-by-the-daemon}/{subdir-owned-by-the-build-user}`
where the build directory is only readable and traversable by the daemon user.

This achieves two things:

1. It prevents builders from making their build directory world-readable
   (or even writeable), which would allow the outside world to interact
   with them.
2. It prevents external processes running as the build user (either
   because that somehow leaked, maybe as a consequence of 1., or because
   `build-users` isn't in use) from gaining access to the build
   directory.
2024-06-21 17:06:19 +02:00
siddhantCodes
85b7989764 fix: handle errors in nix::createDirs
the `std::filesystem::create_directories` can fail due to insufficient
permissions. We convert this error into a `SysError` and catch it
wherever required.
2024-06-20 19:53:25 +05:30
Valentin Gagarin
1c131ec2b7
Port C API docs to Meson (#10936)
* Port C API docs to Meson

* don't cross-compile the docs
2024-06-19 22:43:54 +02:00
PoweredByPie
8b81d083a7 Remove lookupPathForProgram and implement initial runProgram test
Apparently, CreateProcessW already searches path, so manual path search
isn't really necessary.
2024-06-18 01:01:52 -07:00
PoweredByPie
fcb92b4fa4 Fix DWORD vs. int comparison warning 2024-06-17 22:14:38 -07:00
Mingye Wang
ff1fc780d2
optimize-store.cc: Update macos exclusion comments
#2230 broadened the scope of macOS hardlink exclusion but did not change the comments. This was a little confusing for me, so I figured the comments should be updated.
2024-06-18 12:05:59 +08:00
PoweredByPie
4f6e3b9402 Implement tests for lookupPathForProgram and fix bugs caught by tests 2024-06-17 18:46:08 -07:00
PoweredByPie
4662e7d856 Implement windowsEscape 2024-06-17 14:57:57 -07:00
Tom Bereknyei
706edf26eb build: meson for libfetchers 2024-06-17 17:25:56 -04:00
PoweredByPie
b11cf8166f Format runProgram declaration 2024-06-17 13:12:28 -07:00
Valentin Gagarin
6e34c68327 Convert the internal API doc build to Meson 2024-06-17 15:51:58 -04:00
John Ericson
a83d95e26e Integrate perl with the other meson builds
One big dev shell!
2024-06-17 14:48:20 -04:00
PoweredByPie
a58ca342ca Initial runProgram implementation for Windows
This is incomplete; proper shell escaping needs to be done
2024-06-17 11:13:22 -07:00
John Ericson
a1bb668ccb Merge remote-tracking branch 'upstream/master' into rm-createdirs 2024-06-17 12:57:54 -04:00
Robert Hensing
e48abec567
Merge pull request #10916 from jmbaur/read-only-no-chown
Don't chown when local-store is read-only
2024-06-17 13:49:19 +02:00
Eelco Dolstra
48d38b32d2
Merge pull request #10918 from andir/restrict-tarfile-formats
Restrict supported tarball formats to actual Tarballs
2024-06-17 13:20:23 +02:00
PoweredByPie
b0cfac8f93 Fix compile error on windows 2024-06-17 00:03:50 -07:00
Jared Baur
de639ceafe
Don't chown when local-store is read-only
If the local-store is using the read-only flag, the underlying
filesystem might be read-only, thus an attempt to `chown` would always
fail.
2024-06-16 23:03:33 -07:00
Andreas Rammhold
5a9e1c0d20 Restrict supported tarball formats to actual Tarballs
The documentation is clear about the supported formats (with at least
`builtins.fetchTarball`). The way the code was written previously it
supported all the formats that libarchive supported. That is a
surprisingly large amount of formats that are likely not on the radar
of the Nix developers and users. Before people end up relying on
this (or if they do) it is better to break it now before it becomes a
widespread "feature".

Zip file support has been retained as (at least to my knowledge)
historically that has been used to fetch nixpkgs in some shell
expressions *many* years back.

Fixes https://github.com/NixOS/nix/issues/10917
2024-06-15 14:28:20 +02:00
Robert Hensing
573e385a68
Merge pull request #10907 from hercules-ci/issue-10561
C API: Use opaque struct instead of void for `nix_value`
2024-06-15 10:12:13 +02:00
John Ericson
4728840ca8
Merge pull request #10908 from NixOS/meson-libstore
Build `nix-store` with Meson, adjust Perl bindings deps to integrate with other meson builds
2024-06-14 12:16:14 -04:00
Eelco Dolstra
b3bd31c82d
Merge pull request #10878 from kirillrdy/kr/sysv
darwin: allow ipc-sysv* in sandbox
2024-06-14 17:11:20 +02:00
John Ericson
81004a05c6 Build nix-store with Meson
Special thanks to everyone that has worked on a Meson port so far,
@p01arst0rm and @Qyriad in particular.

Co-Authored-By: p01arst0rm <polar@ever3st.com>
Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Artemis Tosini <me@artem.ist>
Co-Authored-By: Felix Uhl <felix.uhl@outlook.com>
Co-Authored-By: Jade Lovelace <lix@jade.fyi>
Co-Authored-By: Lunaphied <lunaphied@lunaphied.me>
Co-Authored-By: Maximilian Bosch <maximilian@mbosch.me>
Co-Authored-By: Pierre Bourdon <delroth@gmail.com>
Co-Authored-By: Qyriad <qyriad@qyriad.me>
Co-Authored-By: Rebecca Turner <rbt@sent.as>
Co-Authored-By: Winter <winter@winter.cafe>
Co-Authored-By: eldritch horrors <pennae@lix.systems>
Co-Authored-By: jade <lix@jade.fyi>
Co-Authored-By: julia <midnight@trainwit.ch>
Co-Authored-By: rebecca “wiggles” turner <rbt@sent.as>
Co-Authored-By: wiggles dog <rbt@sent.as>
Co-Authored-By: fricklerhandwerk <valentin@fricklerhandwerk.de>
Co-authored-by: Eli Schwartz <eschwartz93@gmail.com>
2024-06-14 10:25:14 -04:00
John Ericson
ea8e49bea5 Force the cpuid option for libutil rather than relying on detection
This is more robust, and match's Nixpkgs policy to force enable flags
statically by default (a common distro thing).
2024-06-14 10:25:14 -04:00
John Ericson
2f5fdab06c
Merge pull request #10905 from obsidiansystems/platform-namespace
Put some file descriptor functions in unix and windows namespaces
2024-06-14 08:45:31 -04:00
Robert Hensing
b94e1d6218 C API: Value -> nix_value
See issue https://github.com/NixOS/nix/issues/10434
2024-06-13 18:51:58 +02:00
Robert Hensing
5d8118d9cb C API: Docs 2024-06-13 18:23:21 +02:00
Robert Hensing
c50db4e58c C API: Add nix_value_{inc,dec}ref
- Can be implemented more easily by more eval architectures.
- Better types in generated bindings remove some uncertainty and doubt.
2024-06-13 18:21:04 +02:00
Robert Hensing
0b56c98b1c C API: Value -> nix_value 2024-06-13 18:18:36 +02:00
John Ericson
56f0b5304f Document the nix-util Meson build system more extensively
I hope this will make it easier to maintain, and also make it easier for
others to assist with porting the rest of the build system to Meson.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-13 11:39:59 -04:00
John Ericson
ff87c1a318 Put some file descriptor functions in unix and windows namespaces
It is misleading when platform-specific functions are in the overall
`nix` namespace. More namespaces also makes for nicer doxygen.
2024-06-13 11:18:59 -04:00
Robert Hensing
afdd12be5e
Merge pull request #10900 from obsidiansystems/hash-order
hash: Compare hash algo second for back compat
2024-06-13 01:35:31 +02:00
John Ericson
25a9894943 hash: Compare hash algo second for back compat
Previously (in cfc18a7739), we forgot to
compare the algo at all. This means we keep the same ordering as before
by making the stuff we always have compared take priority.
2024-06-12 18:35:47 -04:00
John Ericson
28d2af4ea6 Build nix-util with Meson
The idea is two-fold:

- Replace autotools with Meson

- Build each library in its own derivation

The interaction of these two features is that Meson's "subprojects"
feature (https://mesonbuild.com/Subprojects) allows us to have single
dev shell for building all libraries still, while also building things
separately. This allows us to break up the build without a huge
productivity lost.

I tested the Linux native build, and NetBSD and Windows cross builds.

Also do some clean ups of the Flake in the process of supporting new
jobs.

Special thanks to everyone that has worked on a Meson port so far,
@p01arst0rm and @Qyriad in particular.

Co-Authored-By: p01arst0rm <polar@ever3st.com>
Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Artemis Tosini <me@artem.ist>
Co-Authored-By: Felix Uhl <felix.uhl@outlook.com>
Co-Authored-By: Jade Lovelace <lix@jade.fyi>
Co-Authored-By: Lunaphied <lunaphied@lunaphied.me>
Co-Authored-By: Maximilian Bosch <maximilian@mbosch.me>
Co-Authored-By: Pierre Bourdon <delroth@gmail.com>
Co-Authored-By: Qyriad <qyriad@qyriad.me>
Co-Authored-By: Rebecca Turner <rbt@sent.as>
Co-Authored-By: Winter <winter@winter.cafe>
Co-Authored-By: eldritch horrors <pennae@lix.systems>
Co-Authored-By: jade <lix@jade.fyi>
Co-Authored-By: julia <midnight@trainwit.ch>
Co-Authored-By: rebecca “wiggles” turner <rbt@sent.as>
Co-Authored-By: wiggles dog <rbt@sent.as>
Co-Authored-By: fricklerhandwerk <valentin@fricklerhandwerk.de>
Co-authored-By: Eli Schwartz <eschwartz93@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-12 18:31:02 -04:00
John Ericson
5b53d8fec3 Factor out GC initialization code
This is not really part of the evaluator: it is just an integration
between Boehm GC and Boost coroutines usable for any purpose. The
evaluator (merely) optionally uses it.
2024-06-12 16:00:03 -04:00
John Ericson
7c2981fc55 Fix FreeBSD build
This restores some CPP'd code that was added in
c18911602e and accidentally lost in
2477e4e3b8.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-06-12 15:59:54 -04:00
Eelco Dolstra
efd4bf6533
Update src/libfetchers/git-utils.cc
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-12 14:41:35 +02:00
Eelco Dolstra
35bdb9cee7 Support hard links in tarballs
Fixes #10395.
2024-06-11 16:05:57 +02:00
Tom Bereknyei
4809e59b7e fix: warn and document when advanced attributes will have no impact due to __structuredAttrs 2024-06-10 09:31:21 -04:00
Tom Bereknyei
1363f51bcb fix: remove usage of XDG_RUNTIME_DIR for TMP 2024-06-10 08:38:57 -04:00
Robert Hensing
f91f34aa65 bdwgc 8.2.4 has sp_corrector
> Support client-defined stack pointer adjustment before thread stack push

-- https://github.com/ivmai/bdwgc/releases/tag/v8.2.4

This fixes an inaccuracy in cc6f315252,
in the update to Nixpkgs 24.05 https://github.com/NixOS/nix/pull/10835

After this fixup, the build log won't ask for an upgrade, and we'll be
able to collect when a coroutine exists, e.g. during filterSource.
2024-06-10 12:57:13 +02:00
Robert Hensing
0a09597790 Typo 2024-06-10 12:46:04 +02:00
Robert Hensing
4755e133c4 Fix warning 2024-06-10 12:46:04 +02:00
siddhantCodes
7a21432e77 fix: catch filesystem_error thrown by createDirs 2024-06-10 11:30:39 +05:30
Kirill Radzikhovskyy
372d5a441e darwin: allow ipc-sysv* in sandbox 2024-06-10 11:21:17 +10:00
siddhantCodes
25b0242ca6 std::filesystem::create_directories for createDirs
The implementation of `nix::createDirs` allows it to be a simple wrapper
around `std::filesystem::create_directories` as its return value is not
used anywhere.
2024-06-09 19:49:39 +05:30
Robert Hensing
0ab9369572
Merge pull request #10854 from DeterminateSystems/thread-safe-RegexCache
Make RegexCache thread-safe
2024-06-08 18:07:35 +02:00
Robert Hensing
e1b3716d50
Merge pull request #10853 from DeterminateSystems/thread-safe-srcToStore
Make EvalState::srcToStore thread-safe
2024-06-06 19:31:46 +02:00
Eelco Dolstra
fd9e49480a PosixSourceAccessor: Use SharedSync 2024-06-06 10:44:53 +02:00
Eelco Dolstra
2f39caf180 Sync: Add support for shared locks 2024-06-06 10:44:09 +02:00
Robert Hensing
dadb6f86cf
Merge pull request #10865 from Mic92/better-cleanup
libutil: guard Finally against invalid exception throws
2024-06-05 23:59:51 +02:00
John Ericson
1a32490aca
Merge pull request #10835 from hercules-ci/update-nixpkgs-24.05
Update to Nixpkgs 24.05
2024-06-05 17:56:16 -04:00
eldritch horrors
e291087747 libutil: guard Finally against invalid exception throws
throwing exceptions is fine, but throwing exceptions during exception
handling is hard enough to do correctly that we should just forbid it
entirely out of an overabundance of caution. in cases where terminate
is the correct answer the users of Finally must call it manually now.

Source: 6c777476c9
2024-06-05 22:45:37 +02:00
Eelco Dolstra
edd445f745
Merge pull request #10861 from DeterminateSystems/fix-optional-deref
PackageInfo::queryDrvPath(): Don't dereference an empty optional
2024-06-05 16:48:09 +02:00
Eelco Dolstra
3e72ed9743 Typo 2024-06-05 16:19:01 +02:00
Eelco Dolstra
d2eeabf3e6 PackageInfo::queryDrvPath(): Don't dereference an empty optional
Fixes a regression introduced in f923ed6b6a.

https://hydra.nixos.org/build/262267313
2024-06-05 16:17:24 +02:00
Enno Richter
80ba7778e7 flake check: Recognize well known homeModule/homeModules attributes 2024-06-05 07:38:20 +02:00
Jörg Thalheim
49c6f34911
docs: fixup description of builtins.importNative (#10810)
There was an argument missing and the fact that the imported function is
called.
2024-06-04 19:55:05 +00:00
Eelco Dolstra
ef140c25d1
Merge pull request #10852 from siddhantk232/add-interrupts
add call to `checkInterrupt` in a bunch of places
2024-06-04 17:21:03 +02:00
Eelco Dolstra
fbbca59453 Make RegexCache thread-safe 2024-06-04 16:59:30 +02:00
Eelco Dolstra
7d295c594e Make EvalState::srcToStore thread-safe 2024-06-04 16:56:06 +02:00
siddhantCodes
8f1a26667e add call to checkInterrupt in a bunch of places
This brings back the old behaviour. We check for interrupts in places
that may iterate over wide directories.
2024-06-04 19:35:40 +05:30
Eelco Dolstra
21be03b233
Merge pull request #10840 from obsidiansystems/libutil-pkg-config
Create and install a `nix-util.pc`
2024-06-04 12:33:37 +02:00
John Ericson
4e0d058fc3 eval.cc: Fix for Windows 2024-06-04 10:18:22 +02:00
Philipp
214051ba79
clarify not on nix_value_force (#10842)
* clarify not on `nix_value_force`

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-06-04 07:41:04 +00:00
John Ericson
06be6812a6 Create and install a nix-util.pc
Before, `-lnixutil` was just stuck in `nix-store.pc`, but that doesn't
seem so nice.

This prepares us to distribute `libnixutil` in a separate package if we
want, but it should be a good change either way. I suspect it wasn't
done before because libutil was an extra unstable interface, but I don't
think we need worry about that. *All* the C++ is less stable than the C
(or that's the goal at least).

For what it's worth, Lix also created this pkg-config file *en passant*
during their rename:
c97e17144e (diff-3c4f60cc44a0e35444c7f45331cfa50f76637118)
2024-06-03 14:14:40 -04:00
Robert Hensing
27f880c098 Format after clang-format update 2024-06-03 18:38:56 +02:00
Eelco Dolstra
da92ad7dd2
Merge pull request #10592 from hercules-ci/builtins-warn
Add `builtins.warn`
2024-06-03 17:16:32 +02:00
Robert Hensing
cc6f315252 nix: Disable GC during coroutine when bdwgc < 8.4
This re-enables support for older bwdgc versions without complicating
the code too much.
Coroutines generally only interfere with GC during source filtering,
so it's not too bad of a regression on older bdwgc.
This seems preferable over conditional compilation to enable the patch
etc; we've already spent a lot of complexity budget on this GC-coroutine
interaction...
2024-06-03 16:37:39 +02:00
Robert Hensing
2477e4e3b8 libexpr: Use GC_set_sp_corrector instead of patch
Manually tested by printing to stderr in both branches (sp in os
stack, or not), and triggering a GC in a filterSource function,
e.g.:

    let
      generateTree = n: if n == 0 then "ha" else { left = generateTree (n - 1); right = generateTree (n - 1); };
    in
      builtins.deepSeq (generateTree 18) ...

Note that the darwin still uses the strategy of disabling GC, despite
having an implementation that compiles. The proper solution will be
enabled and tested later.
2024-06-03 16:37:39 +02:00
Robert Hensing
70b1036224 builtins.warn: Use new EvalBaseError + "evaluation warning" 2024-06-03 16:24:21 +02:00
Robert Hensing
831d96d8d7 builtins.warn: Do not throw EvalError 2024-06-03 16:24:21 +02:00
Robert Hensing
c07500e14d refactor: Extract EvalState::{runDebugRepl,canDebug} 2024-06-03 16:24:21 +02:00
Robert Hensing
da82d67022 builtins.warn: Require string argument
... so that we may perhaps later extend the interface.
Note that Nixpkgs' lib.warn already requires a string coercible
argument, so this is reasonable. Also note that string coercible
values aren't all strings, but in practice, for warn, they are.
2024-06-03 16:24:21 +02:00
Robert Hensing
923cbea2af builtins.warn: Use logWarning
Constructing ErrorInfo is a little awkward for now, but this does
produce a richer log entry.
2024-06-03 16:24:21 +02:00
Robert Hensing
2d4c9d8f4a Add builtins.warn 2024-06-03 16:24:21 +02:00
Eelco Dolstra
54a9fbe5d6 Merge remote-tracking branch 'origin/master' into large-path-warning 2024-06-03 16:17:52 +02:00
Eelco Dolstra
eb0d46fab6
Merge pull request #9897 from bryango/fix-submodule-subdir
libutil/url: fix git+file:./ parse error
2024-06-03 16:04:41 +02:00
Eelco Dolstra
ac3e5d22e3
Merge pull request #10028 from DavHau/fetchTree-shallow-default
fetchTree: shallow git fetching by default
2024-06-03 16:02:34 +02:00
John Ericson
4e62629a2d
Merge pull request #10833 from obsidiansystems/hash-ordering
Modernize `Hash` ordering with C++20 `<=>`
2024-06-03 09:50:04 -04:00
Eelco Dolstra
deac00c6d0 Rename large-path-warning-threshold -> warn-large-path-threshold 2024-06-03 15:49:15 +02:00
Eelco Dolstra
7f5b57d18f Merge remote-tracking branch 'origin/master' into large-path-warning 2024-06-03 15:32:27 +02:00
Eelco Dolstra
ecfad6a828
Merge pull request #10564 from edolstra/remove-forceErrors
AttrCursor: Remove forceErrors
2024-06-03 15:30:01 +02:00
John Ericson
cfc18a7739 Modernize Hash ordering with C++20 <=>
Progress on #10832

This doesn't switch to auto-deriving the fields, but by defining `<=>`
we allow deriving `<=>` in downstream types where `Hash` is used.
2024-06-03 09:24:33 -04:00
Eelco Dolstra
d16fcaee21
Merge pull request #10782 from obsidiansystems/both-connections
Factor our connection code for worker proto like serve proto
2024-06-03 15:10:38 +02:00
John Ericson
84c65135a5 ValidPathInfo JSON format should use null not omit field
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-03 08:21:22 -04:00
John Ericson
213a7a87b4 Decouple within-build (structured attrs) and unstable CLI path info JSON
See code comment for details.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-03 08:21:22 -04:00
Robert Hensing
b74a0df645
Merge pull request #10825 from tie/output-spec-assert
Fix empty outputsToInstall for InstallableAttrPath
2024-06-03 12:27:50 +02:00
Philipp Zander
25e2b1f7f7 improve note in nix_value_force documentation 2024-06-03 09:55:44 +02:00
Ivan Trubach
68090d7ff1 Fix empty outputsToInstall for InstallableAttrPath
Fixes assertion failure if outputsToInstall is empty by defaulting to the "out"
output. That is, behavior between the following commands should be consistent:

	$ nix build --no-link --json .#nothing-to-install-no-out
	error: derivation '/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-nothing-to-install-no-out.drv' does not have wanted outputs 'out'

	$ nix build --no-link --file default.nix --json nothing-to-install-no-out
	error: derivation '/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-nothing-to-install-no-out.drv' does not have wanted outputs 'out'

Real-world example of this issue:

	$ nix build --json .#.legacyPackages.aarch64-linux.texlive.pkgs.iwona
	error: derivation '/nix/store/dj0h6b0pnlnan5nidnhqa0bmzq4rv6sx-iwona-0.995b.drv' does not have wanted outputs 'out'

	$ git rev-parse HEAD
	eee33247cf6941daea8398c976bd2dda7962b125
	$ nix build --json --file . texlive.pkgs.iwona
	nix: src/libstore/outputs-spec.hh:46: nix::OutputsSpec::Names::Names(std::set<std::__cxx11::basic_string<char> >&&): Assertion `!empty()' failed.
	Aborted (core dumped)
2024-06-02 14:26:18 +03:00
John Ericson
e0b159549b Misc Windows fixes
1. Fix build by making the legacy SSH Storey's secret `logFD` setting
   not a setting on Windows. (It doesn't make sense to specify `void *`
   handles by integer cross-proccess, I don't think.)

2. Move some files that don't need to be Unix-only anymore back to their
   original locations.
2024-06-01 19:19:35 -04:00
Robert Hensing
802b4e403b
Merge pull request #10814 from Mic92/fix-nix-edit
Fix nix edit
2024-05-31 13:30:24 +02:00
Robert Hensing
84e116379c
Merge pull request #10812 from Mic92/build-perf
Remove 100s of CPU time (10%) from build times (1465s -> 1302s)
2024-05-31 13:28:24 +02:00