Commit graph

4610 commits

Author SHA1 Message Date
siddhantCodes
7a21432e77 fix: catch filesystem_error thrown by createDirs 2024-06-10 11:30:39 +05:30
Kirill Radzikhovskyy
372d5a441e darwin: allow ipc-sysv* in sandbox 2024-06-10 11:21:17 +10:00
Eelco Dolstra
ef140c25d1
Merge pull request #10852 from siddhantk232/add-interrupts
add call to `checkInterrupt` in a bunch of places
2024-06-04 17:21:03 +02:00
siddhantCodes
8f1a26667e add call to checkInterrupt in a bunch of places
This brings back the old behaviour. We check for interrupts in places
that may iterate over wide directories.
2024-06-04 19:35:40 +05:30
John Ericson
06be6812a6 Create and install a nix-util.pc
Before, `-lnixutil` was just stuck in `nix-store.pc`, but that doesn't
seem so nice.

This prepares us to distribute `libnixutil` in a separate package if we
want, but it should be a good change either way. I suspect it wasn't
done before because libutil was an extra unstable interface, but I don't
think we need worry about that. *All* the C++ is less stable than the C
(or that's the goal at least).

For what it's worth, Lix also created this pkg-config file *en passant*
during their rename:
c97e17144e (diff-3c4f60cc44a0e35444c7f45331cfa50f76637118)
2024-06-03 14:14:40 -04:00
Eelco Dolstra
deac00c6d0 Rename large-path-warning-threshold -> warn-large-path-threshold 2024-06-03 15:49:15 +02:00
Eelco Dolstra
7f5b57d18f Merge remote-tracking branch 'origin/master' into large-path-warning 2024-06-03 15:32:27 +02:00
Eelco Dolstra
d16fcaee21
Merge pull request #10782 from obsidiansystems/both-connections
Factor our connection code for worker proto like serve proto
2024-06-03 15:10:38 +02:00
John Ericson
84c65135a5 ValidPathInfo JSON format should use null not omit field
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-03 08:21:22 -04:00
John Ericson
213a7a87b4 Decouple within-build (structured attrs) and unstable CLI path info JSON
See code comment for details.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-03 08:21:22 -04:00
John Ericson
e0b159549b Misc Windows fixes
1. Fix build by making the legacy SSH Storey's secret `logFD` setting
   not a setting on Windows. (It doesn't make sense to specify `void *`
   handles by integer cross-proccess, I don't think.)

2. Move some files that don't need to be Unix-only anymore back to their
   original locations.
2024-06-01 19:19:35 -04:00
Jade Lovelace
473d2d56fc Remove 100s of CPU time (10%) from build times (1465s -> 1302s)
Result's from Mic92's framework 13th Gen Intel Core i7-1360P:

Before: 3595.92s user 183.01s system 1360% cpu 4:37.74 total
After: 3486.07s user 168.93s system 1354% cpu 4:29.79 total

I saw that boost/lexical_cast was costing about 100s in CPU time on our
compiles. We can fix this trivially by doing explicit template
instantiation in exactly one place and eliminating all other includes of
it, which is a code improvement anyway by hiding the boost.

Before:
```
lix/lix2 » ClangBuildAnalyzer --analyze buildtimeold.bin
Analyzing build trace from 'buildtimeold.bin'...
**** Time summary:
Compilation (551 times):
  Parsing (frontend):         1465.3 s
  Codegen & opts (backend):   1110.9 s

<snip>

**** Expensive headers:
178153 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 3426 ms), included via:
  40x: command.hh
  5x: command-installable-value.hh
  3x: installable-flake.hh
  2x: <direct include>
  2x: installable-attr-path.hh

176217 ms: ../src/libutil/error.hh (included 246 times, avg 716 ms), included via:
  36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh
  12x: globals.hh config.hh experimental-features.hh
  11x: file-system.hh file-descriptor.hh
  6x: serialise.hh strings.hh
  6x: <direct include>
  6x: archive.hh serialise.hh strings.hh
  ...

173243 ms: ../src/libstore/store-api.hh (included 152 times, avg 1139 ms), included via:
  55x: <direct include>
  39x: command.hh installable-value.hh installables.hh
  7x: libexpr.hh
  4x: local-store.hh
  4x: command-installable-value.hh installable-value.hh installables.hh
  3x: binary-cache-store.hh
  ...

170482 ms: ../src/libutil/serialise.hh (included 201 times, avg 848 ms), included via:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh
  14x: store-api.hh nar-info.hh hash.hh
  11x: <direct include>
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh
  6x: fetchers.hh hash.hh
  ...

169397 ms: ../src/libcmd/installables.hh (included 53 times, avg 3196 ms), included via:
  40x: command.hh installable-value.hh
  5x: command-installable-value.hh installable-value.hh
  3x: installable-flake.hh installable-value.hh
  2x: <direct include>
  1x: installable-derived-path.hh
  1x: installable-value.hh
  ...

159740 ms: ../src/libutil/strings.hh (included 221 times, avg 722 ms), included via:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh
  19x: <direct include>
  14x: store-api.hh nar-info.hh hash.hh serialise.hh
  11x: serialise.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh
  ...

156796 ms: ../src/libcmd/command.hh (included 51 times, avg 3074 ms), included via:
  42x: <direct include>
  7x: command-installable-value.hh
  2x: installable-attr-path.hh

150392 ms: ../src/libutil/types.hh (included 251 times, avg 599 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh
  11x: file-system.hh
  10x: globals.hh
  6x: fetchers.hh
  6x: serialise.hh strings.hh error.hh
  5x: archive.hh
  ...

133101 ms: /nix/store/644b90j1vms44nr18yw3520pzkrg4dd1-boost-1.81.0-dev/include/boost/lexical_cast.hpp (included 226 times, avg 588 ms), included via
:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh strings.hh
  19x: file-system.hh
  11x: store-api.hh nar-info.hh hash.hh serialise.hh strings.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  6x: eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  ...

132887 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 507 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh
  16x: gtest.h
  11x: file-system.hh types.hh ref.hh
  10x: globals.hh types.hh ref.hh
  10x: json.hpp
  6x: serialise.hh
  ...

  done in 0.6s.
```

After:
```
lix/lix2 » maintainers/buildtime_report.sh build
Processing all files and saving to '/home/jade/lix/lix2/maintainers/../buildtime.bin'...
  done in 0.6s. Run 'ClangBuildAnalyzer --analyze /home/jade/lix/lix2/maintainers/../buildtime.bin' to analyze it.
Analyzing build trace from '/home/jade/lix/lix2/maintainers/../buildtime.bin'...
**** Time summary:
Compilation (551 times):
  Parsing (frontend):         1302.1 s
  Codegen & opts (backend):    956.3 s

<snip>

**** Expensive headers:
178145 ms: ../src/libutil/error.hh (included 246 times, avg 724 ms), included via:
  36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh
  12x: globals.hh config.hh experimental-features.hh
  11x: file-system.hh file-descriptor.hh
  6x: <direct include>
  6x: serialise.hh strings.hh
  6x: fetchers.hh hash.hh serialise.hh strings.hh
  ...

154043 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 2962 ms), included via:
  40x: command.hh
  5x: command-installable-value.hh
  3x: installable-flake.hh
  2x: <direct include>
  2x: installable-attr-path.hh

153593 ms: ../src/libstore/store-api.hh (included 152 times, avg 1010 ms), included via:
  55x: <direct include>
  39x: command.hh installable-value.hh installables.hh
  7x: libexpr.hh
  4x: local-store.hh
  4x: command-installable-value.hh installable-value.hh installables.hh
  3x: binary-cache-store.hh
  ...

149948 ms: ../src/libutil/types.hh (included 251 times, avg 597 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh
  11x: file-system.hh
  10x: globals.hh
  6x: fetchers.hh
  6x: serialise.hh strings.hh error.hh
  5x: archive.hh
  ...

144560 ms: ../src/libcmd/installables.hh (included 53 times, avg 2727 ms), included via:
  40x: command.hh installable-value.hh
  5x: command-installable-value.hh installable-value.hh
  3x: installable-flake.hh installable-value.hh
  2x: <direct include>
  1x: installable-value.hh
  1x: installable-derived-path.hh
  ...

136585 ms: ../src/libcmd/command.hh (included 51 times, avg 2678 ms), included via:
  42x: <direct include>
  7x: command-installable-value.hh
  2x: installable-attr-path.hh

133394 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 509 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh
  16x: gtest.h
  11x: file-system.hh types.hh ref.hh
  10x: globals.hh types.hh ref.hh
  10x: json.hpp
  6x: serialise.hh
  ...

89315 ms: ../src/libstore/derived-path.hh (included 178 times, avg 501 ms), included via:
  37x: command.hh installable-value.hh installables.hh
  25x: store-api.hh realisation.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh context.hh
  6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh
  6x: libexpr.hh value.hh context.hh
  6x: shared.hh
  ...

87347 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/ostream (included 273 times, avg 319 ms), included via:
  35x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh memory unique_ptr.h
  12x: regex sstream istream
  10x: file-system.hh types.hh ref.hh memory unique_ptr.h
  10x: gtest.h memory unique_ptr.h
  10x: globals.hh types.hh ref.hh memory unique_ptr.h
  6x: fetchers.hh types.hh ref.hh memory unique_ptr.h
  ...

85249 ms: ../src/libutil/config.hh (included 213 times, avg 400 ms), included via:
  37x: command.hh installable-value.hh installables.hh derived-path.hh
  20x: globals.hh
  20x: logging.hh
  16x: store-api.hh logging.hh
  6x: <direct include>
  6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh derived-path.hh
  ...

  done in 0.5s.
```

Adapated from 18aa3e1d57
2024-05-31 13:00:09 +02:00
Robert Hensing
ef5c846e25
Merge pull request #10768 from obsidiansystems/legacy-ssh-expose-ssh-master-for-hydra
Create `CommonSSHStoreConfig::createSSHMaster`
2024-05-29 22:53:29 +02:00
John Ericson
bcdee80a0d More work on the scheduler for windows
- Get a rump derivation goal: hook instance will come later, local
  derivation goal will come after that.

- Start cleaning up the channel / waiting code with an abstraction.
2024-05-28 11:39:49 -04:00
John Ericson
d0c7da131f
Merge pull request #10678 from nix-windows/windows-substitution-goal
Start building the scheduler for Windows
2024-05-27 17:47:29 -04:00
John Ericson
3e9c3738d3 Create CommonSSHStoreConfig::createSSHMaster
By moving `host` to the config, we can do a lot further cleanups and
dedups. This anticipates a world where we always go `StoreReference` ->
`*StoreConfig` -> `Store*` rather than skipping the middle step too.

Progress on #10766

Progress on https://github.com/NixOS/hydra/issues/1164
2024-05-27 16:12:53 -04:00
John Ericson
f71b4da0b3 Factor our connection code for worker proto like serve proto
This increases test coverage, and gets the worker protocol ready to be
used by Hydra.

Why don't we just try to use the store interface in Hydra? Well, the
problem is that the store interface works on connection pools, with each
opreation getting potentially a different connection, but the way temp
roots work requires that we keep one logical "transaction" (temp root
session) using the same connection.

The longer-term solution probably is making connections themselves
implement the store interface, but that is something that builds on
this, so I feel OK that this is not churn in the wrong direction.

Fixes #9584
2024-05-27 00:43:46 -04:00
John Ericson
8ebd99c74e Back in enum values for BuildMode serializer
We don't want to rely on how C assigns numbers for enums in the wire
format. Sure, this is totally determined by the ABI, but it obscures the
code and makes it harder to safely change the enum definition (should we
need to) without accidentally breaking the wire format.
2024-05-27 00:22:55 -04:00
John Ericson
eeb89c28b0 Worker proto use proper serialiser for BuildMode
Do this instead of an unchecked cast

I redid this to use the serialisation framework (including a unit test),
but I am keeping the reference to credit Jade for spotting the issue.

Change-Id: Icf6af7935e8f139bef36b40ad475e973aa48855c
(adapted from commit 2a7a824d83dc5fb33326b8b89625685f283a743b)

Co-Authored-By: Jade Lovelace <lix@jade.fyi>
2024-05-27 00:22:55 -04:00
John Ericson
e0c94b91ee
Merge pull request #10757 from obsidiansystems/fix-4977
Require `drvPath` attribute to end with `.drv`
2024-05-24 12:14:59 -04:00
John Ericson
5384ceacc3 Document field being initialized in Machine constructor 2024-05-23 11:28:25 -04:00
John Ericson
f2bcebc450 Restore exposing machine file parsing
This was accidentally removed in
e989c83b44. I restored it and also did a
few other cleanups:

- Make a static method for namespacing purposes

- Put the test files in the data dir with the other test data

- Avoid mutating globals in the machine config tests

This will be used by Hydra.
2024-05-23 00:03:52 -04:00
John Ericson
4a19f4a866
Merge pull request #9839 from obsidiansystems/more-machine-cleanup
Create `StoreReference` and use it in `Machine`
2024-05-22 17:01:57 -04:00
John Ericson
dc7615dbbb tryUnshareFilesystem: Ignore ENOSYS too
Fixes #10747
2024-05-22 16:07:38 -04:00
John Ericson
d5fdfdc592 unshareFilesystem: Do not assume caller 2024-05-22 16:07:34 -04:00
John Ericson
f923ed6b6a Require drvPath attribute to end with .drv
Fixes #4977
2024-05-22 12:50:24 -04:00
John Ericson
b3ebcc5aad Use the new StoreReference in Machine
This makes the remote builder abstract syntax more robust.
2024-05-22 09:20:15 -04:00
John Ericson
b59a7a14c4 Add StoreReference::render
This will be needed for the next step.

Also allows us to write round trip tests.
2024-05-22 09:20:15 -04:00
John Ericson
c036d75f9e Factor out abstract syntax for Store URIs
Need to decouple parsing from actually opening a store for Machine
configs.

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-22 09:20:15 -04:00
John Ericson
1d6c2316a9 Slightly change formatting style
For long expressions, one argument or parameter per line is just easier.
2024-05-22 09:20:15 -04:00
John Ericson
470c0501eb Ensure all store types support "real" URIs
In particular `local://<path>` and `unix://` (without any path) now
work, and mean the same things as `local` and `daemon`, respectively. We
thus now have the opportunity to desguar `local` and `daemon` early.

This will allow me to make a change to
https://github.com/NixOS/nix/pull/9839 requested during review to
desugar those earlier.

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-05-21 11:56:40 -04:00
John Ericson
8b369f90fd Query path infos (plural) and handshake version minimum for hydra
1. Hydra currently queries for multiple path infos at once, so let us
   make a connection item for that.

2. The minimum of the two versions should always be used, see #9584.
   (The issue remains open because the daemon protocol needs to be
   likewise updated.)
2024-05-20 17:43:18 -04:00
Robert Hensing
67db9e0c64
Merge pull request #10722 from obsidiansystems/ca-obj-docs
Content addressing store objects
2024-05-20 15:58:29 +02:00
John Ericson
1c75af969a Document store object content addressing & improve JSON format
The JSON format no longer uses the legacy ATerm `r:` prefixing nonsese,
but separate fields.

Progress on #9866

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-20 09:34:35 -04:00
Alyssa Ross
979a019014
Improve nix-store --delete failure message
On several occasions I've found myself confused when trying to delete
a store path, because I am told it's still alive, but
nix-store --query --roots doesn't show anything.  Let's save future
users this confusion by mentioning that a path might be alive due to
having referrers, not just roots.
2024-05-17 18:20:22 +02:00
Robert Hensing
303268bb71
Merge pull request #10479 from obsidiansystems/ca-fso-docs
Document file system object content addressing
2024-05-15 22:52:53 +02:00
John Ericson
043135a848 Document file system object content addressing
In addition:

- Take the opportunity to add a bunch more missing hyperlinks, too.

- Remove some glossary entries that are now subsumed by dedicated pages.
  We used to not be able to do this without breaking link fragments, but
  now we can, so pick up where we left off.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-15 16:28:48 -04:00
siddhantCodes
39e8aad446 Merge branch 'master' of github.com:NixOS/nix 2024-05-13 18:43:12 +05:30
siddhantCodes
62e1ea2f4b use path for from arg in nix::copyFile 2024-05-13 16:10:21 +05:30
siddhantCodes
4d0777ca69 fix: copy fileName before calling std::distance 2024-05-13 15:36:00 +05:30
Eelco Dolstra
cf3b044b7e Make large path warnings human-readable 2024-05-13 11:52:38 +02:00
Eelco Dolstra
dbe1b51580 Add setting to warn about copying/hashing large paths
This is useful for diagnosing whether an evaluation is copying large
paths to the store. Example:

   $ nix build .#packages.x86_64-linux.default --large-path-warning-threshold 1000000
   warning: copied large path '/home/eelco/Dev/nix-master/' to the store (6271792 bytes)
   warning: copied large path '«github:NixOS/nixpkgs/b550fe4b4776908ac2a861124307045f8e717c8e?narHash=sha256-7kkJQd4rZ%2BvFrzWu8sTRtta5D1kBG0LSRYAfhtmMlSo%3D»/' to the store (155263768 bytes)
   warning: copied large path '«github:libgit2/libgit2/45fd9ed7ae1a9b74b957ef4f337bc3c8b3df01b5?narHash=sha256-oX4Z3S9WtJlwvj0uH9HlYcWv%2Bx1hqp8mhXl7HsLu2f0%3D»/' to the store (22175416 bytes)
   warning: copied large path '/nix/store/z985088mcd6w23qwdlirsinnyzayagki-source' to the store (5885872 bytes)
2024-05-13 11:52:27 +02:00
siddhantCodes
ccf94545db rename copy -> copyFile and remove old copyFile
the old `copyFile` was just a wrapper that was calling the `copy`
function. This wrapper function is removed and the `copy` function is
renamed to `copyFile`.
2024-05-12 19:20:17 +05:30
siddhantCodes
d3b7367c80 inline usage of nix::getFileType and remove it 2024-05-12 18:58:05 +05:30
siddhantCodes
4537663740 inline the usage of nix::renameFile
use `std::filesystem::rename` everywhere and remove `nix::renameFile`
2024-05-12 18:40:16 +05:30
siddhantCodes
1db7d1b840 inline the usage of nix::readDirectory
`nix::readDirectory` is removed. `std::filesystem::directory_iterator`
is used directly in places that used this util.
2024-05-12 17:42:18 +05:30
John Ericson
39b2a399ad Start building the scheduler for Windows
Building derivations is a lot harder, but the downloading goals is
portable enough.

The "common channel" code is due to Volth. I wonder if there is a way we
can factor it out into separate functions / files to avoid some
within-function CPP.

Co-authored-by: volth <volth@volth.com>
2024-05-10 20:23:59 -04:00
John Ericson
e0ff8da9d5 Build the local store on Windows
Fixes #10558

Co-Authored-By: Eugene Butler <eugene@eugene4.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-05-10 13:05:23 -04:00
John Ericson
0998a3ac01 Remove LocalStore::OptimiseStats::blocksFreed as it is dead code 2024-05-10 12:55:21 -04:00
siddhantCodes
ddea4c6deb rm isLink
isLink util is removed in favour of std::filesystem::is_symlink
2024-05-08 19:59:37 +05:30
Siddhant Kumar
fcbc36cf78
Use std::filesystem::path in more places (#10657)
Progress on #9205

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>

* Get rid of `PathNG`, just use `std::filesystem::path`
2024-05-07 22:28:50 +00:00
John Ericson
72a0d4b022 Try to fix macOS Nixpkgs lib test failure
Sometimes we read a directory with children we cannot stat. It's a pitty
we even try to stat at all (wasteful) in the `DT_UNKNOWN` case, but at
least this should get rid of the failure.
2024-05-07 16:21:02 -04:00
John Ericson
a3c573950b Replace our DirEntry with std::filesystem's 2024-05-07 16:21:02 -04:00
John Ericson
c371070580 Use std::filesystem functions in more places
This makes for shorter and more portable code.

The only tricky part is catching exceptions: I just searched for near by
`catch (Error &)` or `catch (SysError &)` and adjusted them to `catch
(std::filesystem::filesystem_error &)` according to my human judgement.

Good for windows portability; will help @siddhantk232 with his GSOC
project.
2024-05-07 00:16:54 -04:00
Eelco Dolstra
ef28c7329c Rename makeFSSourceAccessor -> getFSSourceAccessor()
This makes it clearer that it returns a shared accessor object.
2024-05-06 19:16:52 +02:00
Eelco Dolstra
eab2919119 Use SourcePath in more places
Now that SourcePath uses a SourceAccessor instead of an InputAccessor,
we can use it in function signatures instead of passing a
SourceAccessor and CanonPath separately.
2024-05-06 19:05:42 +02:00
Théophane Hufschmitt
9bd1191fcc libstore: check additionalSandboxProfile
Make sure that `extraSandboxProfile` is set before we check whether it's
empty or not (in the `sandbox=true` case).

Also adds a test case for this.

Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
2024-05-06 17:37:08 +02:00
Charlie Moog
71c66de227
document store url trusted=true option behavior 2024-05-05 17:37:04 +00:00
John Ericson
037c8d771d Fix format errors
Fix formatting violations, update blacklist to reflect moved files.

PR #10556 passed CI before the new formating rules were added, and our
CI has the race condition of allowing old results, resulting in master
getting broken.
2024-05-02 21:42:28 -04:00
John Ericson
1948ec390c
Merge pull request #10556 from nix-windows/uds-remote-on-windows
Enable the `unix://` store on Windows
2024-05-02 09:53:00 -04:00
Dylan Green
aa165301d1
Pathlocks Implementation for Windows (#10586)
Based on Volth's original port.

Co-authored-by: volth <volth@volth.com>
2024-04-22 15:08:10 +00:00
Robert Hensing
6fd2f42c2d
Merge pull request #10573 from RCoeurjoly/Rename_SearchPath
Rename SearchPath to LookupPath and searchPath to lookupPath
2024-04-21 17:33:46 +02:00
Robert Hensing
e05b58b060 init: Add flag to avoid loading configuration 2024-04-20 01:45:04 +02:00
John Ericson
b973cd494f Enable the unix:// store on Windows
Windows now has some basic Unix Domain Socket support, see
https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/

Building `nix daemon` on Windows I've left for later, because the daemon
currently forks per connection but this is not an option on Windows. But
we can get the client part working right away.
2024-04-18 16:58:32 -04:00
John Ericson
3a3c205fa7 Use rand not random for creating GC root indirect links
I don't think fewer bits matters for this, and `rand` but not `random`
is available on Windows.
2024-04-18 16:57:11 -04:00
Eelco Dolstra
40499bca99
Merge pull request #10549 from alyssais/exportReferencesGraph
Fix exportReferencesGraph when given store subpath
2024-04-18 17:23:12 +02:00
John Ericson
fb9f4208ed
Don't include linux/ in #include
The linux dirs are conditionally added to the `-I` path.
2024-04-18 12:20:20 +02:00
Maximilian Bosch
ba68045187
libstore/local-derivation-goal: prohibit creating setuid/setgid binaries
With Linux kernel >=6.6 & glibc 2.39 a `fchmodat2(2)` is available that
isn't filtered away by the libseccomp sandbox.

Being able to use this to bypass that restriction has surprising results
for some builds such as lxc[1]:

> With kernel ≥6.6 and glibc 2.39, lxc's install phase uses fchmodat2,
> which slips through 9b88e52846/src/libstore/build/local-derivation-goal.cc (L1650-L1663).
> The fixupPhase then uses fchmodat, which fails.
> With older kernel or glibc, setting the suid bit fails in the
> install phase, which is not treated as fatal, and then the
> fixup phase does not try to set it again.

Please note that there are still ways to bypass this sandbox[2] and this is
mostly a fix for the breaking builds.

This change works by creating a syscall filter for the `fchmodat2`
syscall (number 452 on most systems). The problem is that glibc 2.39
and seccomp 2.5.5 are needed to have the correct syscall number available
via `__NR_fchmodat2` / `__SNR_fchmodat2`, but this flake is still on
nixpkgs 23.11. To have this change everywhere and not dependent on the
glibc this package is built against, I added a header
"fchmodat2-compat.hh" that sets the syscall number based on the
architecture. On most platforms its 452 according to glibc with a few
exceptions:

    $ rg --pcre2 'define __NR_fchmodat2 (?!452)'
    sysdeps/unix/sysv/linux/x86_64/x32/arch-syscall.h
    58:#define __NR_fchmodat2 1073742276

    sysdeps/unix/sysv/linux/mips/mips64/n32/arch-syscall.h
    67:#define __NR_fchmodat2 6452

    sysdeps/unix/sysv/linux/mips/mips64/n64/arch-syscall.h
    62:#define __NR_fchmodat2 5452

    sysdeps/unix/sysv/linux/mips/mips32/arch-syscall.h
    70:#define __NR_fchmodat2 4452

    sysdeps/unix/sysv/linux/alpha/arch-syscall.h
    59:#define __NR_fchmodat2 562

I tested the change by adding the diff below as patch to
`pkgs/tools/package-management/nix/common.nix` & then built a VM from
the following config using my dirty nixpkgs master:

    {
      vm = { pkgs, ... }: {
        virtualisation.writableStore = true;
        virtualisation.memorySize = 8192;
        virtualisation.diskSize = 12 * 1024;
        nix.package = pkgs.nixVersions.nix_2_21;
      };
    }

The original issue can be triggered via

    nix build -L github:nixos/nixpkgs/d6dc19adbda4fd92fe9a332327a8113eaa843894#lxc \
      --extra-experimental-features 'nix-command flakes'

however the problem disappears with this patch applied.

Closes #10424

[1] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2031073804
[2] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2030844251
2024-04-18 12:20:20 +02:00
John Ericson
6fa3656a32 Make a few commands that were Unix-only no longer
Also clean up some more linux-specific (`setPersonality`) code in
alignment with recent best practices.
2024-04-17 16:20:56 -04:00
Alyssa Ross
0774e8ba33
Fix exportReferencesGraph when given store subpath
With Nix 2.3, it was possible to pass a subpath of a store path to
exportReferencesGraph:

	with import <nixpkgs> {};

	let
	  hello = writeShellScriptBin "hello" ''
	    echo ${toString builtins.currentTime}
	  '';
	in

	writeClosure [ "${hello}/bin/hello" ]

This regressed with Nix 2.4, with a very confusing error message, that
presumably indicates it was unintentional:

	error: path '/nix/store/3gl7kgjr4pwf03f0x70dgx9ln3bhl7zc-hello/bin/hello' is not in the Nix store
2024-04-17 21:51:59 +02:00
John Ericson
21d9412ddc Improve local-overlay docs in a few ways
In response to https://discourse.nixos.org/t/super-colliding-nix-stores/28462/24
2024-04-17 15:28:19 -04:00
John Ericson
1063aa502a Relax store path canonicalization 2024-04-17 12:26:10 -04:00
John Ericson
b529d91902 Prefix -DNIX_ paths to be windows-complient for windows
This is a hacky solution, but it will do for now.
2024-04-17 12:26:10 -04:00
John Ericson
8433027e35 Build a minimized Nix with MinGW
At this point many features are stripped out, but this works:

- Can run libnix{util,store,expr} unit tests
- Can run some Nix commands

Co-Authored-By volth <volth@volth.com>
Co-Authored-By Brian McKenna <brian@brianmckenna.org>
2024-04-17 12:26:10 -04:00
John Ericson
2248a3f545 Create no-op Window pathlocks implementation
This keeps the call sites simple, eventually this should be filled in.
2024-04-17 11:48:14 -04:00
Théophane Hufschmitt
d2a07a96ba
Merge pull request #10467 from edolstra/nix-shell-symlink
nix shell: Handle output paths that are symlinks
2024-04-16 12:31:14 +02:00
John Ericson
65cc237b3a
Merge pull request #10482 from tweag/fix-symlink-in-sandbox
Fix the access of symlinks to host files in the sandbox
2024-04-15 09:29:00 -04:00
Roland Coeurjoly
40a6a9fdb8 Rename SearchPath to LookupPath and searchPath to lookupPath 2024-04-13 17:35:15 +02:00
Valentin Gagarin
13c2005e7d
add intermediate variables and clarifying comments (#9274)
* add intermediate variables and clarifying comments

Co-authored-by: Alexander Groleau <alex@proof.construction>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-04-12 17:43:35 +02:00
Théophane Hufschmitt
acbb1523c1 Fix the access of symlinks to host files in the sandbox
https://github.com/NixOS/nix/pull/10456 fixed the addition of symlink
store paths to the sandbox, but also made it so that the hardcoded
sandbox paths (like `/etc/hosts`) were now bind-mounted without
following the possible symlinks. This made these files unreadable if
there were symlinks (because the sandbox would now contain a symlink to
an unreachable file rather than the underlying file).
In particular, this broke FOD derivations on NixOS as `/etc/hosts` is a
symlink there.

Fix that by canonicalizing all these hardcoded sandbox paths before
adding them to the sandbox.
2024-04-12 15:57:53 +02:00
John Ericson
5b9cb8b372
Merge pull request #10412 from roberth/c-string-context
C API: Add `nix_string_realise`
2024-04-11 12:07:31 -04:00
Robert Hensing
ed13cf05a2 build-hook: Allow empty
Like always declining; local builds only, as can be inferred from the
docs. (Not worth spending too many words on this pretty obvious
behavior, I think. Also, plans to remove it? https://github.com/NixOS/nix/issues/1221)
2024-04-11 17:39:16 +02:00
Eelco Dolstra
85b9f4ef4f nix shell: Handle output paths that are symlinks
This requires moving resolveSymlinks() into SourceAccessor. Also, it
requires LocalStoreAccessor::maybeLstat() to work on parents of the
store (to avoid an error like "/nix is not in the store").

Fixes #10375.
2024-04-10 23:49:19 +02:00
Théophane Hufschmitt
ae4737294e doBind: Use our own lstat wrapper
Doesn't change much, but brings a bit more consistency to the code
2024-04-10 15:20:10 +02:00
Théophane Hufschmitt
913db9f738 Fix permission denied when building symlink derivation which points to a symlink out of the store
Bind-mounting symlinks is apparently not possible, which is why the
thing was failing.

Fortunately, symlinks are small, so we can fallback to copy them at no cost.

Fix https://github.com/NixOS/nix/issues/9579

Co-authored-by: Artturin <Artturin@artturin.com>
2024-04-10 15:19:18 +02:00
John Ericson
fef952e258
Merge pull request #8397 from NixLayeredStore/overlayfs-store
Local Overlay Store
2024-04-07 22:57:01 -04:00
Robert Hensing
5f9aaa86eb
Merge pull request #10416 from obsidiansystems/cgroup-linux-only
Make `cgroup.{cc,hh}` linux-only files
2024-04-07 16:52:31 +02:00
John Ericson
c99c80f075 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2024-04-05 16:32:02 -04:00
John Ericson
5a365b0c89
Delete dead openFile in binary-cache-store.cc (#10418)
d64cb33e90 / #5111 previously deleted the dead code where this was used, but missed this.
2024-04-05 18:31:43 +00:00
John Ericson
513634ab5b Make cgroup.{cc,hh} linux-only files
Forcing a conditional include, vs making the headers content
conditional, I think is more maintainable.

It is also how the other platform-specific headers (like
`namespaces.hh`) have been adapted.
2024-04-05 12:29:14 -04:00
John Ericson
50f621b241 Better signals interface
This avoids some CPP and accidentally using Unix stuff in client code.
2024-04-05 01:38:28 -04:00
John Ericson
1577b5fa67 Make SQLite busy back-off logic portable
Use C++ standard library not Unix functions for sleeping and randomness.

Suggested by @edolstra in https://github.com/NixOS/nix/pull/8901#discussion_r1550416615
2024-04-04 12:43:33 -04:00
Robert Hensing
12ec3154b8
Merge pull request #8699 from tweag/nix-c-bindings
(Towards) stable C bindings for libutil, libexpr
2024-04-04 17:50:52 +02:00
HaeNoe
50cb14fcf9
Improve checked json casting (#10087)
This introduces new utility functions to get elements from JSON — in an ergonomic way and with nice error messages if the expected type does not match.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-04-03 18:04:00 +00:00
John Ericson
02fa20622f Start factoring out Unix assumptions
This splits files and adds new identifiers in preperation for supporting
windows, but no Windows-specific code is actually added yet.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-04-02 14:43:38 -04:00
John Ericson
852391765d Add unix (and linux) dirs
In the Nix commit, platform-specific sources will go here.
2024-04-02 12:55:23 -04:00
Eelco Dolstra
bb5764e578
Merge pull request #10301 from a-n-n-a-l-e-e/wal-persist
enable persistent WAL mode for sqlite db
2024-04-02 12:02:12 +02:00
John Ericson
9b88e52846
Merge pull request #10362 from obsidiansystems/maybeLstat
Factor out `nix::maybeLstat`
2024-03-30 11:23:29 -04:00
John Ericson
e4d9b207c2 Factor out isRootUser function 2024-03-29 16:33:01 -04:00
John Ericson
8be347afca Factor out nix::maybeLstat
This function is nice for more than `PosixSourceAccessor`. We can make a
few things simpler with it.

Note that the error logic slightly changes in some of the call sites, in
that we also count `ENOTDIR` and not just `ENOENT` as not having the
file, but that should be fine.
2024-03-29 16:32:53 -04:00
José Luis Lafuente
c57de60522
C API: Keep the structure flat
See https://github.com/NixOS/nix/pull/10329
2024-03-28 19:00:04 +01:00
José Luis Lafuente
d96b52bd8b
C api: nix_export_std_string -> nix_observe_string 2024-03-28 10:52:02 +01:00
José Luis Lafuente
31fbb24329
C API: refactor nix_store_realise 2024-03-28 10:52:02 +01:00
José Luis Lafuente
34d15e8f2f
C API: rename nix_store_build -> nix_store_realise 2024-03-28 10:52:01 +01:00
José Luis Lafuente
c49b88b066
C API: update docs based on PR feedback 2024-03-28 10:52:01 +01:00
José Luis Lafuente
24c8f6864d
C API: if store doesn't have a version, return an empty string 2024-03-28 10:52:00 +01:00
José Luis Lafuente
d5ec1d0617
C API: nix_store_open, check for empty strings 2024-03-28 10:51:59 +01:00
José Luis Lafuente
24604d024a
C API: fix docs build after rebase 2024-03-28 10:51:59 +01:00
José Luis Lafuente
41f1669dea
C API: add tests for libutil and libstore 2024-03-28 10:50:02 +01:00
Yorick van Pelt
9e423dee11
C API: update after rebase 2024-03-28 10:47:55 +01:00
Yorick van Pelt
e1bb799da9
C API: reformat according to proposed clang-format file 2024-03-28 10:47:55 +01:00
Yorick van Pelt
9d380c0f76
C API: clarify some documentation 2024-03-28 10:47:55 +01:00
Yorick
40f5d48d3c
Apply documentation suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-03-28 10:47:55 +01:00
Yorick van Pelt
e642bbc2a7
C API: move to src/lib*/c/ 2024-03-28 10:46:39 +01:00
Yorick van Pelt
df9401eb4e
nix_api_store: add nix_init_plugins 2024-03-28 10:39:07 +01:00
Yorick van Pelt
dc0f7d8f96
initPlugins: run nix_plugin_entry() on dlopen'd plugins
Only when it exists.
2024-03-28 10:39:07 +01:00
Yorick van Pelt
b0741f7128
external-api-doc: introduce and improve documentation 2024-03-28 10:39:06 +01:00
Yorick van Pelt
1777e4a5bb
nix_api_store: add userdata param to nix_store_build 2024-03-28 10:39:05 +01:00
Yorick van Pelt
c3b5b8eb62
nix_api_expr, store: fix minor documentation issues 2024-03-28 10:39:05 +01:00
Yorick van Pelt
1d41600498
libstore: add C bindings 2024-03-28 10:39:04 +01:00
John Ericson
77205b2042 Allow for ergnomically putting Unix-only files in subdirs by creating INLCUDE_$(pkg) vars
Separate platform-specific files will allow avoiding a lot of CPP.
2024-03-27 12:01:59 -04:00
annalee
7205a6bbc9
enable persistent WAL mode for sqlite db
allow processes without write access to the directory containing the db
to read the db when all connections are closed. Without this setting and
with WAL enabled and no open db connections unprivileged processes will
fail to open the db due the WAL files not existing and not able to
create them. When the WAL files are persistent unprivileged processeses
can read the db when there are no open connections.

Additionally, journal_size_limit is set to 2^40, which results in the
WAL files being truncated to 0 on exit, as well as limiting the WAL
files to 2^40 bytes following a checkpoint.

https://www.sqlite.org/c3ref/c_fcntl_begin_atomic_write.html#sqlitefcntlpersistwal
https://www.sqlite.org/pragma.html#pragma_journal_size_limit
ed517a7082/src/wal.c (L2518)

Fixes https://github.com/NixOS/nix/issues/10300
2024-03-27 15:36:09 +00:00
Robert Hensing
8b16cced18 Add build-dir setting 2024-03-25 19:48:59 +01:00
Robert Hensing
b1fe388d33 Remove uncalled for message 2024-03-25 19:48:57 +01:00
Robert Hensing
850c9a6caf HttpBinaryCacheStore: Remove *all* trailing slashes 2024-03-24 00:52:14 +01:00
Robert Hensing
dd26f41379 local-derivation-goal.cc: Remove *all* trailing slashes 2024-03-24 00:52:14 +01:00
Robert Hensing
fd31945742 local-derivation-goal.cc: Reuse defaultTempDir() 2024-03-24 00:45:15 +01:00
Robert Hensing
c3fb2aa1f9 fix: Treat empty TMPDIR as unset
Fixes an instance of

    nix: src/libutil/util.cc:139: nix::Path nix::canonPath(PathView, bool): Assertion `path != ""' failed.

... which I've been getting in one of my shells for some reason.
I have yet to find out why TMPDIR was empty, but it's no reason for
Nix to break.
2024-03-24 00:42:31 +01:00
Félix Baylac-Jacqué
40a7929c8e Daemon: warn when an untrusted user cannot override a setting
In a daemon-based Nix setup, some options cannot be overridden by a
client unless the client's user is considered trusted.

Currently, if an untrusted user tries to override one of those
options, we are silently ignoring it.

This can be pretty confusing in certain situations.

e.g. a user thinks he disabled the sandbox when in reality he did not.

We are now sending a warning message letting know the user some options
have been ignored.

Related to #1761.

This is a cherry-pick of 9e0f5f803f.
The above commit has been reverted by
a59e77d9e5 to prevent spamming warnings
with experimental features, but these are now totally ignored on the
daemon side, so there's no reason for the revert any more.
2024-03-20 17:55:35 +01:00
John Ericson
18945e3f44 Merge branch 'master' into overlayfs-store 2024-03-18 16:43:33 -04:00
Théophane Hufschmitt
a3163b9eab Fix the outputs moving on macOS 2024-03-07 14:52:40 +01:00
Eelco Dolstra
da62528487
Merge pull request from GHSA-2ffj-w4mj-pg37
Copy built outputs
2024-03-07 11:56:24 +01:00
Théophane Hufschmitt
fe42a0ead7
Documentation typo 2024-03-06 09:10:32 +01:00
Théophane Hufschmitt
7764edf0e4
Merge pull request #10078 from szlend/fix-macos-local-network-sandbox
Fix `__darwinAllowLocalNetworking` sandbox
2024-03-04 14:34:57 +01:00
Théophane Hufschmitt
c3878f510e Copy the output of fixed-output derivations before registering them
It is possible to exfiltrate a file descriptor out of the build sandbox
of FODs, and use it to modify the store path after it has been
registered.
To avoid that issue, don't register the output of the build, but a copy
of it (that will be free of any leaked file descriptor).
2024-03-01 09:31:28 +01:00
John Ericson
5a2985431c Revert "Revert "Merge pull request #9546 from NixOS/nixos-23.11""
This reverts commit d6d7d2cb46.
2024-02-29 14:52:31 -05:00
John Ericson
d6d7d2cb46 Revert "Merge pull request #9546 from NixOS/nixos-23.11"
This reverts commit 587c7dcb2b, reversing
changes made to 864fc85fc8.
2024-02-29 14:39:29 -05:00
John Ericson
bcd6b33dbc Polish local overlay store docs 2024-02-29 11:58:53 -05:00
John Ericson
cb4f85f11c Merge branch 'master' into overlayfs-store 2024-02-29 09:54:21 -05:00
Eelco Dolstra
65bb12ba78 Fix gcc 12 warnings 2024-02-28 22:59:20 +01:00
Théophane Hufschmitt
da90be789d Fix a too smart implicit cast
Apparently gcc is able to implicitly cast from `FileIngestionMethod` to
`ContentAddressMethod`, but clang isn't. So explicit the cast
2024-02-28 08:00:17 +01:00
John Ericson
d4ad1fcf30 Avoid creating temporary store object for git over the wire
Instead, serialize as NAR and send that over, then rehash sever side.
This is alorithmically simpler, but comes at the cost of a newer
parameter to `Store::addToStoreFromDump`.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-02-27 11:27:34 -05:00
John Ericson
201551c937 Add Git object hashing to the store layer
Part of RFC 133

Extracted from our old IPFS branches.

Co-Authored-By: Matthew Bauer <mjbauer95@gmail.com>
Co-Authored-By: Carlo Nucera <carlo.nucera@protonmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Florian Klink <flokli@flokli.de>
2024-02-27 11:27:34 -05:00
Yueh-Shun Li
fd47f76da9 treewide: hash type -> hash algorithm
"hash type" -> "hash algorithm" in all comments, documentation, and
messages.

ht -> ha, [Hh]ashType -> [HhashAlgo] for all local variables and
function arguments. No API change is made.

Continuation of 5334c9c792 and 837b889c41.
2024-02-26 18:09:06 +08:00
Simon Žlender
d60c3f7f7c Fix __darwinAllowLocalNetworking sandbox
The sandbox rule `(allow network* (local ip))` doesn't do what it
implies. Adding this rule permits all network traffic. We should be
matching on (remote ip "localhost:*")` instead.
2024-02-25 23:00:57 +01:00
Eelco Dolstra
cabee98152 Tarball fetcher: Use the content-addressed Git cache
Backported from the lazy-trees branch.
2024-02-20 12:57:36 +01:00
Eelco Dolstra
2b946aad8e
Merge pull request #10006 from edolstra/fix-impure-fetchurl
<nix/fetchurl.nix>: Restore support for "impure = true"
2024-02-13 22:50:55 +01:00
John Ericson
41dd9857c7 Proper parse and render functions for FileIngestionMethod and ContentAddressMethod
No outward facing behavior is changed.

Older methods with same names that operate on on method + algo pair (for
old-style `<method>:algo`) are renamed to `*WithAlgo`.)

The functions are unit-tested in the same way the names for the hash
algorithms are tested.
2024-02-13 10:30:16 -05:00
Eelco Dolstra
b439b1fc66
Merge pull request #9993 from edolstra/builtin-paths
builtin:{unpack-channel,buildenv}: Get output path from the derivation
2024-02-13 14:57:13 +01:00
Valentin Gagarin
39c353f6fa
reword description of the cores setting (#9522)
* reword description of the `cores` setting

- be precise about the `builder` executable
- clearly distinguish between `builder` and job parallelism
- clarify the role of `mkDerivation` in the example
- remove prose for the default, it's shown programmatically
- mention relation to `max-jobs`
2024-02-13 13:52:51 +00:00
Eelco Dolstra
bb63bd50e6 <nix/fetchurl.nix>: Restore support for "impure = true" 2024-02-13 14:14:20 +01:00
Valentin Gagarin
fd82ba0985
extract reference documentation on remote builds (#9526)
- move all reference documentation to the `builders` configuration setting
- reword documentation on machine specification, add examples
- disable showing the default value, as it rendered as `@/dummy/machines`, which is wrong
- highlight the examples
- link to the configuration docs for distributed builds
- builder -> build machine

Co-authored-by: Janik H <janik@aq0.de>
2024-02-13 14:13:56 +01:00
Valentin Gagarin
e37d502895
add instructions to wipe the substituter lookup cache (#9498)
* add instructions to wipe the substituter lookup cache
2024-02-13 13:34:51 +01:00
Anton Samokhvalov
64cbd4c05a
Update nar-info-disk-cache.cc
fix case when asserts are no-op, like in release build
2024-02-12 23:37:40 +03:00
John Ericson
5169f5f4d9
Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-02-12 12:01:54 -05:00
John Ericson
eb76b35efa Merge remote-tracking branch 'upstream/master' into store-path-complete-construction 2024-02-12 11:22:54 -05:00
Eelco Dolstra
a9b69b2fff builtin:{unpack-channel,buildenv}: Get output path from the derivation
Similar to 1ee42c5b88, get the "out"
path from the derivation (and complain if it doesn't exist), rather
than getting it from the environment.
2024-02-12 16:34:59 +01:00
Eelco Dolstra
619ca631d0 Fix "may be used uninitialized" warning 2024-02-12 15:29:48 +01:00
Eelco Dolstra
fb5a792280 runPostBuildHook(): Be less chatty
Don't spam the user with "running post-build-hook" messages. It's up
to the post-build hook if it has something interesting to say.
2024-02-09 15:55:24 +01:00
Rebecca Turner
c0e7f50c1a
Rename hintfmt to HintFmt 2024-02-08 11:58:25 -08:00
Rebecca Turner
149bd63afb
Cleanup fmt.hh
When I started contributing to Nix, I found the mix of definitions and
names in `fmt.hh` to be rather confusing, especially the small
difference between `hintfmt` and `hintformat`. I've renamed many classes
and added documentation to most definitions.

- `formatHelper` is no longer exported.
- `fmt`'s documentation is now with `fmt` rather than (misleadingly)
  above `formatHelper`.
- `yellowtxt` is renamed to `Magenta`.

  `yellowtxt` wraps its value with `ANSI_WARNING`, but `ANSI_WARNING`
  has been equal to `ANSI_MAGENTA` for a long time. Now the name is
  updated.
- `normaltxt` is renamed to `Uncolored`.
- `hintfmt` has been merged into `hintformat` as extra constructor
  functions.
- `hintformat` has been renamed to `hintfmt`.
- The single-argument `hintformat(std::string)` constructor has been
  renamed to a static member `hintformat::interpolate` to avoid pitfalls
  with using user-generated strings as format strings.
2024-02-08 11:51:03 -08:00
Théophane Hufschmitt
1ba9780cf5
Merge pull request #9834 from 9999years/structured-errors
Towards structured error classes
2024-02-08 20:00:25 +01:00
Théophane Hufschmitt
fb78a99e04
Merge pull request #9924 from 9999years/rename-yellowtxt
Rename `yellowtxt` -> `magentatxt`
2024-02-08 10:01:30 +01:00
Eelco Dolstra
a6737b7e17 CanonPath, SourcePath: Change operator + to /
This is less confusing and makes it more similar to std::filesystem::path.
2024-02-05 15:17:39 +01:00
Rebecca Turner
a7939a6c2a
Rename yellowtxt -> magentatxt
`yellowtxt` wraps its value with `ANSI_WARNING`, but `ANSI_WARNING` has
been equal to `ANSI_MAGENTA` for a long time. Now the name is updated.
2024-02-03 19:28:11 -08:00
Rebecca Turner
c6a89c1a16
libexpr: Support structured error classes
While preparing PRs like #9753, I've had to change error messages in
dozens of code paths. It would be nice if instead of

    EvalError("expected 'boolean' but found '%1%'", showType(v))

we could write

    TypeError(v, "boolean")

or similar. Then, changing the error message could be a mechanical
refactor with the compiler pointing out places the constructor needs to
be changed, rather than the error-prone process of grepping through the
codebase. Structured errors would also help prevent the "same" error
from having multiple slightly different messages, and could be a first
step towards error codes / an error index.

This PR reworks the exception infrastructure in `libexpr` to
support exception types with different constructor signatures than
`BaseError`. Actually refactoring the exceptions to use structured data
will come in a future PR (this one is big enough already, as it has to
touch every exception in `libexpr`).

The core design is in `eval-error.hh`. Generally, errors like this:

    state.error("'%s' is not a string", getAttrPathStr())
      .debugThrow<TypeError>()

are transformed like this:

    state.error<TypeError>("'%s' is not a string", getAttrPathStr())
      .debugThrow()

The type annotation has moved from `ErrorBuilder::debugThrow` to
`EvalState::error`.
2024-02-01 16:39:38 -08:00
Eelco Dolstra
b8b739e484 builtin:fetchurl: Get output hash info from the drv 2024-02-01 22:01:02 +01:00
Eelco Dolstra
1ee42c5b88 builtin:fetchurl: Ensure a fixed-output derivation
Previously we didn't check that the derivation was fixed-output, so
you could use builtin:fetchurl to impurely fetch a file.
2024-02-01 21:46:01 +01:00
John Ericson
dc439eaf23 Fill in missing markdown link dest 2024-02-01 11:20:19 -05:00
John Ericson
31881d651a Merge remote-tracking branch 'upstream/master' into overlayfs-store 2024-02-01 11:07:47 -05:00
Eelco Dolstra
ef6d055ace
Merge pull request #9896 from hercules-ci/tidy-tidy
Disable a slightly annoying clang-tidy check
2024-02-01 09:02:46 +01:00
Robert Hensing
0f2e9e6bd2 Typo 2024-02-01 01:01:04 +01:00
Robert Hensing
4072a8fea0
Merge pull request #9867 from hercules-ci/issue-912
#912 allow leading period
2024-01-31 19:10:59 +01:00
Robert Hensing
f1b4663805 Disallow store path names that are . or .. (plus opt. -)
As discussed in the maintainer meeting on 2024-01-29.

Mainly this is to avoid a situation where the name is parsed and
treated as a file name, mostly to protect users.
.-* and ..-* are also considered invalid because they might strip
on that separator to remove versions. Doesn't really work, but that's
what we decided, and I won't argue with it, because .-* probably
doesn't seem to have a real world application anyway.
We do still permit a 1-character name that's just "-", which still
poses a similar risk in such a situation. We can't start disallowing
trailing -, because a non-zero number of users will need it and we've
seen how annoying and painful such a change is.

What matters most is preventing a situation where . or .. can be
injected, and to just get this done.
2024-01-31 18:35:19 +01:00
Robert Hensing
40254092dd
Merge pull request #9770 from hercules-ci/refactor-rename-derivation-isPure
Refactor rename derivation type `isPure`
2024-01-27 11:24:20 +01:00
Robert Hensing
9ddd0f2af8 Revert "StorePath: reject names starting with '.'"
This reverts commit 24bda0c7b3.
2024-01-27 11:18:03 +01:00
Robert Hensing
49b25ea85c refactor: Impure derivation type isPure -> isImpure
To quote the method doc:

Non-impure derivations can still behave impurely, to the degree permitted
by the sandbox. Hence why this method isn't `isPure`: impure derivations
are not the negation of pure derivations. Purity can not be ascertained
except by rather heavy tools.
2024-01-27 11:00:10 +01:00
John Ericson
a9e10a1dbd Make StoreConfig::getDefaultSystemFeatures a static method
This makes something in Hydra bit simpler. If someday the default
depends on the other config options, we can always change it back.
2024-01-24 21:34:31 -05:00
John Ericson
1e24db6f9a Convert Machine::speedFactor from a non-neg int to a non-neg float
The short motivation is to match Hydra, so we can de-dup.

The long version is layed out in
https://github.com/NixOS/nix/issues/9840.
2024-01-24 01:03:07 -05:00
John Ericson
0aa85088de Factor out Machine::systemSupported
There's just enough logic (the `"builtin"` special case) that makes this
worthy of its own method.
2024-01-23 12:53:51 -05:00
John Ericson
870acc2892 Add API docs to Machine methods 2024-01-23 12:50:48 -05:00
John Ericson
739032762a Make Machine::systemTypes a set not vector
This is more conceptually correct (the order does not matter), and also
matches what Hydra already does.

(Nix and Hydra matching is needed for dedup
https://github.com/NixOS/hydra/issues/1164)
2024-01-23 12:30:26 -05:00
Eelco Dolstra
b6aee9a93f
Merge pull request #9833 from obsidiansystems/ssh-arg-split
Make `SSHMaster::startCommand` work on an args list
2024-01-23 14:34:41 +01:00
John Ericson
6365bbfa81 Improve the FileSystemObjectSink interface
More invariants are enforced in the type, and less state needs to be
stored in the main sink itself. The method here is roughly that known as
"session types".

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 18:01:21 -05:00
John Ericson
966d6fcd01 ParseSink -> FileSystemObjectSink
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 18:01:18 -05:00
John Ericson
b71673109c Make SSHMaster::startCommand work on an args list
This avoids split-on-whitespace errors:

- No more `bash -c` needed

- No more `shellEscape` needed

- `remote-program` ssh store setting also cleanly supports args (e.g.
  `nix daemon`)

- `ssh` uses `--` to separate args for SSH from args for the command to
  run.

and will help with Hydra dedup.

Some code taken from #6628.

Co-Authored-By: Alexander Bantyev <balsoft@balsoft.ru>
2024-01-22 17:46:57 -05:00
John Ericson
e960b28230 Factor our ServeProto::BasicServerConnection::handshake
We'll need this for unit testing.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 12:43:11 -05:00
John Ericson
4a5ca576da Factor out ServeProto::BasicClientConnection::handshake Hydra to share 2024-01-22 12:43:11 -05:00
John Ericson
4580bed3e4 LegacySSHStore::openConnection move more logic inside catch block
Broader error handling logic is more robust.
2024-01-22 12:43:11 -05:00
John Ericson
ce2f714e6d Start factoring out the serve protocol for Hydra to share
Factor out `ServeProto::BasicClientConnection` for Hydra to share

- `queryValidPaths`: Hydra uses the lock argument differently than Nix,
  so we un-hard-code it.

- `buildDerivationRequest`: Just the request half, as Hydra does some
  things between requesting and responding.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 12:43:03 -05:00
John Ericson
b2066c3574
Merge pull request #9521 from fricklerhandwerk/conf-reword-max-jobs
reword description of the `max-jobs` setting
2024-01-22 08:29:12 -05:00
John Ericson
edf3ecc497 Document JSON formats
Good to document these formats separately from commands that happen to
use them.

Eventually I would like this and `builtins.derivation` to refer to a
store section on derivations that is authoritative, but that doesn't yet
exist, and will take some time to make. So I think we're just best off
merging this now as is.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-01-20 17:03:47 -05:00
Robert Hensing
382fa51ff0
Merge pull request #9794 from hercules-ci/queryPathInfoFromClientCache
refactor: Extract `Store::queryPathInfoFromClientCache`
2024-01-19 17:33:52 +01:00
Robert Hensing
d19627e8b4 refactor: Remove throw from queryPathInfoFromClientCache
Return a value instead of throwing.

Rather than the more trivial refactor of wrapping the return value in
another std::optional, we retain the meaning of the outer optional:
"we know at least something."

So we have changed:
return nullopt    -> return nullopt
throw InvalidPath -> return make_optional(nullptr)
return vpi        -> return make_optional(vpi)
2024-01-19 17:01:19 +01:00
Robert Hensing
8983ee8b2e refactor: Un-callback transform Store::queryPathInfoFromClientCache
This part of the code was not necessarily callback based.
Removing CPS is always nice; particularly if there's no loss of
functionality, like here.
2024-01-19 17:01:19 +01:00
John Ericson
a34ec0bd12 Include store path exact spec in the docs
This is niche, but deserves to be in the manual because it is describing
behavior visible to the outside world, not mere implementation details.
2024-01-18 22:19:14 -05:00
Eelco Dolstra
dca0a80240 copyStorePath(): Bail out early if the store path already exists
In rare cases (e.g. when using allowSubstitutes = false), it's
possible that we simultaneously have a DerivationGoal *and* a
SubstitutionGoal building the same path. So if a DerivationGoal
already built the path while the SubstitutionGoal was waiting for a
download slot, it saves us a superfluous download to exit early.
2024-01-18 17:19:15 +01:00
Eelco Dolstra
a18d8d688a LocalStore::addToStore(): Ignore exceptions from parseDump()
In the "discard" case (i.e. when the store path already exists
locally), when we call parseDump() from a Finally and it throws an
exception (e.g. if the download of the NAR fails), Nix crashes:

   terminate called after throwing an instance of 'nix::SubstituteGone'
     what():  error: file 'nar/06br3254rx4gz4cvjzxlv028jrx80zg5i4jr62vjmn416dqihgr7.nar.xz' does not exist in binary cache 'http://localhost'
   Aborted (core dumped)
2024-01-18 17:19:15 +01:00
Eelco Dolstra
ab786e22f1 Show what goal is waiting for a build slot 2024-01-18 17:19:15 +01:00
Théophane Hufschmitt
e652322a61
Merge pull request #9799 from obsidiansystems/push-add-to-store-from-dump-unsupported-down
Push `addToStoreFromDump` `unsupported(...)` down `Store` class hierarchy
2024-01-18 16:54:44 +01:00
John Ericson
574db83504 Push addToStoreFromDump unsupported(...) down Store class hierarchy
Instead of having it be the default method in `Store` itself, have it be
the implementation in `DummyStore` and `LegacySSHStore`. Then just the
implementations which fail to provide the method pay the "penalty" of
dealing with the icky `unimplemented` function for non-compliance.

Picks up where #8217. Getting close to no `unsupported` in the `Store`
interface itself!

More progress on issue #5729.
2024-01-17 23:46:03 -05:00
Eelco Dolstra
a3cf27ca47 Print a more helpful message if the daemon crashes
Instead of

   error: unexpected end-of-file

you now get

   error: Nix daemon disconnected unexpectedly (maybe it crashed?)
2024-01-17 22:54:02 +01:00
Eelco Dolstra
39ab50f9ee Store::buildPaths(): Fix display of store paths
This was broken in 7ac39ff05c.
2024-01-17 21:41:37 +01:00
Robert Hensing
d0a284284b refactor: Extract simply, awkwardly Store::queryPathInfoFromClientCache
This is useful for determining quickly which substituters to query.

An alternative would be for users to invoke the narinfo cache db directly,
so why do we need this change?

 - It is easier to use. I believe Nix itself should also use it.

 - This way, the narinfo cache db remains an implementation detail.

 - Callers get to use the in-memory cache as well.
2024-01-17 17:54:19 +01:00
Robert Hensing
01271f2996
Merge pull request #9786 from hercules-ci/package
DerivationInfo -> PackageInfo
2024-01-16 22:43:14 +01:00
Robert Hensing
ea6aa5ffd8 Package{,Info}: comments 2024-01-16 15:44:02 +01:00
Eelco Dolstra
0b1d93d2ba Sleep a bit between attempts to connect to the root server 2024-01-16 15:26:17 +01:00
Eelco Dolstra
302625e83b Merge remote-tracking branch 'origin/master' into handle-missing-gc-socket 2024-01-16 13:18:58 +01:00
Las Safin
f61d951909
Avoid unnecessary copy of goal log
The data was (accidentally?) copied into a std::string,
even though the string is immediately converted into a std::string_view.
The code has been changed to construct a std::string_view directly,
such that one copy less happens.
2024-01-13 19:32:37 +00:00
John Ericson
6208ca7209 Separate SystemError from SysError
Most of this is a `catch SysError` -> `catch SystemError` sed. This
is a rather pure-churn change I would like to get out of the way. **The
intersting part is `src/libutil/error.hh`.**

On Unix, we will only throw the `SysError` concrete class, which has
the same constructors that `SystemError` used to have.

On Windows, we will throw `WinError` *and* `SysError`. `WinError`
(which will be created in a later PR), will use a `DWORD` instead of
`int` error value, and `GetLastError()`, which is the Windows equivalent
of the `errno` machinery. Windows will *also* use `SysError` because
Window's "libc" (MSVCRT) implements the POSIX interface, and we use it
too.

As the docs describe, while we *throw* one of the 3 choices above (2
concrete classes or the alias), we should always *catch* `SystemError`.
This ensures no matter how the implementation changes for Windows (e.g.
between `SysError` and `WinError`) the catching logic stays the same
and stays correct.

Co-Authored-By volth <volth@volth.com>
Co-Authored-By Eugene Butler <eugene@eugene4.com>
2024-01-12 12:00:33 -05:00
Eelco Dolstra
3e23759834 gc-non-blocking.sh: Add explanation
Also name the _NIX_TEST_GC_SYNC environment variables logically.
2024-01-12 12:38:55 +01:00
Eelco Dolstra
5703c31325 Merge remote-tracking branch 'origin/master' into handle-missing-gc-socket 2024-01-12 12:26:25 +01:00
John Ericson
84502674f8
Merge pull request #9736 from obsidiansystems/mingw-makefiles
Some small Makefile improvements to prepare for Windows support
2024-01-11 10:58:55 -05:00
John Ericson
423484ad26 Only link with -pthread on Unix
We don't want this with MinGW.
2024-01-10 20:38:39 -05:00
John Ericson
f9e5eb5f0a Make indentation in makesfiles consistent
Tab (as required) for rules, two spaces for `if`...`endif`.
2024-01-10 20:26:34 -05:00
Rebecca Turner
4feb7d9f71
Combine AbstractPos, PosAdapter, and Pos
Also move `SourcePath` into `libutil`.

These changes allow `error.hh` and `error.cc` to access source path and
position information, which we can use to produce better error messages
(for example, we could consider omitting filenames when two or more
consecutive stack frames originate from the same file).
2024-01-08 10:59:41 -08:00
John Ericson
ff6de4a9ee
Merge pull request #9662 from shlevy/flat-fixed-references-assert
Improve error message for fixed-outputs with references.
2024-01-08 10:46:37 -05:00
Eelco Dolstra
faf87b51f7
Show why GC socket connection was refused
Co-authored-by: John Ericson <git@JohnEricson.me>
2024-01-08 14:14:36 +01:00
Shea Levy
eeb2f083c5
Improve error message for fixed-outputs with references.
This codepath is possible, e.g. with a dockerTools.pullImage of an image with a Nix store.
2024-01-07 07:32:31 -05:00
Eelco Dolstra
dedbbbb451
Merge pull request #9670 from DavHau/log-lines
saner default for log-lines: change to 25
2024-01-05 14:39:50 +01:00
Eelco Dolstra
965cfe9688
Merge pull request #9687 from edolstra/withFramedSink-ctrl-c-hang
withFramedSink(): Receive interrupts on the stderr thread
2024-01-04 17:05:18 +01:00
John Ericson
12bb8cdd38 Signer infrastructure: Prep for #9076
This sets up infrastructure in libutil to allow for signing other than
by a secret key in memory. #9076 uses this to implement remote signing.

(Split from that PR to allow reviewing in smaller chunks.)

Co-Authored-By: Raito Bezarius <masterancpp@gmail.com>
2024-01-03 16:13:55 -05:00
Eelco Dolstra
295a2ff8bd Make some more threads receive interrupts
Shouldn't hurt to do this. In particular, this should speed up
shutting down the PathSubstitutionGoal thread if it's copying from a
remote store.
2024-01-03 19:30:02 +01:00
Eelco Dolstra
24e70489e5 withFramedSink(): Receive interrupts on the stderr thread
Otherwise Nix deadlocks when Ctrl-C is received in withFramedSink():
the parent thread will wait forever for the stderr thread to shut
down.

Fixes the hang reported in https://github.com/NixOS/nix/issues/7245#issuecomment-1770560923.
2024-01-03 19:14:50 +01:00
John Ericson
2b20f36f95 Fix NetBSD build
There was still a mistake after my earlier
a7115a47ef and
e13fc0bbdb. This finally gets it right.
2024-01-02 12:33:51 -05:00
Robert Hensing
3511430902
Merge pull request #9673 from pennae/drv-parse-opts
optimize derivation parsing
2023-12-31 13:49:03 +01:00
pennae
c62686a95b reduce copies during drv parsing
many paths need not be heap-allocated, and derivation env name/valye
pairs can be moved into the map.

before:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.883 s ±  0.016 s    [User: 5.250 s, System: 1.424 s]
  Range (min … max):    6.860 s …  6.905 s    10 runs

after:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.868 s ±  0.027 s    [User: 5.194 s, System: 1.466 s]
  Range (min … max):    6.828 s …  6.913 s    10 runs
2023-12-30 18:44:15 +01:00
pennae
02c64abf1e use translation table for drv string parsing
the table is very small compared to cache sizes and a single indexed
load is faster than three comparisons.

before:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.907 s ±  0.012 s    [User: 5.272 s, System: 1.429 s]
  Range (min … max):    6.893 s …  6.926 s    10 runs

after:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.883 s ±  0.016 s    [User: 5.250 s, System: 1.424 s]
  Range (min … max):    6.860 s …  6.905 s    10 runs
2023-12-30 18:44:15 +01:00
pennae
79d3d412ca optimize derivation string parsing
a bunch of derivation strings contain no escape sequences. we can
optimize for this fact by first scanning for the end of a derivation
string and simply returning the contents unmodified if no escape
sequences were found. to make this even more efficient we can also use
BackedStringViews to avoid copies, avoiding heap allocations for
transient data.

before:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.952 s ±  0.015 s    [User: 5.294 s, System: 1.452 s]
  Range (min … max):    6.926 s …  6.974 s    10 runs

after:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.907 s ±  0.012 s    [User: 5.272 s, System: 1.429 s]
  Range (min … max):    6.893 s …  6.926 s    10 runs
2023-12-30 18:44:10 +01:00
pennae
99a691c8a1 don't use istreams in hot paths
istream sentry objects are very expensive for single-character
operations, and since we don't configure exception masks for the
istreams used here they don't even do anything. all we need is
end-of-string checks and an advancing position in an immutable memory
buffer, both of which can be had for much cheaper than istreams allow.

the effect of this change is most apparent on empty stores.

before:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      7.167 s ±  0.013 s    [User: 5.528 s, System: 1.431 s]
  Range (min … max):    7.147 s …  7.182 s    10 runs

after:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.963 s ±  0.011 s    [User: 5.330 s, System: 1.421 s]
  Range (min … max):    6.943 s …  6.974 s    10 runs
2023-12-30 00:45:10 +01:00
DavHau
b6313f64f7 saner default for log-lines: change to 25
This seems to be a much saner default. 10 lines are just not enough in so many cases.
2023-12-27 19:57:27 +07:00
Brian Le
e2399fc949 Change "dervation" typos to "derivation" 2023-12-26 17:12:28 -05:00
Robert Hensing
ee439734e9
Merge pull request #9582 from pennae/misc-opts
a packet of small optimizations
2023-12-22 17:00:59 +01:00
pennae
cc4038d541 use std::tie() for macro-generated operators
as written the comparisons generate copies, even though it looks as
though they shouldn't.

before:

  Time (mean ± σ):      4.396 s ±  0.002 s    [User: 3.894 s, System: 0.501 s]
  Range (min … max):    4.393 s …  4.399 s    10 runs

after:

  Time (mean ± σ):      4.260 s ±  0.003 s    [User: 3.754 s, System: 0.505 s]
  Range (min … max):    4.257 s …  4.266 s    10 runs
2023-12-19 19:32:16 +01:00
pennae
0218e4e6c3 memset less in addToStoreFromDump
resizing a std::string clears the newly added bytes, which is not
necessary here and comes with a ~1.4% slowdown on our test nixos config.

〉 nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'

before:

  Time (mean ± σ):      4.486 s ±  0.003 s    [User: 3.978 s, System: 0.507 s]
  Range (min … max):    4.482 s …  4.492 s    10 runs

after:

  Time (mean ± σ):      4.429 s ±  0.002 s    [User: 3.929 s, System: 0.500 s]
  Range (min … max):    4.427 s …  4.433 s    10 runs
2023-12-19 19:32:16 +01:00
Eelco Dolstra
7cfd6c0efe
Merge pull request #9325 from NixOS/accessor-add-to-store
Content addressing and adding to store cleanup
2023-12-19 15:10:31 +01:00
Andrew Marshall
7526b7ded6 Allow access to /dev/stderr in Darwin sandbox
We allow /dev/stdout, so why not this? Since it is process-local,
anyway, should not be possible to escape sandbox using it.
2023-12-18 19:33:20 -05:00
John Ericson
ed26b186fb Remove now-redundant text-hashing store methods
`addTextToStore` and `computeStorePathFromDump` are now redundant.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:44:10 -05:00
John Ericson
dfc876531f Organize content addressing, use SourceAccessor with Store::addToStore
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:41:54 -05:00
Valentin Gagarin
4f95800854 add cross-reference 2023-12-18 11:41:52 +01:00
Matthew Bauer
bcbdb09ccf Add eval-system option
`eval-system` option overrides just the value of `builtins.currentSystem`.
This is more useful than overriding `system` since you can build these
derivations on remote builders which can work on the given system.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-14 19:04:00 -05:00
John Ericson
e13fc0bbdb Fix sys/xattr.h check
I wrote the `configure.ac` wrong, and so we just got no builds
supporting ACLs.

Also, it needs to be more precise because Darwin puts other stuff in
that same header, evidently.
2023-12-14 10:03:48 -05:00
Ramses
1e3d811840
worker protocol: serialise cgroup stats in BuildResult (#9598)
By doing so, they get reported when building through the daemon via either `unix://` or `ssh-ng://`.
2023-12-13 16:37:17 -05:00
Théophane Hufschmitt
b1c633c6bb
Merge pull request #9600 from SharzyL/fix_nix_copy
fix: nix copy ssh-ng:// not respecting --substitute-on-destination
2023-12-13 18:08:38 +01:00
SharzyL
04f454f2a0
fix: nix copy ssh-ng:// not respecting --substitute-on-destination 2023-12-13 10:30:28 +08:00
tomberek
7026abfdde
Merge pull request #9523 from fricklerhandwerk/conf-reword-always-allow-substitutes
reword documentation on settings and attributes related to substitution
2023-12-12 20:09:48 -05:00
tomberek
09041071bf
Merge pull request #9525 from fricklerhandwerk/conf-reword-builders-use-substitutes
reword description of the `builders-use-substitutes` setting
2023-12-12 20:08:00 -05:00
Ben Radford
4a2cee8e6c
Document expected filesystem layout and OverlayFS mount command. 2023-12-11 18:55:39 +00:00
John Ericson
b3bdd70ea2 Clarify toUpperPath docs
We're just mapping store paths to host OS paths, there is no checking
what is actually at this location.
2023-12-11 13:43:17 -05:00
John Ericson
b21ee60594 Get rid of verifyAllValidPaths boolean blindness 2023-12-11 13:28:40 -05:00