Since withFramedSink() is now used a lot more than in the past (for
every addToStore() variant), we were creating a lot of threads, e.g.
nix flake show --no-eval-cache --all-systems github:NixOS/nix/afdd12be5e19c0001ff3297dea544301108d298
would create 46418 threads. While threads on Linux are cheap, this is
still substantial overhead.
So instead, just poll from FramedSink before every write whether there
are pending messages from the daemon. This could slightly increase the
latency on log messages from the daemon, but not on exceptions (which
were only synchronously checked from FramedSink anyway).
This speeds up the command above from 19.2s to 17.5s on my machine (a
9% speedup).
* add cross-references to `nix-path` overriding
while this information is already present in the settings, it's more
likely to be first accessed through the "lookup path" page, which
currently requires following two links to get to the practically
important bits.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
This is because with the split packages of the Meson build, we simply
have no idea what directory the binaries will be installed in when we
build the library.
In the process of doing so, consolidate and make more sophisticated the
logic to cope with a few corner cases (e.g. `NIX_BIN_DIR` exists, but no
binaries are inside it).
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
In d60c3f7f7c, this was changed to close a
hole in the sandbox. Unfortunately, this was too restrictive such that it
made local port binding fail, thus making derivations that needed
`__darwinAllowLocalNetworking` gain nearly nothing, and thus largely
fail (as the primary use for it is to enable port binding).
This unfortunately does mean that a sandboxed build process can, in
coordination with an actor outside the sandbox, escape the sandbox by
binding a port and connecting to it externally to send data. I do not
see a way around this with my experimentation and understanding of the
(quite undocumented) macOS sandbox profile API. Notably it seems not
possible to use the sandbox to do any of:
- Restrict the remote IP of inbound network requests
- Restrict the address being bound to
As such, the `(local ip "*:*")` here appears to be functionally no
different than `(local ip "localhost:*")` (however it *should* be
different than removing the filter entirely, as that would make it also
apply to non-IP networking). Doing `(allow network-inbound (require-all
(local ip "localhost:*") (remote ip "localhost:*")))` causes listening
to fail.
Note that `network-inbound` implies `network-bind`.
This ended up motivating a good deal of other infra improvements in
order to get Windows right:
- `OsString` to complement `std::filesystem::path`
- env var code for working with the underlying `OsString`s
- Rename `PATHNG_LITERAL` to `OS_STR`
- `NativePathTrait` renamed to `OsPathTrait`, given a character template
parameter until #9205 is complete.
Split `tests.cc` matching split of `util.{cc,hh}` last year.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
The test split matches PR #8920, so the utility files and tests files
are once again to 1-1. The string changes continues what was started in
PR #11093.
In the FFI world we have many tools that are not gcc/clang and therefore
not always support the latest C standard. This fixes support with cffi
i.e. used in https://github.com/tweag/python-nix
It was failing with:
error: AWS error fetching 'nix-cache-info': The specified bucket does not exist
because `S3BinaryCacheStoreImpl` had a `bucketName` field that
shadowed the inherited `bucketName from `S3BinaryCacheStoreConfig`.
We didn't even realize you *could* use this syntax with -E and -f, much
less that the attribute path could be *empty*.
Change-Id: Id1a6715609f3a76a5ce477bd43a7832effbbe07b
* docs: unify documentation on search paths
- put all the information on search path semantics into `builtins.findFile`
- put all the information on determining the value of `builtins.nixPath` into the
`nix-path` setting
maybe `builtins.nixPath` is a better place for this, but those bits
can still be moved around now that it's all next to each other.
- link to the syntax page for lookup paths from all places that are
concerned with it
- add or clarify examples
- add a test verifying a claim from documentation
This also bans various sneaking of negative numbers from the language
into unsuspecting builtins as was exposed while auditing the
consequences of changing the Nix language integer type to a newtype.
It's unlikely that this change comprehensively ensures correctness when
passing integers out of the Nix language and we should probably add a
checked-narrowing function or something similar, but that's out of scope
for the immediate change.
During the development of this I found a few fun facts about the
language:
- You could overflow integers by converting from unsigned JSON values.
- You could overflow unsigned integers by converting negative numbers
into them when going into Nix config, into fetchTree, and into flake
inputs.
The flake inputs and Nix config cannot actually be tested properly
since they both ban thunks, however, we put in checks anyway because
it's possible these could somehow be used to do such shenanigans some
other way.
Note that Lix has banned Nix language integer overflows since the very
first public beta, but threw a SIGILL about them because we run with
-fsanitize=signed-overflow -fsanitize-undefined-trap-on-error in
production builds. Since the Nix language uses signed integers, overflow
was simply undefined behaviour, and since we defined that to trap, it
did.
Trapping on it was a bad UX, but we didn't even entirely notice
that we had done this at all until it was reported as a bug a couple of
months later (which is, to be fair, that flag working as intended), and
it's got enough production time that, aside from code that is IMHO buggy
(and which is, in any case, not in nixpkgs) such as
https://git.lix.systems/lix-project/lix/issues/445, we don't think
anyone doing anything reasonable actually depends on wrapping overflow.
Even for weird use cases such as doing funny bit crimes, it doesn't make
sense IMO to have wrapping behaviour, since two's complement arithmetic
overflow behaviour is so *aggressively* not what you want for *any* kind
of mathematics/algorithms. The Nix language exists for package
management, a domain where bit crimes are already only dubiously in
scope to begin with, and it makes a lot more sense for that domain for
the integers to never lose precision, either by throwing errors if they
would, or by being arbitrary-precision.
Fixes: https://github.com/NixOS/nix/issues/10968
Original-CL: https://gerrit.lix.systems/c/lix/+/1596
Change-Id: I51f253840c4af2ea5422b8a420aa5fafbf8fae75
The actual motive here is the avoidance of integer overflow if we were
to make these use checked NixInts and retain the subtraction.
However, the actual *intent* of this code is a three-way comparison,
which can be done with operator<=>, so we should just do *that* instead.
Change-Id: I7f9a7da1f3176424b528af6d1b4f1591e4ab26bf
