Commit graph

224 commits

Author SHA1 Message Date
Robert Hensing
18485d2d53
Merge pull request #11188 from lf-/jade/kill-int-overflow
Ban integer overflow in the Nix language
2024-08-11 04:24:16 +02:00
John Ericson
6c861b9c51 Factor out lookupExecutable and other PATH improvments
This ended up motivating a good deal of other infra improvements in
order to get Windows right:

- `OsString` to complement `std::filesystem::path`

- env var code for working with the underlying `OsString`s

- Rename `PATHNG_LITERAL` to `OS_STR`

- `NativePathTrait` renamed to `OsPathTrait`, given a character template
  parameter until #9205 is complete.

Split `tests.cc` matching split of `util.{cc,hh}` last year.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-08-07 18:12:58 -04:00
John Ericson
9d2d4d11e6 Split tests, organize more string functions
The test split matches PR #8920, so the utility files and tests files
are once again to 1-1. The string changes continues what was started in
PR #11093.
2024-08-05 12:50:13 -04:00
Eelco Dolstra
ed0934b884
Merge pull request #11140 from DeterminateSystems/protocol-features
WorkerProto: Support fine-grained protocol feature negotiation
2024-07-31 17:47:38 +02:00
Jade Lovelace
7b6622d733 language: cleanly ban integer overflows
This also bans various sneaking of negative numbers from the language
into unsuspecting builtins as was exposed while auditing the
consequences of changing the Nix language integer type to a newtype.

It's unlikely that this change comprehensively ensures correctness when
passing integers out of the Nix language and we should probably add a
checked-narrowing function or something similar, but that's out of scope
for the immediate change.

During the development of this I found a few fun facts about the
language:
- You could overflow integers by converting from unsigned JSON values.
- You could overflow unsigned integers by converting negative numbers
  into them when going into Nix config, into fetchTree, and into flake
  inputs.

  The flake inputs and Nix config cannot actually be tested properly
  since they both ban thunks, however, we put in checks anyway because
  it's possible these could somehow be used to do such shenanigans some
  other way.

Note that Lix has banned Nix language integer overflows since the very
first public beta, but threw a SIGILL about them because we run with
-fsanitize=signed-overflow -fsanitize-undefined-trap-on-error in
production builds. Since the Nix language uses signed integers, overflow
was simply undefined behaviour, and since we defined that to trap, it
did.

Trapping on it was a bad UX, but we didn't even entirely notice
that we had done this at all until it was reported as a bug a couple of
months later (which is, to be fair, that flag working as intended), and
it's got enough production time that, aside from code that is IMHO buggy
(and which is, in any case, not in nixpkgs) such as
https://git.lix.systems/lix-project/lix/issues/445, we don't think
anyone doing anything reasonable actually depends on wrapping overflow.

Even for weird use cases such as doing funny bit crimes, it doesn't make
sense IMO to have wrapping behaviour, since two's complement arithmetic
overflow behaviour is so *aggressively* not what you want for *any* kind
of mathematics/algorithms. The Nix language exists for package
management, a domain where bit crimes are already only dubiously in
scope to begin with, and it makes a lot more sense for that domain for
the integers to never lose precision, either by throwing errors if they
would, or by being arbitrary-precision.

Fixes: https://github.com/NixOS/nix/issues/10968
Original-CL: https://gerrit.lix.systems/c/lix/+/1596

Change-Id: I51f253840c4af2ea5422b8a420aa5fafbf8fae75
2024-07-30 18:13:05 -07:00
Jade Lovelace
e28cb67d41 libutil: add checked arithmetic tools
This is in preparation for adding checked arithmetic to the evaluator.

Change-Id: I6e115ce8f5411feda1706624977a4dcd5efd4d13
2024-07-30 18:13:05 -07:00
Pino Toscano
a1ccf60613 tests: define fallback PATH_MAX
Few filesystem-related tests rely on PATH_MAX for buffers, and PATH_MAX
is optional in POSIX (and not available on the Hurd). To make them build
and pass, provide a fallback definition of PATH_MAX in case not
available.

Ideally speaking, the tests ought to not unconditionally rely on
PATH_MAX, do alternative strategies (e.g. dynamically allocate buffers,
expand them as needed, etc); OTOH this is test code, so it would be more
work that what it would be worth, so IMHO the define fallback is good
enough.
2024-07-30 05:34:34 +02:00
Eelco Dolstra
e0012b97ab Split tarball-specific logic from GitFileSystemObjectSink 2024-07-29 14:26:25 +02:00
Eelco Dolstra
5e83c0427f Fix test 2024-07-26 20:46:07 +02:00
Robert Hensing
6ec123ad6b
Merge pull request #11131 from rhendric/rhendric/pipe-operators
libexpr: experimental pipe operators
2024-07-25 16:58:43 +02:00
John Ericson
3b49f7a143
Deduplicate our many package.nix a bit (#11175)
- They should all be built in parallel

- They should all use strict deps by default
2024-07-25 03:12:39 +00:00
Ryan Hendrickson
e086d5d899 libexpr: experimental pipe operators 2024-07-24 13:17:28 -04:00
Eelco Dolstra
3be7c0037e WorkerProto: Support fine-grained protocol feature negotiation
Currently, the worker protocol has a version number that we increment
whenever we change something in the protocol. However, this can cause
a collision between Nix PRs / forks that make protocol changes
(e.g. PR #9857 increments the version, which could collide with
another PR). So instead, the client and daemon now exchange a set of
protocol features (such as `auth-forwarding`). They will use the
intersection of the sets of features, i.e. the features they both
support.

Note that protocol features are completely distinct from
`ExperimentalFeature`s.
2024-07-24 16:23:37 +02:00
John Ericson
046cc8cbb3
Merge pull request #11156 from obsidiansystems/fix-bsd-pthread
Fix some BSD builds missing pthread functions
2024-07-22 13:58:41 -04:00
John Ericson
d3cee8160c Add missing threads deps 2024-07-22 11:46:54 -04:00
John Ericson
eea63d5f99 Run unit tests with wine
Fix #10547

The tests do not all run successfully, but that is a separate issue.
2024-07-22 10:31:26 -04:00
John Ericson
dc6dbbc1a5 Test exes do not need dev outputs 2024-07-22 10:26:05 -04:00
Farid Zakaria
57399bfc0e
Refactor unix domain socket store config (#11109)
Following what is outlined in #10766 refactor the uds-remote-store such
that the member variables (state) don't live in the store itself but in
the config object.

Additionally, the config object includes a new necessary constructor
that takes a scheme & authority.

Tests are commented out because of linking errors with the current config system.
When there is a new config system we can reenable them.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-07-17 23:32:27 -04:00
Las Safin
5b6a21acc5
Avoid casting function pointer in libutil test support
Casting function pointers seems to be almost always UB.
See https://stackoverflow.com/questions/559581/casting-a-function-pointer-to-another-type

Fixed by doing the casting of `void*` to `std::string*` inside the function instead.

Caught by UBSan.
2024-07-16 22:01:34 +00:00
John Ericson
808082ea03 Ensure we can construct remote store configs in isolation
Progress towards #10766

I thought that #10768 achieved, but when I went to use this stuff (in
Hydra), turns out it did not. (Those `using FooConfig;` lines were not
working --- they are so finicky!) This PR gets the job done, and adds
some trivial unit tests to make sure I did what I intended.

I had to add add a header to expose `SSHStoreConfig`, after which the
preexisting `ssh-store-config.*` were very confusingly named files, so I
renamed them to `common-ssh-store-config.hh` to match the type defined
therein.
2024-07-15 17:32:49 -04:00
Robert Hensing
61a4d3d45c getSnippetUpTo: Return optional
This makes it possible to certain discern failures from empty
snippets, which I think is an ok review comment.

Maybe it should do so for swapped column indexes too, but I'm not
sure.

I don't think it matters in the grand scheme. We don't even have
a real use case for `nullopt` now anyway.

Since we don't have a use case, I'm not applying this logic to
higher level functions yet.
2024-07-15 20:10:45 +02:00
Robert Hensing
e5af7cbeb9 libutil: Add Pos::getSnippetUpTo(Pos) 2024-07-15 18:41:11 +02:00
Robert Hensing
d3e49ac881 dropEmptyInitThenConcatStringsSep -> concatStringSep: shortRefs are not empty 2024-07-13 03:06:24 +02:00
Robert Hensing
a681d354e7 Add fresh concatStringsSep without bug
The buggy version was previously renamed to
dropEmptyInitThenConcatStringsSep
2024-07-13 03:06:24 +02:00
Robert Hensing
1a8defd06f Refactor: rename C++ concatStringsSep -> dropEmptyInitThenConcatStringsSep 2024-07-13 03:05:50 +02:00
John Ericson
bc83b9dc1f Remove comparator.hh and switch to <=> in a bunch of places
Known behavior changes:

- `MemorySourceAccessor`'s comparison operators no longer forget to
  compare the `SourceAccessor` base class.

Progress on #10832

What remains for that issue is hopefully much easier!
2024-07-12 14:54:18 -04:00
John Ericson
337a5a23b7
Merge pull request #11089 from NixOS/warnings-includes
Fix warnings and optimize includes
2024-07-12 10:29:26 -04:00
Robert Hensing
8df041cbc6 Solve unused header warnings reported by clangd 2024-07-12 15:37:54 +02:00
John Ericson
3fc77f281e No global settings in libnixfetchers and libnixflake
Progress on #5638

There are still a global fetcher and eval settings, but they are pushed
down into `libnixcmd`, which is a lot less bad a place for this sort of
thing.

Continuing process pioneered in
52bfccf8d8.
2024-07-12 08:50:28 -04:00
Robert Hensing
4fd8f19ecf Fix build to use CanonPath in new FSO sinks 2024-07-11 12:14:48 +02:00
Robert Hensing
86420753ec Merge remote-tracking branch 'upstream/master' into support-hardlinks-in-tarballs 2024-07-11 11:43:02 +02:00
John Ericson
c5284a84f3 Forgot to include config-expr.hh in some places 2024-07-08 10:37:34 -04:00
Robert Hensing
bea54d116e Add resolvePath, filesetToSource indirections for Nixpkgs 2024-07-06 19:49:55 +02:00
Robert Hensing
0729f0a113 packaging: Pass version directly 2024-07-06 17:52:57 +02:00
Robert Hensing
ddff76f667
Merge pull request #10973 from NixOS/meson-libexpr
Meson build for libexpr libflake, external C API, unit tests
2024-07-05 20:27:12 +02:00
John Ericson
e4056b9afd
Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-04 17:48:27 -04:00
John Ericson
30de61f16d
Merge pull request #11018 from siddhantk232/canonpath-fs-sink
Use `CanonPath` in `fs-sink.hh`
2024-07-03 10:36:18 -04:00
siddhantCodes
2cf24a2df0 fix tests and minor changes
- use the iterator in `CanonPath` to count `level`
- use the `CanonPath::basename` method
- use `CanonPath::root` instead of `CanonPath{""}`
- remove `Path` and `PathView`, use `std::filesystem::path` directly
2024-07-03 17:43:55 +05:30
John Ericson
b0bc2a97bf Put unit tests back in old build system for now 2024-07-02 09:26:22 -04:00
John Ericson
451f8a8c19 Put back files for now
We'll revert this sometime later
2024-07-02 09:26:22 -04:00
John Ericson
17a8c2bfce Unit tests and external libraries 2024-07-02 09:23:24 -04:00
John Ericson
31257009e1 Meson build for libexpr and libflake 2024-07-02 09:23:24 -04:00
Jörg Thalheim
101915c9b7 enable -Werror=unused-result
Inspired by
010ff57ebb

From the original PR:

> We do not have any of these warnings appearing at the moment, but
> it seems like a good idea to enable [[nodiscard]] checking anyway.
> Once we start introducing more functions with must-use conditions we will
> need such checking, and the rust stdlib has proven them very useful.
2024-07-02 08:46:06 +02:00
Robert Hensing
f0329568b5 GitFileSystemObjectSink: catch an overflow 2024-07-01 17:29:07 +02:00
Robert Hensing
a409c1a882 Start unit testing GitFileSystemObjectSink 2024-07-01 17:29:07 +02:00
Robert Hensing
992912f3b4 test-support: Add TracingFileSystemObjectSink 2024-07-01 17:29:07 +02:00
siddhantCodes
72bb530141 use CanonPath in fs-sink and its derivatives 2024-06-30 19:03:15 +05:30
John Ericson
0084a486cc Split out a new libnixflake
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-26 19:56:21 -04:00
Robert Hensing
85de5a60c7 Use lib instead of explicit fileset passing 2024-06-26 04:11:20 +02:00
John Ericson
65d7c80365
Merge pull request #10955 from NixOS/meson-nix-util-c
Build nix-util-c with meson + unit tests
2024-06-25 19:06:06 -04:00