Commit graph

18012 commits

Author SHA1 Message Date
Robert Hensing
7c5a0b06a4 tests/nixos/remote-builds: Wait for multi-user
This should make the test more robust, considering the strange hang
in https://hydra.nixos.org/build/267517233/nixlog/8

`builder` seems to have reached `multi-user.target` before the
SSH connection was established, but this seems to be coincidental.
This does tell us that enforcing this has a minimal cost in terms
of runtime.

Waiting for `multi-user.target` on the client is honestly paranoid,
but flaky tests are very bad for productivity.
2024-07-27 13:08:30 +02:00
Robert Hensing
f4464873f5 tests/nixos/remote-builds: Print hello world to stderr
Trying to learn more about enigmatic spurious hang at
https://hydra.nixos.org/build/267517233/nixlog/8
- builder1 seems to have started properly
- ssh connection and session are established
- ssh client doesn't exit or client.succeed does not return
  for some reason.

Seeing the stdout on the console might give a tiny bit more info.
2024-07-27 13:01:56 +02:00
Robert Hensing
04c20dc0c0
Merge pull request #11197 from NixOS/aws-sdk-cpp-like-nixpkgs
dependencies: Centralize aws-sdk-cpp and sync with Nixpkgs
2024-07-27 03:30:39 +02:00
Robert Hensing
17b5d40445 package.nix: Empty build inputs if not doBuild 2024-07-27 02:39:55 +02:00
Robert Hensing
22f943bb1f dependencies: Centralize aws-sdk-cpp and sync with Nixpkgs
By syncing with Nixpkgs, we reuse the same derivation, which is
generally a good idea, and has the benefit that it is transitively
a channel blocker.

Changes:

- https://github.com/NixOS/nixpkgs/pull/163313 (SuperSandro2000)

  > nix: disable big-parallel for aws-sdk-cpp

  > aws-sdk-cpp only takes ~1m52s on a 4 core machine under 50% load
  > which does not justify the requirement on big parallel.

  > Tested with `nix-build -A nixVersions.nix_2_6.aws-sdk-cpp`.

  > I can finally build nix without requiring a big-parallel machine.

- https://github.com/NixOS/nixpkgs/pull/227506 (Artturin)

  > nix: use [ ] instead null to empty requiredSystemFeatures

  > fixes 'error: value is null while a list was expected' with 'nixpkgs.hostPlatform.gcc.arch = "x86_64";'
2024-07-27 02:16:05 +02:00
Robert Hensing
95845d92f7
Merge pull request #11192 from DeterminateSystems/store-sharedsync
Store: Use SharedSync
2024-07-27 01:32:13 +02:00
Robert Hensing
861bd102a6
Merge pull request #11167 from NixOS/repl-test-rejiggle
Fix repl test for `buildReadlineNoMarkdown`
2024-07-27 00:55:57 +02:00
Robert Hensing
88e8c9017a
Merge pull request #11187 from Mic92/diff-closure-fix
diff-closures: fix a use after free
2024-07-27 00:52:38 +02:00
Eelco Dolstra
c1f7ba7a98
Merge pull request #11190 from DeterminateSystems/unnecessary-eval-string
nix repl: Remove unnecessary call to evalString
2024-07-26 22:27:51 +02:00
Eelco Dolstra
ea46264bd3 Store: Use SharedSync for state 2024-07-26 16:14:03 +02:00
Eelco Dolstra
ce663d75e3 LRUCache: Mark size() as const 2024-07-26 16:13:00 +02:00
Eelco Dolstra
2141a52ca3 nix repl: Remove unnecessary call to evalString
This crashes with the multithreaded evaluator, which checks against
attempts to finish an already finished value.
2024-07-26 15:40:32 +02:00
Robert Hensing
c274e005b6
Merge pull request #11186 from cole-h/fixup-docs-build
Fix reference to experimental features docs
2024-07-25 21:47:18 +02:00
Jade Lovelace
07aeedd37e diff-closures: remove gratuitous copy
This was done originally because std::smatch does not accept `const char
*` as iterators. However, this was because we should have been using
std::cmatch instead.

(cherry picked from commit 12a5838d11)
2024-07-25 21:41:31 +02:00
Jade Lovelace
492715c0bb diff-closures: fix a use after free
Found by looking for interesting asan reports from the test suite.

What happened here is that name got overwritten, but it was what
actually held the backing memory for the thing it got overwritten by,
which was a by-reference value coming out of std::regex.

Due to absurd reasons I cannot seem to use a string_view iterator here,
so I just copy the string with a longer lifetime instead. idk lol

==3796364==ERROR: AddressSanitizer: heap-use-after-free on address 0x503000014c61 at pc 0x74843523bf1d bp 0x7ffc68351330 sp 0x7ffc68350af0
READ of size 3 at 0x503000014c61 thread T0
    0 0x74843523bf1c in __asan_memcpy (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x159f1c)
    1 0x6403cf6cbff4 in std::char_traits<char>::copy(char*, char const*, unsigned long) /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/char_traits.h:445:33
    <...>
    7 0x6403cf6cbff4 in std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::str() const /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/regex.h:966:6
    8 0x6403cf6cbff4 in std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::operator std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>() const /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/regex.h:955:16
    9 0x6403cf6cbff4 in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:37:26
    10 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
    11 0x6403cf873331 in CmdProfileDiffClosures::run(nix::ref<nix::Store>) /home/jade/lix/lix2/build/src/nix/profile.cc:479:17
    <...>

0x503000014c61 is located 17 bytes inside of 21-byte region [0x503000014c50,0x503000014c65)
freed by thread T0 here:
    0 0x748435250470 in operator delete(void*) (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x16e470)
    <...>
    6 0x6403cf6cbda2 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::~basic_string() /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/basic_string.h:792:9
    7 0x6403cf6cbda2 in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:36:13
    8 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
    <...>

previously allocated by thread T0 here:
    0 0x74843524fa38 in operator new(unsigned long) (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x16da38)
    <...>
    9 0x6403cf6cb68c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::basic_string<std::basic_string_view<char, std::char_traits<char>>, void>(std::basic_string_view<char, std::char_traits<char>> const&, std::allocator<char> const&) /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/basic_string.h:784:4
    10 0x6403cf6cb68c in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:33:21
    11 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
    12 0x6403cf873331 in CmdProfileDiffClosures::run(nix::ref<nix::Store>) /home/jade/lix/lix2/build/src/nix/profile.cc:479:17
    <...>

(cherry-picked from b9b1bbd22f)
2024-07-25 21:41:31 +02:00
Cole Helbling
90459e60dc Fix reference to experimental features docs
Arose because https://github.com/NixOS/nix/pull/9014 merged before
https://github.com/NixOS/nix/pull/11131, but the latter did not rebase /
merge against the latest master.
2024-07-25 21:41:31 +02:00
Cole Helbling
6e178cd899 Fix reference to experimental features docs
Arose because https://github.com/NixOS/nix/pull/9014 merged before
https://github.com/NixOS/nix/pull/11131, but the latter did not rebase /
merge against the latest master.
2024-07-25 11:38:45 -07:00
Robert Hensing
6ec123ad6b
Merge pull request #11131 from rhendric/rhendric/pipe-operators
libexpr: experimental pipe operators
2024-07-25 16:58:43 +02:00
Robert Hensing
76f04b4146
Merge pull request #11111 from NixOS/grep-safety-AGAIN
Grep newline safety
2024-07-25 16:30:33 +02:00
Eelco Dolstra
499a9b5f89
Merge pull request #11182 from Mic92/less-alloc
build-remote: only allocate storeUri once
2024-07-25 16:23:06 +02:00
Robert Hensing
55a654abfd Make panic() and unreachable() robust
Plus one or two tweaks.
2024-07-25 15:50:01 +02:00
Robert Hensing
baa28159d3
Update tests/functional/test-infra.sh
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-07-25 15:38:02 +02:00
Jörg Thalheim
9b5ce9acc2 build-remote: only allocate storeUri once
also it's probably not much overhead compared to the networking stuff
it, but it's less code at least.
2024-07-25 14:04:03 +02:00
Eelco Dolstra
99c20d6624
Merge pull request #11179 from obsidiansystems/misc-fix
Misc fixes
2024-07-25 14:01:54 +02:00
John Ericson
ca1e1e983f
Merge pull request #11180 from Mic92/override-warnings
add werror=suggest-override
2024-07-25 03:00:50 -04:00
John Ericson
36252fed6b
Merge pull request #11176 from obsidiansystems/flake-test-group
Put flake functional tests in their own group
2024-07-25 02:41:11 -04:00
Jörg Thalheim
63e50a4b56 add werror=suggest-override
Improves code readability by making overrides explicit.
Inspired by lix code-base
2024-07-25 07:41:12 +02:00
Jörg Thalheim
2c07ea8abb build-utils-meson: remove oudated meson build comment 2024-07-25 07:40:36 +02:00
Jörg Thalheim
6c38bc0952 {src/perl,build-utils-meson/diagnostics}: sort cflags
This makes them easier to copy between places.
2024-07-25 07:38:00 +02:00
Robert Hensing
6662c9012d
Merge pull request #11177 from obsidiansystems/skip-exit-code
Change skipped test error code from 99 to 77
2024-07-25 06:59:18 +02:00
John Ericson
90f7f2139e
Merge pull request #11115 from NixOS/doc-derivation
Document builtins.derivation
2024-07-25 00:21:53 -04:00
John Ericson
1ae5738317 Fix some warnings
I think they came from the last Nixpkgs bump.
2024-07-25 00:02:43 -04:00
John Ericson
e92dd06a7b build-remote: Cope with long store URLs by falling back on hashing
I hit this in the Meson port of the functional tests, because the use of
standalone build directories.
2024-07-25 00:00:52 -04:00
John Ericson
3b49f7a143
Deduplicate our many package.nix a bit (#11175)
- They should all be built in parallel

- They should all use strict deps by default
2024-07-25 03:12:39 +00:00
Valentin Gagarin
a2fed6db9e
manual: Contributing -> Development, Hacking -> Building (#9014)
* manual: Contributing -> Development, Hacking -> Building

what's currently called "hacking" are really instructions for setting up
a development environment and compiling from source. we have
a contribution guide in the repo (which rightly focuses on GitHub
workflows), and the material in the manual is more about working
on the code itself.

since we'd otherwise have three headings that amount to "Building Nix",
this change also moves the "classic Nix" instructions to the top.

we may want to reorganise this in the future, and bring
contributor-oriented information closer to the code, but for now let's
stick to more accurate names to ease navigation.
2024-07-25 02:53:06 +00:00
John Ericson
dcbe2453f5 Change skipped test error code from 99 to 77
Meson uses a venerable GNU convention described in
https://www.gnu.org/software/automake/manual/html_node/Scripts_002dbased-Testsuites.html
in which:

> When no test protocol is in use, an exit status of 0 from a test
> script will denote a success, an exit status of 77 a skipped test, an
> exit status of 99 a hard error, and any other exit status will denote
> a failure.

77 is thus what we want, not 99.
2024-07-24 22:36:43 -04:00
John Ericson
f4915af71c Put flake functional tests in their own group
This is a nice thing to have, and it made it easier to work on the
Meson-ifcation of these functional tests too.
2024-07-24 22:31:37 -04:00
Valentin Gagarin
dba1142c01
docs: add identifiers (#11174)
* docs: add identifiers

* clarify attribute set notation and add examples

* add definition of names

Co-authored-by: Ryan Hendrickson <ryan.hendrickson@alum.mit.edu>
2024-07-25 03:45:34 +02:00
Valentin Gagarin
e062021314
fix NIX_PATH for real (#11079)
* fix NIX_PATH overriding

- test restricted evaluation
- test precedence for setting the search path

Co-authored-by: Robert Hensing <robert@roberthensing.nl>
Co-authored-by: John Ericson <git@JohnEricson.me>
2024-07-24 21:17:15 +00:00
Ryan Hendrickson
e086d5d899 libexpr: experimental pipe operators 2024-07-24 13:17:28 -04:00
Robert Hensing
3172e88af5 Make abort() call sites log first 2024-07-24 16:52:04 +02:00
Eelco Dolstra
874c1bdbbf
Merge pull request #11169 from NixOS/flake-update
Flake update
2024-07-24 16:52:03 +02:00
Eelco Dolstra
b13ba7490c
Merge pull request #11162 from NixOS/rename-stack-limit
Rename osStackLow -> osStackLimit
2024-07-24 16:18:26 +02:00
Robert Hensing
171ef75218 Rename pre-commit-hooks -> git-hooks-nix
Following the upstream rename
2024-07-24 15:55:57 +02:00
Robert Hensing
c316f1557d flake: Switch to nixos-24.05 channel
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/d0907b75146a0ccc1ec0d6c3db287ec287588ef6?narHash=sha256-PhmkdTJs2SfqKzSyDB74rDKp1MH4mGk0pG/%2BWqrnGEw%3D' (2024-07-24)
  → 'github:NixOS/nixpkgs/63d37ccd2d178d54e7fb691d7ec76000740ea24a?narHash=sha256-7cCC8%2BTdq1%2B3OPyc3%2BgVo9dzUNkNIQfwSDJ2HSi2u3o%3D' (2024-07-21)
2024-07-24 15:31:30 +02:00
Robert Hensing
f9a23c8d21 flake.nix: Remove outdated comment 2024-07-24 15:30:28 +02:00
Robert Hensing
650f189402 flake.nix: Pin libgit2 to a release 2024-07-24 15:24:10 +02:00
Robert Hensing
a16df88252 flake.lock: Update
Flake lock file updates:

• Updated input 'flake-compat':
    'github:edolstra/flake-compat/35bb57c0c8d8b62bbfd284272c928ceb64ddbde9?narHash=sha256-4gtG9iQuiKITOjNQQeQIpoIB6b16fm%2B504Ch3sNKLd8%3D' (2023-01-17)
  → 'github:edolstra/flake-compat/0f9255e01c2351cc7d116c072cb317785dd33b33?narHash=sha256-kvjfFW7WAETZlt09AgDn1MrtKzP7t90Vf7vypd3OL1U%3D' (2023-10-04)
• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/9126214d0a59633752a136528f5f3b9aa8565b7d?narHash=sha256-sB4SWl2lX95bExY2gMFG5HIzvva5AVMJd4Igm%2BGpZNw%3D' (2024-04-01)
  → 'github:hercules-ci/flake-parts/9227223f6d922fee3c7b190b2cc238a99527bbb7?narHash=sha256-pQMhCCHyQGRzdfAkdJ4cIWiw%2BJNuWsTX7f0ZYSyz0VY%3D' (2024-07-03)
• Updated input 'libgit2':
    'github:libgit2/libgit2/45fd9ed7ae1a9b74b957ef4f337bc3c8b3df01b5?narHash=sha256-oX4Z3S9WtJlwvj0uH9HlYcWv%2Bx1hqp8mhXl7HsLu2f0%3D' (2023-10-18)
  → 'github:libgit2/libgit2/503b66cf00ad7dca940148529f60b1a409ccc462?narHash=sha256-tDUQi%2Bs8sxJ30SmUH7Ln9WmDz5jGatlgKumjwi7KnCo%3D' (2024-07-17)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/be3ca229c85e978880babdeda9748b14e6aa008f?narHash=sha256-L61BXz7n/yNzOeZ3FqlnUmxj4145JOVeq9fvQTQzbNM%3D' (2024-07-21)
  → 'github:NixOS/nixpkgs/d0907b75146a0ccc1ec0d6c3db287ec287588ef6?narHash=sha256-PhmkdTJs2SfqKzSyDB74rDKp1MH4mGk0pG/%2BWqrnGEw%3D' (2024-07-24)
• Updated input 'pre-commit-hooks':
    'github:cachix/pre-commit-hooks.nix/40e6053ecb65fcbf12863338a6dcefb3f55f1bf8?narHash=sha256-nMirxrGteNAl9sWiOhoN5tIHyjBbVi5e2tgZUgZlK3Y%3D' (2024-04-12)
  → 'github:cachix/pre-commit-hooks.nix/f451c19376071a90d8c58ab1a953c6e9840527fd?narHash=sha256-6FPUl7HVtvRHCCBQne7Ylp4p%2BdpP3P/OYuzjztZ4s70%3D' (2024-07-15)
• Removed input 'pre-commit-hooks/flake-utils'
2024-07-24 15:18:50 +02:00
Robert Hensing
68693276f9 Update fixupBoehmStackPointer doc
lo might have made sense in the bdwgc code, maybe?, but not here.
2024-07-24 13:02:49 +02:00
Robert Hensing
907b0a371a Rename osStackLow -> osStackLimit
This is in accordance with ARM's naming convention.
"Low" is confusing, because it could refer to either the cold end
of the stack as an abstract data type, or a low address.
These are different places, because the stack grows down through
the address space.
2024-07-24 13:02:49 +02:00