Commit graph

18512 commits

Author SHA1 Message Date
Eelco Dolstra
71865dee2d Fix fetchTarball docs 2024-07-29 15:04:55 +02:00
Robert Hensing
6e3bba5e26
Merge pull request #11171 from DeterminateSystems/speed-up-tarball-downloads
Increase download buffer size and improve tarball import logging
2024-07-29 15:02:35 +02:00
Eelco Dolstra
7c18b4d060 Don't dereference top-level regular files
Since this yielded an empty directory as far back as Nix 2.3, we don't
really need special handling for executables vs non-executables.
2024-07-29 14:34:02 +02:00
Eelco Dolstra
e0012b97ab Split tarball-specific logic from GitFileSystemObjectSink 2024-07-29 14:26:25 +02:00
Eelco Dolstra
3c0963487e
Merge pull request #11196 from NixOS/rename-lock-read
Rename SyncBase::read() -> readLock()
2024-07-29 14:10:46 +02:00
Eelco Dolstra
836d24d6e8
Merge pull request #11209 from pinotree/libutil-current-process-includes
libutil: fix/improve includes in current-process.cc
2024-07-29 14:01:02 +02:00
Eelco Dolstra
9e2bed7827
Merge pull request #11206 from tie/getxattr-enotsup
libstore: return ENOTSUP for getxattr functions
2024-07-29 14:00:36 +02:00
Eelco Dolstra
3faa77bb82
Merge pull request #11200 from NixOS/buildNoTests-no-unit-tests
buildNoTests: Disable unit tests
2024-07-29 13:57:23 +02:00
Eelco Dolstra
673f2dcadb
Merge pull request #11203 from pinotree/libutil-ctor-remove-template-id
libutil: remove template id from constructors
2024-07-29 13:56:55 +02:00
Pino Toscano
c34077578e libutil: fix/improve includes in current-process.cc
- move <sys/resource.h> from a __linux__ block to a !_WIN32 block: this
  matches what the actual code does, using getrlimit() & setrlimit() in
  !_WIN32 blocks
- drop <sys/mount.h>, which is not portable, and it is not used
2024-07-28 17:33:24 +02:00
Pino Toscano
96e06b2b06 libutil: remove template id from constructors
This is not allowed in C++20, and GCC 14 warns about it:

../src/libutil/ref.hh:26:20: warning: template-id not allowed for constructor in C++20 [-Wtemplate-id-cdtor]
   26 |     explicit ref<T>(const std::shared_ptr<T> & p)
      |                    ^
../src/libutil/ref.hh:26:20: note: remove the '< >'
../src/libutil/ref.hh:33:21: warning: template-id not allowed for constructor in C++20 [-Wtemplate-id-cdtor]
   33 |     explicit ref<T>(T * p)
      |                     ^
../src/libutil/ref.hh:33:21: note: remove the '< >'
2024-07-28 16:35:09 +02:00
Robert Hensing
0e151bcbf0
Merge pull request #11204 from pinotree/libcmd-editline-helpers
libcmd: do not compile editline helpers when building w/ readline
2024-07-28 16:10:32 +02:00
Valentin Gagarin
933f2c086a
docs: fix link to building instructions (#11207) 2024-07-28 13:34:48 +00:00
Ivan Trubach
1b47748e5a libstore: return ENOTSUP for getxattr functions
This change updates the seccomp profile to return ENOTSUP for getxattr
functions family. This reflects the behavior of filesystems that don’t
support extended attributes (or have an option to disable them), e.g.
ext2.

The current behavior is confusing for some programs because we can read
extended attributes, but only get to know that they are not supported
when setting them. In addition to that, ACLs on Linux are implemented
via extended attributes internally and if we don’t return ENOTSUP, acl
library converts file mode to ACL.
https://git.savannah.nongnu.org/cgit/acl.git/tree/libacl/acl_get_file.c?id=d9bb1759d4dad2f28a6dcc8c1742ff75d16dd10d#n69
2024-07-28 13:28:52 +03:00
Pino Toscano
e0198c513a libcmd: do not compile editline helpers when building w/ readline
The internal "completionCallback" and "listPossibleCallback" helpers
are used only when building with editline; hence, do not build then
when using readline, matching their usage in
"ReadlineLikeInteracter::init()".
2024-07-28 11:40:16 +02:00
Robert Hensing
aa2b1d10e2
Update doc/manual/rl-next/10734-nix3-build-show-all-fod-errors-with-keep-going.md 2024-07-27 14:58:57 +02:00
Robert Hensing
9f1e73ed37
Merge pull request #11199 from NixOS/troubleshoot-remote-build-tests
Troubleshoot remote build tests
2024-07-27 14:25:10 +02:00
Robert Hensing
cc5b8cdc85 buildNoTests: Disable unit tests
This seems to have been the intent all along.

The odd combination of unit tests, but no functional tests caused a
build error where some data for the unit test was source-filtered out.
Apparently. It's unclear to me why that happened, so I'm proposing this
alternate "fix" to get the buildNoTests to pass.

It would be nice to test more configurations, but this mode of building
is on the way out anyway, so let's just make it pass and see what
configurations make sense to test as part of the meson migration.
2024-07-27 13:42:03 +02:00
Robert Hensing
7c5a0b06a4 tests/nixos/remote-builds: Wait for multi-user
This should make the test more robust, considering the strange hang
in https://hydra.nixos.org/build/267517233/nixlog/8

`builder` seems to have reached `multi-user.target` before the
SSH connection was established, but this seems to be coincidental.
This does tell us that enforcing this has a minimal cost in terms
of runtime.

Waiting for `multi-user.target` on the client is honestly paranoid,
but flaky tests are very bad for productivity.
2024-07-27 13:08:30 +02:00
Robert Hensing
f4464873f5 tests/nixos/remote-builds: Print hello world to stderr
Trying to learn more about enigmatic spurious hang at
https://hydra.nixos.org/build/267517233/nixlog/8
- builder1 seems to have started properly
- ssh connection and session are established
- ssh client doesn't exit or client.succeed does not return
  for some reason.

Seeing the stdout on the console might give a tiny bit more info.
2024-07-27 13:01:56 +02:00
Robert Hensing
04c20dc0c0
Merge pull request #11197 from NixOS/aws-sdk-cpp-like-nixpkgs
dependencies: Centralize aws-sdk-cpp and sync with Nixpkgs
2024-07-27 03:30:39 +02:00
Robert Hensing
17b5d40445 package.nix: Empty build inputs if not doBuild 2024-07-27 02:39:55 +02:00
Robert Hensing
22f943bb1f dependencies: Centralize aws-sdk-cpp and sync with Nixpkgs
By syncing with Nixpkgs, we reuse the same derivation, which is
generally a good idea, and has the benefit that it is transitively
a channel blocker.

Changes:

- https://github.com/NixOS/nixpkgs/pull/163313 (SuperSandro2000)

  > nix: disable big-parallel for aws-sdk-cpp

  > aws-sdk-cpp only takes ~1m52s on a 4 core machine under 50% load
  > which does not justify the requirement on big parallel.

  > Tested with `nix-build -A nixVersions.nix_2_6.aws-sdk-cpp`.

  > I can finally build nix without requiring a big-parallel machine.

- https://github.com/NixOS/nixpkgs/pull/227506 (Artturin)

  > nix: use [ ] instead null to empty requiredSystemFeatures

  > fixes 'error: value is null while a list was expected' with 'nixpkgs.hostPlatform.gcc.arch = "x86_64";'
2024-07-27 02:16:05 +02:00
Robert Hensing
6af40f488a Rename SyncBase::read() -> readLock()
Make it explicit so it's clear what it's about when I and other
contributors read its call sites.
2024-07-27 01:39:13 +02:00
Robert Hensing
95845d92f7
Merge pull request #11192 from DeterminateSystems/store-sharedsync
Store: Use SharedSync
2024-07-27 01:32:13 +02:00
Robert Hensing
861bd102a6
Merge pull request #11167 from NixOS/repl-test-rejiggle
Fix repl test for `buildReadlineNoMarkdown`
2024-07-27 00:55:57 +02:00
Robert Hensing
88e8c9017a
Merge pull request #11187 from Mic92/diff-closure-fix
diff-closures: fix a use after free
2024-07-27 00:52:38 +02:00
Eelco Dolstra
c1f7ba7a98
Merge pull request #11190 from DeterminateSystems/unnecessary-eval-string
nix repl: Remove unnecessary call to evalString
2024-07-26 22:27:51 +02:00
Eelco Dolstra
5e83c0427f Fix test 2024-07-26 20:46:07 +02:00
Eelco Dolstra
b88950ec77 Update fetchTree docs 2024-07-26 20:34:04 +02:00
Eelco Dolstra
06b686b62d Handle tarballs that don't consist of a single top-level directory
Fixes #4785 (top-level directories are no longer merged into one).

Fixes #10983 (top-level non-directories are no longer discarded).
2024-07-26 20:24:58 +02:00
Eelco Dolstra
d9ba2a1634 Fix error message 2024-07-26 19:06:49 +02:00
Eelco Dolstra
ea46264bd3 Store: Use SharedSync for state 2024-07-26 16:14:03 +02:00
Eelco Dolstra
ce663d75e3 LRUCache: Mark size() as const 2024-07-26 16:13:00 +02:00
Eelco Dolstra
6d843ce9fe Provide std::hash<Symbol> 2024-07-26 16:06:09 +02:00
Eelco Dolstra
2141a52ca3 nix repl: Remove unnecessary call to evalString
This crashes with the multithreaded evaluator, which checks against
attempts to finish an already finished value.
2024-07-26 15:40:32 +02:00
Ryan Hendrickson
18db46a6cb parser.y: GLR -> LALR 2024-07-25 15:48:58 -04:00
Ryan Hendrickson
6e3b9e6a4d parser.y: eliminate conflicts 2024-07-25 15:48:58 -04:00
Ryan Hendrickson
b0a8430e85 parser.y: move attr doc setting into addAttr 2024-07-25 15:48:58 -04:00
Ryan Hendrickson
429a197d24 parser.y: use names where I'll be refactoring 2024-07-25 15:48:58 -04:00
Robert Hensing
c274e005b6
Merge pull request #11186 from cole-h/fixup-docs-build
Fix reference to experimental features docs
2024-07-25 21:47:18 +02:00
Jade Lovelace
07aeedd37e diff-closures: remove gratuitous copy
This was done originally because std::smatch does not accept `const char
*` as iterators. However, this was because we should have been using
std::cmatch instead.

(cherry picked from commit 12a5838d11)
2024-07-25 21:41:31 +02:00
Jade Lovelace
492715c0bb diff-closures: fix a use after free
Found by looking for interesting asan reports from the test suite.

What happened here is that name got overwritten, but it was what
actually held the backing memory for the thing it got overwritten by,
which was a by-reference value coming out of std::regex.

Due to absurd reasons I cannot seem to use a string_view iterator here,
so I just copy the string with a longer lifetime instead. idk lol

==3796364==ERROR: AddressSanitizer: heap-use-after-free on address 0x503000014c61 at pc 0x74843523bf1d bp 0x7ffc68351330 sp 0x7ffc68350af0
READ of size 3 at 0x503000014c61 thread T0
    0 0x74843523bf1c in __asan_memcpy (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x159f1c)
    1 0x6403cf6cbff4 in std::char_traits<char>::copy(char*, char const*, unsigned long) /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/char_traits.h:445:33
    <...>
    7 0x6403cf6cbff4 in std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::str() const /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/regex.h:966:6
    8 0x6403cf6cbff4 in std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::operator std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>() const /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/regex.h:955:16
    9 0x6403cf6cbff4 in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:37:26
    10 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
    11 0x6403cf873331 in CmdProfileDiffClosures::run(nix::ref<nix::Store>) /home/jade/lix/lix2/build/src/nix/profile.cc:479:17
    <...>

0x503000014c61 is located 17 bytes inside of 21-byte region [0x503000014c50,0x503000014c65)
freed by thread T0 here:
    0 0x748435250470 in operator delete(void*) (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x16e470)
    <...>
    6 0x6403cf6cbda2 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::~basic_string() /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/basic_string.h:792:9
    7 0x6403cf6cbda2 in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:36:13
    8 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
    <...>

previously allocated by thread T0 here:
    0 0x74843524fa38 in operator new(unsigned long) (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x16da38)
    <...>
    9 0x6403cf6cb68c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::basic_string<std::basic_string_view<char, std::char_traits<char>>, void>(std::basic_string_view<char, std::char_traits<char>> const&, std::allocator<char> const&) /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/basic_string.h:784:4
    10 0x6403cf6cb68c in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:33:21
    11 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
    12 0x6403cf873331 in CmdProfileDiffClosures::run(nix::ref<nix::Store>) /home/jade/lix/lix2/build/src/nix/profile.cc:479:17
    <...>

(cherry-picked from b9b1bbd22f)
2024-07-25 21:41:31 +02:00
Cole Helbling
90459e60dc Fix reference to experimental features docs
Arose because https://github.com/NixOS/nix/pull/9014 merged before
https://github.com/NixOS/nix/pull/11131, but the latter did not rebase /
merge against the latest master.
2024-07-25 21:41:31 +02:00
Cole Helbling
6e178cd899 Fix reference to experimental features docs
Arose because https://github.com/NixOS/nix/pull/9014 merged before
https://github.com/NixOS/nix/pull/11131, but the latter did not rebase /
merge against the latest master.
2024-07-25 11:38:45 -07:00
Robert Hensing
6ec123ad6b
Merge pull request #11131 from rhendric/rhendric/pipe-operators
libexpr: experimental pipe operators
2024-07-25 16:58:43 +02:00
Robert Hensing
76f04b4146
Merge pull request #11111 from NixOS/grep-safety-AGAIN
Grep newline safety
2024-07-25 16:30:33 +02:00
Eelco Dolstra
499a9b5f89
Merge pull request #11182 from Mic92/less-alloc
build-remote: only allocate storeUri once
2024-07-25 16:23:06 +02:00
Robert Hensing
55a654abfd Make panic() and unreachable() robust
Plus one or two tweaks.
2024-07-25 15:50:01 +02:00
Robert Hensing
f0fe1d880d
Update doc/manual/rl-next/10734-nix3-build-show-all-fod-errors-with-keep-going.md
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2024-07-25 15:39:15 +02:00