Commit graph

4370 commits

Author SHA1 Message Date
siddhantCodes
8f1a26667e add call to checkInterrupt in a bunch of places
This brings back the old behaviour. We check for interrupts in places
that may iterate over wide directories.
2024-06-04 19:35:40 +05:30
siddhantCodes
39e8aad446 Merge branch 'master' of github.com:NixOS/nix 2024-05-13 18:43:12 +05:30
siddhantCodes
62e1ea2f4b use path for from arg in nix::copyFile 2024-05-13 16:10:21 +05:30
siddhantCodes
4d0777ca69 fix: copy fileName before calling std::distance 2024-05-13 15:36:00 +05:30
siddhantCodes
ccf94545db rename copy -> copyFile and remove old copyFile
the old `copyFile` was just a wrapper that was calling the `copy`
function. This wrapper function is removed and the `copy` function is
renamed to `copyFile`.
2024-05-12 19:20:17 +05:30
siddhantCodes
d3b7367c80 inline usage of nix::getFileType and remove it 2024-05-12 18:58:05 +05:30
siddhantCodes
4537663740 inline the usage of nix::renameFile
use `std::filesystem::rename` everywhere and remove `nix::renameFile`
2024-05-12 18:40:16 +05:30
siddhantCodes
1db7d1b840 inline the usage of nix::readDirectory
`nix::readDirectory` is removed. `std::filesystem::directory_iterator`
is used directly in places that used this util.
2024-05-12 17:42:18 +05:30
John Ericson
e0ff8da9d5 Build the local store on Windows
Fixes #10558

Co-Authored-By: Eugene Butler <eugene@eugene4.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-05-10 13:05:23 -04:00
John Ericson
0998a3ac01 Remove LocalStore::OptimiseStats::blocksFreed as it is dead code 2024-05-10 12:55:21 -04:00
siddhantCodes
ddea4c6deb rm isLink
isLink util is removed in favour of std::filesystem::is_symlink
2024-05-08 19:59:37 +05:30
Siddhant Kumar
fcbc36cf78
Use std::filesystem::path in more places (#10657)
Progress on #9205

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>

* Get rid of `PathNG`, just use `std::filesystem::path`
2024-05-07 22:28:50 +00:00
John Ericson
72a0d4b022 Try to fix macOS Nixpkgs lib test failure
Sometimes we read a directory with children we cannot stat. It's a pitty
we even try to stat at all (wasteful) in the `DT_UNKNOWN` case, but at
least this should get rid of the failure.
2024-05-07 16:21:02 -04:00
John Ericson
a3c573950b Replace our DirEntry with std::filesystem's 2024-05-07 16:21:02 -04:00
John Ericson
c371070580 Use std::filesystem functions in more places
This makes for shorter and more portable code.

The only tricky part is catching exceptions: I just searched for near by
`catch (Error &)` or `catch (SysError &)` and adjusted them to `catch
(std::filesystem::filesystem_error &)` according to my human judgement.

Good for windows portability; will help @siddhantk232 with his GSOC
project.
2024-05-07 00:16:54 -04:00
Eelco Dolstra
ef28c7329c Rename makeFSSourceAccessor -> getFSSourceAccessor()
This makes it clearer that it returns a shared accessor object.
2024-05-06 19:16:52 +02:00
Eelco Dolstra
eab2919119 Use SourcePath in more places
Now that SourcePath uses a SourceAccessor instead of an InputAccessor,
we can use it in function signatures instead of passing a
SourceAccessor and CanonPath separately.
2024-05-06 19:05:42 +02:00
Théophane Hufschmitt
9bd1191fcc libstore: check additionalSandboxProfile
Make sure that `extraSandboxProfile` is set before we check whether it's
empty or not (in the `sandbox=true` case).

Also adds a test case for this.

Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
2024-05-06 17:37:08 +02:00
Charlie Moog
71c66de227
document store url trusted=true option behavior 2024-05-05 17:37:04 +00:00
John Ericson
037c8d771d Fix format errors
Fix formatting violations, update blacklist to reflect moved files.

PR #10556 passed CI before the new formating rules were added, and our
CI has the race condition of allowing old results, resulting in master
getting broken.
2024-05-02 21:42:28 -04:00
John Ericson
1948ec390c
Merge pull request #10556 from nix-windows/uds-remote-on-windows
Enable the `unix://` store on Windows
2024-05-02 09:53:00 -04:00
Dylan Green
aa165301d1
Pathlocks Implementation for Windows (#10586)
Based on Volth's original port.

Co-authored-by: volth <volth@volth.com>
2024-04-22 15:08:10 +00:00
Robert Hensing
6fd2f42c2d
Merge pull request #10573 from RCoeurjoly/Rename_SearchPath
Rename SearchPath to LookupPath and searchPath to lookupPath
2024-04-21 17:33:46 +02:00
Robert Hensing
e05b58b060 init: Add flag to avoid loading configuration 2024-04-20 01:45:04 +02:00
John Ericson
b973cd494f Enable the unix:// store on Windows
Windows now has some basic Unix Domain Socket support, see
https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/

Building `nix daemon` on Windows I've left for later, because the daemon
currently forks per connection but this is not an option on Windows. But
we can get the client part working right away.
2024-04-18 16:58:32 -04:00
John Ericson
3a3c205fa7 Use rand not random for creating GC root indirect links
I don't think fewer bits matters for this, and `rand` but not `random`
is available on Windows.
2024-04-18 16:57:11 -04:00
Eelco Dolstra
40499bca99
Merge pull request #10549 from alyssais/exportReferencesGraph
Fix exportReferencesGraph when given store subpath
2024-04-18 17:23:12 +02:00
John Ericson
fb9f4208ed
Don't include linux/ in #include
The linux dirs are conditionally added to the `-I` path.
2024-04-18 12:20:20 +02:00
Maximilian Bosch
ba68045187
libstore/local-derivation-goal: prohibit creating setuid/setgid binaries
With Linux kernel >=6.6 & glibc 2.39 a `fchmodat2(2)` is available that
isn't filtered away by the libseccomp sandbox.

Being able to use this to bypass that restriction has surprising results
for some builds such as lxc[1]:

> With kernel ≥6.6 and glibc 2.39, lxc's install phase uses fchmodat2,
> which slips through 9b88e52846/src/libstore/build/local-derivation-goal.cc (L1650-L1663).
> The fixupPhase then uses fchmodat, which fails.
> With older kernel or glibc, setting the suid bit fails in the
> install phase, which is not treated as fatal, and then the
> fixup phase does not try to set it again.

Please note that there are still ways to bypass this sandbox[2] and this is
mostly a fix for the breaking builds.

This change works by creating a syscall filter for the `fchmodat2`
syscall (number 452 on most systems). The problem is that glibc 2.39
and seccomp 2.5.5 are needed to have the correct syscall number available
via `__NR_fchmodat2` / `__SNR_fchmodat2`, but this flake is still on
nixpkgs 23.11. To have this change everywhere and not dependent on the
glibc this package is built against, I added a header
"fchmodat2-compat.hh" that sets the syscall number based on the
architecture. On most platforms its 452 according to glibc with a few
exceptions:

    $ rg --pcre2 'define __NR_fchmodat2 (?!452)'
    sysdeps/unix/sysv/linux/x86_64/x32/arch-syscall.h
    58:#define __NR_fchmodat2 1073742276

    sysdeps/unix/sysv/linux/mips/mips64/n32/arch-syscall.h
    67:#define __NR_fchmodat2 6452

    sysdeps/unix/sysv/linux/mips/mips64/n64/arch-syscall.h
    62:#define __NR_fchmodat2 5452

    sysdeps/unix/sysv/linux/mips/mips32/arch-syscall.h
    70:#define __NR_fchmodat2 4452

    sysdeps/unix/sysv/linux/alpha/arch-syscall.h
    59:#define __NR_fchmodat2 562

I tested the change by adding the diff below as patch to
`pkgs/tools/package-management/nix/common.nix` & then built a VM from
the following config using my dirty nixpkgs master:

    {
      vm = { pkgs, ... }: {
        virtualisation.writableStore = true;
        virtualisation.memorySize = 8192;
        virtualisation.diskSize = 12 * 1024;
        nix.package = pkgs.nixVersions.nix_2_21;
      };
    }

The original issue can be triggered via

    nix build -L github:nixos/nixpkgs/d6dc19adbda4fd92fe9a332327a8113eaa843894#lxc \
      --extra-experimental-features 'nix-command flakes'

however the problem disappears with this patch applied.

Closes #10424

[1] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2031073804
[2] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2030844251
2024-04-18 12:20:20 +02:00
John Ericson
6fa3656a32 Make a few commands that were Unix-only no longer
Also clean up some more linux-specific (`setPersonality`) code in
alignment with recent best practices.
2024-04-17 16:20:56 -04:00
Alyssa Ross
0774e8ba33
Fix exportReferencesGraph when given store subpath
With Nix 2.3, it was possible to pass a subpath of a store path to
exportReferencesGraph:

	with import <nixpkgs> {};

	let
	  hello = writeShellScriptBin "hello" ''
	    echo ${toString builtins.currentTime}
	  '';
	in

	writeClosure [ "${hello}/bin/hello" ]

This regressed with Nix 2.4, with a very confusing error message, that
presumably indicates it was unintentional:

	error: path '/nix/store/3gl7kgjr4pwf03f0x70dgx9ln3bhl7zc-hello/bin/hello' is not in the Nix store
2024-04-17 21:51:59 +02:00
John Ericson
21d9412ddc Improve local-overlay docs in a few ways
In response to https://discourse.nixos.org/t/super-colliding-nix-stores/28462/24
2024-04-17 15:28:19 -04:00
John Ericson
1063aa502a Relax store path canonicalization 2024-04-17 12:26:10 -04:00
John Ericson
b529d91902 Prefix -DNIX_ paths to be windows-complient for windows
This is a hacky solution, but it will do for now.
2024-04-17 12:26:10 -04:00
John Ericson
8433027e35 Build a minimized Nix with MinGW
At this point many features are stripped out, but this works:

- Can run libnix{util,store,expr} unit tests
- Can run some Nix commands

Co-Authored-By volth <volth@volth.com>
Co-Authored-By Brian McKenna <brian@brianmckenna.org>
2024-04-17 12:26:10 -04:00
John Ericson
2248a3f545 Create no-op Window pathlocks implementation
This keeps the call sites simple, eventually this should be filled in.
2024-04-17 11:48:14 -04:00
Théophane Hufschmitt
d2a07a96ba
Merge pull request #10467 from edolstra/nix-shell-symlink
nix shell: Handle output paths that are symlinks
2024-04-16 12:31:14 +02:00
John Ericson
65cc237b3a
Merge pull request #10482 from tweag/fix-symlink-in-sandbox
Fix the access of symlinks to host files in the sandbox
2024-04-15 09:29:00 -04:00
Roland Coeurjoly
40a6a9fdb8 Rename SearchPath to LookupPath and searchPath to lookupPath 2024-04-13 17:35:15 +02:00
Valentin Gagarin
13c2005e7d
add intermediate variables and clarifying comments (#9274)
* add intermediate variables and clarifying comments

Co-authored-by: Alexander Groleau <alex@proof.construction>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-04-12 17:43:35 +02:00
Théophane Hufschmitt
acbb1523c1 Fix the access of symlinks to host files in the sandbox
https://github.com/NixOS/nix/pull/10456 fixed the addition of symlink
store paths to the sandbox, but also made it so that the hardcoded
sandbox paths (like `/etc/hosts`) were now bind-mounted without
following the possible symlinks. This made these files unreadable if
there were symlinks (because the sandbox would now contain a symlink to
an unreachable file rather than the underlying file).
In particular, this broke FOD derivations on NixOS as `/etc/hosts` is a
symlink there.

Fix that by canonicalizing all these hardcoded sandbox paths before
adding them to the sandbox.
2024-04-12 15:57:53 +02:00
John Ericson
5b9cb8b372
Merge pull request #10412 from roberth/c-string-context
C API: Add `nix_string_realise`
2024-04-11 12:07:31 -04:00
Robert Hensing
ed13cf05a2 build-hook: Allow empty
Like always declining; local builds only, as can be inferred from the
docs. (Not worth spending too many words on this pretty obvious
behavior, I think. Also, plans to remove it? https://github.com/NixOS/nix/issues/1221)
2024-04-11 17:39:16 +02:00
Eelco Dolstra
85b9f4ef4f nix shell: Handle output paths that are symlinks
This requires moving resolveSymlinks() into SourceAccessor. Also, it
requires LocalStoreAccessor::maybeLstat() to work on parents of the
store (to avoid an error like "/nix is not in the store").

Fixes #10375.
2024-04-10 23:49:19 +02:00
Théophane Hufschmitt
ae4737294e doBind: Use our own lstat wrapper
Doesn't change much, but brings a bit more consistency to the code
2024-04-10 15:20:10 +02:00
Théophane Hufschmitt
913db9f738 Fix permission denied when building symlink derivation which points to a symlink out of the store
Bind-mounting symlinks is apparently not possible, which is why the
thing was failing.

Fortunately, symlinks are small, so we can fallback to copy them at no cost.

Fix https://github.com/NixOS/nix/issues/9579

Co-authored-by: Artturin <Artturin@artturin.com>
2024-04-10 15:19:18 +02:00
John Ericson
fef952e258
Merge pull request #8397 from NixLayeredStore/overlayfs-store
Local Overlay Store
2024-04-07 22:57:01 -04:00
Robert Hensing
5f9aaa86eb
Merge pull request #10416 from obsidiansystems/cgroup-linux-only
Make `cgroup.{cc,hh}` linux-only files
2024-04-07 16:52:31 +02:00
John Ericson
c99c80f075 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2024-04-05 16:32:02 -04:00
John Ericson
5a365b0c89
Delete dead openFile in binary-cache-store.cc (#10418)
d64cb33e90 / #5111 previously deleted the dead code where this was used, but missed this.
2024-04-05 18:31:43 +00:00