Commit graph

11135 commits

Author SHA1 Message Date
Eelco Dolstra
5ca2f58798 Improve use-case-hack description slightly 2024-09-09 14:29:05 +02:00
Eelco Dolstra
4c7a6ffee7
Merge pull request #11380 from roberth/eco-friendly-progress-bar
progress-bar: Only write when truly updated
2024-09-09 13:58:42 +02:00
Eelco Dolstra
5e337ee60d
Merge pull request #11412 from parkerhoyes/daemon-cgroup
Move daemon process into sub-cgroup
2024-09-09 13:49:31 +02:00
Robert Hensing
c955563b64 fix: Avoid deadlock in ProgressBar::redraw() 2024-09-08 11:44:24 +02:00
Robert Hensing
e10ea78f93 refact: Inline ProgressBar::draw(state, newOutput), inline local output 2024-09-08 01:23:01 +02:00
Robert Hensing
047d9643b5 refact: Extract ProgressBar::redraw(newOutput) 2024-09-08 01:23:01 +02:00
Robert Hensing
9df5236c46 progress-bar: Only write when truly updated 2024-09-08 01:04:14 +02:00
Robert Hensing
791f943761
Merge pull request #11437 from fricklerhandwerk/default-nix-path-docs
docs: explain provenance of default `nix-path` values
2024-09-07 22:38:50 +01:00
Eelco Dolstra
9fcb588dd8 RestoreSink::createDirectory(): Use append()
On macOS, `mkdir("x/')` behaves differently than `mkdir("x")` if `x` is
a dangling symlink (the formed succeed while the latter fails). So make
sure we always strip the trailing slash.
2024-09-05 22:21:53 +02:00
Eelco Dolstra
3557587381 Detect NAR directory entries that collide with another path after case-hacking
The test was made by @puckipedia.
2024-09-05 19:26:10 +02:00
Eelco Dolstra
da1ad28912 Test that nix-store --restore fails if the output already exists
This restores the behaviour from before the std::filesystem
refactorings.
2024-09-05 16:48:43 +02:00
Jörg Thalheim
5a5a010120 Revert "fix: Error on malformed URI query parameter"
This reverts commit c9f45677b5.

This now triggers on simple cases like `nix build .#nix`.
Reverting for now.
2024-09-05 15:18:16 +02:00
Jörg Thalheim
a81083d080 Revert "Update src/libutil/url.cc"
This reverts commit 9b1cefe27e.
2024-09-05 15:18:16 +02:00
Jörg Thalheim
70c52d72f4 builtins.unpackChannel: wrap filesystem errors and sanitize channelName
Otherwise these errors are not caught correctly
2024-09-05 14:02:00 +02:00
Jörg Thalheim
05a1ffe236 repl: wrap std::filesystem error into SysError
/tmp/ecstatic-euler-mAFGV7
% /home/joerg/git/nix/build/subprojects/nix/nix repl
Nix 2.25.0
Type :? for help.

after doing rm /tmp/ecstatic-euler-mAFGV7 this will result in:

nix-repl> :lf .
error: cannot determine current working directory: No such file or directory

Before it would make the repl crash

/tmp/clever-hermann-MCm7A9
% /home/joerg/git/nix/build/subprojects/nix/nix repl
Nix 2.25.0
Type :? for help.
nix-repl> :lf .
error: filesystem error: cannot get current path: No such file or directory
2024-09-05 14:00:39 +02:00
Jörg Thalheim
22ba4dc78d builtins.readDir: fix nix error trace on filesystem errors
Before:

nix-env % ./src/nix/nix eval --impure --expr 'let f = builtins.readDir "/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo"; in f' --show-trace
error: filesystem error: directory iterator cannot open directory: No such file or directory [/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo]

After:

error:
       … while calling the 'readDir' builtin
         at «string»:1:9:
            1| let f = builtins.readDir "/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo"; in f
             |         ^

       error: reading directory '/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo': No such file or directory
2024-09-05 14:00:39 +02:00
Valentin Gagarin
17655ecfef docs: explain provenance of default nix-path values
this should make it more obvious how things are related to each other, and also
hopefully expose the historical context without having to say on every
corner that these details are accounting for legacy decisions.
2024-09-05 04:35:06 +02:00
Eelco Dolstra
495d32e1b8 NAR parser: Fix check for duplicate / incorrectly sorted entries
"prevName" was always empty because it was declared in the wrong scope.
2024-09-04 21:43:59 +02:00
Parker Hoyes
bd6ae2f3b9 Use getCurrentCgroup() in getMaxCPU() 2024-09-04 19:10:31 +00:00
Parker Hoyes
03484641a1 Simplify getRootCgroup()
Static local initializers are atomic in C++.
2024-09-04 18:11:16 +00:00
Jeremy Kolb
8152c5c828
Remote nullptr
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-09-04 12:55:32 -04:00
Jeremy Kolb
46f6505865 Pull fut.get() out of the lock
This is https://gerrit.lix.systems/c/lix/+/1462 by @jade_

see: https://git.lix.systems/lix-project/lix/issues/366
see: https://gerrit.lix.systems/c/lix/+/1462
2024-09-04 10:14:51 -04:00
Eelco Dolstra
9d24080090 Git fetcher: Ignore .gitmodules entries that are not submodules
Fixes #10739.
2024-09-04 14:46:33 +02:00
Parker Hoyes
46b3188045 Move daemon process into sub-cgroup
The daemon process is now moved into a new sub-cgroup called nix-daemon when the
daemon starts. This is necessary to abide by the no-processes-in-inner-nodes
rule, because the service cgroup becomes an inner node when the child cgroups
for the build are created (see LocalDerivationGoal::startBuilder()).

See #9675
2024-09-03 17:18:01 +00:00
Eelco Dolstra
784a8436a0
Merge pull request #11402 from DeterminateSystems/fix-max-substitution-jobs
Respect max-substitution-jobs again
2024-09-03 13:35:34 +02:00
Sandro
13100eaa4f
Fix link anchor 2024-09-02 23:25:44 +02:00
Eelco Dolstra
b7acd1c414 "unsigned" -> size_t
Slight cleanup.
2024-09-02 17:28:55 +02:00
Eelco Dolstra
a33cb8af56 Respect max-substitution-jobs again
This broke in #11005. Any number of PathSubstitutionGoals would
be woken up by a single build slot becoming available. If there
are a lot of substitution goals active, this could lead to us
running out of file descriptors (especially on macOS where the
default limit is 256).
2024-09-02 17:28:11 +02:00
Alyssa Ross
9cc550d652
Don't refer to public keys as secret keys in error
This constructor is used for public keys as well.
2024-08-31 16:00:11 +02:00
Robert Hensing
b89eca9aec
Merge pull request #11349 from bryanhonof/bryanhonof.check-query-for-equals
Warn on malformed URI query parameter
2024-08-28 21:56:00 +02:00
Bryan Honof
9b1cefe27e
Update src/libutil/url.cc
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-08-28 18:48:18 +02:00
Robert Hensing
57c48304bb fixup: Release odb 2024-08-28 17:49:22 +02:00
Robert Hensing
fb8d3ed150 fixup: sync -> flush
The latter is not used for memory synchronization things.
2024-08-28 17:49:22 +02:00
Robert Hensing
97ff2ed455 Sync tarball cache within tarball cache Activity 2024-08-28 17:49:22 +02:00
Robert Hensing
d0f8a92363 Make tarball cache more interruptible 2024-08-28 17:49:22 +02:00
Robert Hensing
5dd6c4f062 libgit2, GitRepo: Write thin packfiles
libgit2 didn't write thin ones, hence the patch.

This should improve performance on systems with weak I/O in ~/.cache,
especially in terms of operations per second, or where system calls
are slower. (macOS, VMs?)
2024-08-28 17:49:21 +02:00
Jan Hrcek
cb4b9be458
Fix few duplicate word occurrences (#11381) 2024-08-28 07:01:56 +00:00
John Ericson
c7ec33605e Meson misc things
Meson-ify a few things, scripts, completions, etc. Should make our Meson
build complete except for docs.

Co-Authored-By: Qyriad <qyriad@qyriad.me>
Co-Authored-By: eldritch horrors <pennae@lix.systems>
2024-08-27 10:19:55 -04:00
John Ericson
8bce63f30a More std::filesystem for nix-collect-garbage
Co-Authored-By: siddhantCodes <siddhantk232@gmail.com>
2024-08-26 17:29:23 -04:00
John Ericson
a97a08411c More support for std::filepath in libnixutil
We're not replacing `Path` in exposed definitions in many cases, but
just adding alternatives. This will allow us to "top down" change `Path`
to `std::fileysystem::path`, and then we can remove the `Path`-using
utilities which will become unused.

Also add some test files which we forgot to include in the libutil unit
tests `meson.build`.

Co-Authored-By: siddhantCodes <siddhantk232@gmail.com>
2024-08-26 17:23:56 -04:00
John Ericson
dbabfc92d4 Make sure we have an execvpe on Windows too
Necessary to fix a build (that was already broken in other ways) after
PR #11021.
2024-08-26 15:45:55 -04:00
Robert Hensing
88998fae74
Merge pull request #11304 from hercules-ci/repl-doc-functor
`:doc`: support `__functor`
2024-08-26 17:20:37 +02:00
tomberek
96a2ddab27
Merge pull request #10910 from hercules-ci/c-api-nix_clear_err
C API: `nix_clear_err`
2024-08-26 10:50:26 -04:00
tomberek
440de80d34
Merge pull request #11021 from hercules-ci/issue-11010
Fix SSH invocation when local SHELL misbehaves
2024-08-26 10:40:51 -04:00
Robert Hensing
77ddcbe12e
getDoc: Explain why we partially apply __functor 2024-08-26 16:15:13 +02:00
Robert Hensing
8bd835b7bc
Merge pull request #11278 from Mic92/close-fds
libutil: rename and optimize closeMostFDs
2024-08-26 15:51:34 +02:00
Jörg Thalheim
096bec8eb2 Revert "Merge pull request #11300 from noamraph/homeless-shelter-to-proc"
This reverts commit 43e82c9446, reversing
changes made to d79b9bdec0.

Since /proc/homeless-shelter returns a different errno than /homeless-shelter (ENOENT vs EACCES), we need to revert this change.
Software depends on this error code i.e. cargo and therefore breaks.
2024-08-25 09:08:17 +02:00
Bryan Honof
c9f45677b5
fix: Error on malformed URI query parameter
Signed-off-by: Bryan Honof <bryanhonof@gmail.com>
2024-08-23 22:04:37 +02:00
Emery Hemingway
5d28a00322 Add nix_store_copy_closure to libstore-c 2024-08-23 12:49:39 +03:00
Robert Hensing
0bcc83d0e7
Merge pull request #11348 from DeterminateSystems/remove-makeMutable
Remove dead makeMutable() function
2024-08-23 11:08:28 +02:00
Eelco Dolstra
1facc3e35e
Merge pull request #7126 from squalus/fsync-store-paths
Add fsync-store-paths option
2024-08-22 17:45:11 +02:00
Eelco Dolstra
21a164aa03 Fix hang
Signed-off-by: Eelco Dolstra <edolstra@gmail.com>
2024-08-22 15:26:34 +02:00
Eelco Dolstra
915db74dbf
Merge pull request #11343 from DeterminateSystems/no-framedsink-threads
withFramedSink(): Don't use a thread to monitor the other side
2024-08-22 14:23:19 +02:00
Eelco Dolstra
fac756fed4 Add FIXME 2024-08-21 21:08:26 +02:00
Eelco Dolstra
270c8469d7 select() -> poll() for Windows compat 2024-08-21 20:54:02 +02:00
Eelco Dolstra
ce7cf4a2d3
Update src/libutil/serialise.hh
Co-authored-by: John Ericson <git@JohnEricson.me>
2024-08-21 19:50:24 +02:00
Eelco Dolstra
e8752ca57a Add FIXME 2024-08-21 17:05:36 +02:00
Eelco Dolstra
efbf499635 Remove redundant " 2024-08-21 16:50:18 +02:00
Eelco Dolstra
3fe1b60c57 Remove dead makeMutable() function
This code wasn't being compiled because the #if conditional didn't
trigger. Anyway, Nix 0.15 is 15 years old so we don't need to keep
this around.
2024-08-21 16:45:46 +02:00
Eelco Dolstra
e049d38290 Merge remote-tracking branch 'origin/master' into fsync-store-paths 2024-08-21 16:37:21 +02:00
eldritch horrors
03b258bf97 libutil: rename and optimize closeMostFDs
this is only used to close non-stdio files in derivation sandboxes. we
may as well encode that in its name, drop the unnecessary integer set,
and use close_range to deal with the actual closing of files. not only
is this clearer, it also makes sandbox setup on linux fast by 1ms each

(cherry-picked and adapted from
c7d97802e4)

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
2024-08-21 06:45:31 +02:00
John Ericson
af26fe3934
Merge pull request #11281 from siddhantk232/path-in-exec
Use `std::filesystem::path` in end executables
2024-08-20 12:35:57 -04:00
Eelco Dolstra
257470b58d Merge remote-tracking branch 'origin/master' into flip-coroutines 2024-08-20 17:19:17 +02:00
tomberek
43e82c9446
Merge pull request #11300 from noamraph/homeless-shelter-to-proc
Make $HOME=/proc/homeless-shelter instead of /homeless-shelter
2024-08-19 23:26:41 -04:00
tomberek
d79b9bdec0
Merge pull request #11246 from NixOS/tomberek.ssl_warning
feat: better warning for common SSL error
2024-08-19 13:51:42 -04:00
tomberek
1ff0a983d6
Merge pull request #11322 from NixOS/eval-cache-doc
doc: Manage expectations for eval-cache
2024-08-19 13:50:15 -04:00
Noam Yorav-Raphael
62b9a26f60 Set $HOME=/proc/homeless-shelter on Linux, and /homeless-shelter on OSX. 2024-08-19 20:18:08 +03:00
Eelco Dolstra
39daa4a0d3 withFramedSink(): Don't use a thread to monitor the other side
Since withFramedSink() is now used a lot more than in the past (for
every addToStore() variant), we were creating a lot of threads, e.g.

  nix flake show --no-eval-cache --all-systems github:NixOS/nix/afdd12be5e19c0001ff3297dea544301108d298

would create 46418 threads. While threads on Linux are cheap, this is
still substantial overhead.

So instead, just poll from FramedSink before every write whether there
are pending messages from the daemon. This could slightly increase the
latency on log messages from the daemon, but not on exceptions (which
were only synchronously checked from FramedSink anyway).

This speeds up the command above from 19.2s to 17.5s on my machine (a
9% speedup).
2024-08-19 18:15:15 +02:00
John Ericson
84ea12ad7f Fix build errors on Windows 2024-08-19 11:02:46 -04:00
Tom Bereknyei
3e5bf90341 feat: better warning for common SSL errors 2024-08-19 11:01:56 -04:00
Robert Hensing
b52e58903e doc: Manage expectations for eval-cache
Incorrectly high expectations lead to frustration for users who
stick around to experience how useless it is for e.g. a devShell

https://functional.cafe/@arianvp/112976284363120036:

> Flakes doesn't have eval caching. It has command line argument
> caching. It literally just stores the cli argument you passed
> in a sqlite database and yes that's as useless as it sounds

> When I discovered flakes had no expression level caching whatsoever
> I kind of felt lied to and betrayed.
2024-08-19 16:52:36 +02:00
John Ericson
67a54d47c5 Merge remote-tracking branch 'upstream/master' into path-in-exec 2024-08-19 10:42:12 -04:00
Tom Bereknyei
59db8fd62b fix: check to see if there are any lines before 2024-08-18 22:35:54 -04:00
tomberek
1c5ad159d6
Merge pull request #10980 from kjeremy/flake-show-description
nix flake show: add the description if it exists
2024-08-18 16:31:44 -04:00
siddhantCodes
9ccad9708e Don't use OS_STR in nix-channel.cc 2024-08-18 18:27:20 +05:30
siddhantCodes
58ef129502 Merge branch 'master' of github.com:NixOS/nix into path-in-exec 2024-08-17 20:05:31 +05:30
Robert Hensing
c458598647
Merge pull request #11297 from shivaraj-bh/flake-apps-description
`nix flake show`: Support `meta` attribute for `apps`
2024-08-17 13:12:01 +02:00
shivaraj-bh
adabca6e4f nix flake check: Add apps check; Check if formatter is a derivation 2024-08-17 15:50:06 +05:30
shivaraj-bh
a5f6ee8550 nix flake show: Support meta attribute for apps
Metadata information for flake apps will be useful while exploring a
flake using `nix flake show`
2024-08-17 15:50:06 +05:30
tomberek
9e37a93229
Merge pull request #11270 from amarshall/fix-darwin-sandbox-local-networking
libstore: fix port binding in __darwinAllowLocalNetworking sandbox
2024-08-16 23:17:21 -04:00
tomberek
92df2a7cb2
Merge pull request #11285 from DeterminateSystems/downloadTarball-cacheable
fetchers::downloadTarball(): Return a cacheable accessor
2024-08-16 23:05:49 -04:00
Robert Hensing
a03bb4455c Fix SSH invocation when local SHELL misbehaves
Setting it to /bin/sh will make it more predictable when users have
their favorite shell in SHELL, which might not behave as expected.
For instance, a bad rc file could send something to stdout before
our LocalCommand gets to write "started".

This may help https://github.com/NixOS/nix/issues/11010
2024-08-16 15:53:30 +02:00
Robert Hensing
c4192a6617 Add nix::execvpe 2024-08-16 15:53:30 +02:00
Robert Hensing
72a4d1f52d Add :doc support for __functor 2024-08-15 13:04:34 +02:00
Robert Hensing
6068e32aa7 refactor: Extract EvalState::addCallDepth 2024-08-15 13:04:34 +02:00
John Ericson
34fe2478a2 Build Functional tests with Meson
Co-Authored-By: Qyriad <qyriad@qyriad.me>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-08-14 15:35:40 -04:00
Noam Yorav-Raphael
77d84a8d8b /homeless-shelter -> /proc/homeless/shelter
This makes it so even root can't create $HOME, for example by running `mkdir -p $HOME/.cache/foo`.
2024-08-14 14:35:42 +03:00
Robert Hensing
622c402659
Merge pull request #11292 from tomberek/tomberek.symbolstring
fix: use SymbolStr in constructor
2024-08-13 23:57:59 +02:00
Valentin Gagarin
4956e7c44c
add cross-references to nix-path overriding (#11288)
* add cross-references to `nix-path` overriding

while this information is already present in the settings, it's more
likely to be first accessed through the "lookup path" page, which
currently requires following two links to get to the practically
important bits.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-08-13 19:22:32 +02:00
John Ericson
95fe9f5ba1 Fix Meson installation of the Nix CLI
Co-Authored-By: Qyriad <qyriad@qyriad.me>
2024-08-13 09:22:06 -04:00
Tom Bereknyei
f22bf867eb fix: use SymbolStr in constructor 2024-08-12 22:18:14 -04:00
Jeremy Kolb
d49e14ba4a Take ANSI and tree characters into account 2024-08-12 14:49:52 -04:00
John Ericson
58b03ef1cd Move NIX_BIN_DIR and all logic using it to the Nix executable itself
This is because with the split packages of the Meson build, we simply
have no idea what directory the binaries will be installed in when we
build the library.

In the process of doing so, consolidate and make more sophisticated the
logic to cope with a few corner cases (e.g. `NIX_BIN_DIR` exists, but no
binaries are inside it).

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-08-12 12:29:17 -04:00
Eelco Dolstra
9f6ee93f48 fetchers::downloadTarball(): Return a cacheable accessor
downloadTarball() is used by `-I foo=<url>` etc. fetchToStore() needs
the accessor to have a fingerprint to enable caching.

Fixes #11271.
2024-08-12 15:48:26 +02:00
siddhantCodes
d98e06a581 Use std::filesystem::path in more executables 2024-08-11 20:13:47 +05:30
siddhantCodes
8e70f6f850 Use std::filesystem::path in profile.cc ...
...and `run.cc`
2024-08-11 19:56:06 +05:30
siddhantCodes
0abc664a78 Use std::filesystem in eval and flake ...
... executables
2024-08-11 19:53:34 +05:30
siddhantCodes
2c12a6962e Remove unused variable 2024-08-11 19:18:04 +05:30
siddhantCodes
70dde8c70c Use std::filesystem::path in build.cc 2024-08-11 16:30:59 +05:30
Robert Hensing
18485d2d53
Merge pull request #11188 from lf-/jade/kill-int-overflow
Ban integer overflow in the Nix language
2024-08-11 04:24:16 +02:00