Commit graph

8539 commits

Author SHA1 Message Date
Alexander Bantyev
5291a82cd9
SSHMaster: pause logger to show password prompt
Pause logger before starting SSH connections, and resume it after the
connection is established, so that SSH password prompts are not erased
by the logger's updates.
2023-03-22 09:45:08 +04:00
Alexander Bantyev
85df7e7ea2
Logger, ProgressBar: add a way to pause/resume
Add new virtual methods pause and resume to the Logger class, and
implement them in ProgressBar to allow to pause the bar refreshing.
2023-03-22 09:45:08 +04:00
Eelco Dolstra
16cb9b9f04 Fix rendering of fetchGit documentation
stripIndentation() doesn't support tabs, so the entire markdown ended
up indented and thus rendered as a code block.
2023-03-21 14:48:15 +01:00
Eelco Dolstra
3ac3e31c4d Fix misrendering of 'nix store --help'
There are no categories underneath 'nix store', so having 'nix store
copy-log' in a category rendered as ':'.
2023-03-21 12:02:19 +01:00
Eelco Dolstra
515662ad70 Cleanup 2023-03-20 18:06:08 +01:00
Eelco Dolstra
16db8dc96f Open slave pseudoterminal before CLONE_NEWUSER
Otherwise, when running as root and user namespaces are enabled,
opening the slave fails with EPERM.

Fixes "opening pseudoterminal slave: Permission denied" followed by a
hang (https://hydra.nixos.org/build/213104244), and "error: getting
sandbox mount namespace: No such file or directory" (#8072), which
happens when the child fails very quickly and consequently reading
/proc/<child>/ns fails.
2023-03-20 17:58:36 +01:00
figsoda
54b8da0b10 docs: quote URL literals 2023-03-19 11:28:10 -04:00
John Ericson
790dd2555b
Merge pull request #7891 from mupdt/shared-fs-out-links
no-op refactor: extract outLink generation into a function
2023-03-17 10:57:07 -04:00
Eelco Dolstra
7f46ebcf90
Merge pull request #8049 from edolstra/unexpected-eof
Fix "unexpected EOF" errors on macOS
2023-03-16 16:13:42 +01:00
Linus Heckemann
af4cbdafe7 LocalDerivationGoal: set NIX_ATTRS_*_FILE correctly for sandboxed builds 2023-03-16 15:26:52 +01:00
John Ericson
eb56cb7cc7
Merge pull request #7750 from obsidiansystems/no-args-prepare
Make command infra less stateful and more regular
2023-03-15 17:23:40 -04:00
John Ericson
bc23a44c54 Make command infra less stateful and more regular
Already, we had classes like `BuiltPathsCommand` and `StorePathsCommand`
which provided alternative `run` virtual functions providing the
implementation with more arguments. This was a very nice and easy way to
make writing command; just fill in the virtual functions and it is
fairly clear what to do.

However, exception to this pattern were `Installable{,s}Command`. These
two classes instead just had a field where the installables would be
stored, and various side-effecting `prepare` and `load` machinery too
fill them in. Command would wish out those fields.

This isn't so clear to use.

What this commit does is make those command classes like the others,
with richer `run` functions.

Not only does this restore the pattern making commands easier to write,
it has a number of other benefits:

- `prepare` and `load` are gone entirely! One command just hands just
  hands off to the next.

- `useDefaultInstallables` because `defaultInstallables`. This takes
  over `prepare` for the one case that needs it, and provides enough
  flexiblity to handle `nix repl`'s idiosyncratic migration.

- We can use `ref` instead of `std::shared_ptr`. The former must be
  initialized (so it is like Rust's `Box` rather than `Option<Box>`,
  This expresses the invariant that the installable are in fact
  initialized much better.

  This is possible because since we just have local variables not
  fields, we can stop worrying about the not-yet-initialized case.

- Fewer lines of code! (Finally I have a large refactor that makes the
  number go down not up...)

- `nix repl` is now implemented in a clearer way.

The last item deserves further mention. `nix repl` is not like the other
installable commands because instead working from once-loaded
installables, it needs to be able to load them again and again.

To properly support this, we make a new superclass
`RawInstallablesCommand`. This class has the argument parsing and
completion logic, but does *not* hand off parsed installables but
instead just the raw string arguments.

This is exactly what `nix repl` needs, and allows us to instead of
having the logic awkwardly split between `prepare`,
`useDefaultInstallables,` and `load`, have everything right next to each
other. I think this will enable future simplifications of that argument
defaulting logic, but I am saving those for a future PR --- best to keep
code motion and more complicated boolean expression rewriting separate
steps.

The "diagnostic ignored `-Woverloaded-virtual`" pragma helps because C++
doesn't like our many `run` methods. In our case, we don't mind the
shadowing it all --- it is *intentional* that the derived class only
provides a `run` method, and doesn't call any of the overridden `run`
methods.

Helps with https://github.com/NixOS/rfcs/pull/134
2023-03-15 16:29:07 -04:00
John Ericson
0a140a92bb
Merge pull request #7690 from ShamrockLee/nix-hash-sri
nix-hash: support base-64 and SRI format
2023-03-15 16:02:46 -04:00
Yueh-Shun Li
c27d358abb nix-hash: support base-64 and SRI format
Add the --base64 and --sri flags for the Base64 and SRI format output.

Add the --base16 flag to explicitly specify the hexadecimal format.

Add the --to-base64 and --to-sri flag to convert a hash to the above
mentioned format.
2023-03-16 03:08:42 +08:00
figsoda
31d588db8b respect allRefs=1 when using nix flake prefetch 2023-03-15 11:29:01 -04:00
Eelco Dolstra
5eb8bfd0f1 Remove a variable in LocalDerivationGoal 2023-03-15 10:59:10 +01:00
Eelco Dolstra
6029c763c2 Change builderOut from Pipe to AutoCloseFD 2023-03-15 10:59:10 +01:00
Eelco Dolstra
c536e00c9d Move pseudoterminal slave open to child
Hopefully this fixes "unexpected EOF" failures on macOS
(#3137, #3605, #7242, #7702).

The problem appears to be that under some circumstances, macOS
discards the output written to the slave side of the
pseudoterminal. Hence the parent never sees the "sandbox initialized"
message from the child, even though it succeeded. The conditions are:

* The child finishes very quickly. That's why this bug is likely to
  trigger in nix-env tests, since that uses a builtin builder. Adding
  a short sleep before the child exits makes the problem go away.

* The parent has closed its duplicate of the slave file
  descriptor. This shouldn't matter, since the child has a duplicate
  as well, but it does. E.g. moving the close to the bottom of
  startBuilder() makes the problem go away. However, that's not a
  solution because it would make Nix hang if the child dies before
  sending the "sandbox initialized" message.

* The system is under high load. E.g. "make installcheck -j16" makes
  the issue pretty reproducible, while it's very rare under "make
  installcheck -j1".

As a fix/workaround, we now open the pseudoterminal slave in the
child, rather than the parent. This removes the second condition
(i.e. the parent no longer needs to close the slave fd) and I haven't
been able to reproduce the "unexpected EOF" with this.
2023-03-15 10:58:37 +01:00
Eelco Dolstra
19326ac297 Simplify commonChildInit() 2023-03-14 18:58:12 +01:00
Robert Hensing
a387f46967
Merge pull request #8033 from lbodor/stop-adding-dot-to-nix-dev-env-path
`print-dev-env`: stop inadvertently adding `.` to `PATH`
2023-03-13 19:52:41 +01:00
Eelco Dolstra
fcd0b0fbd5 Fix macOS warning 2023-03-13 05:31:03 -07:00
lbodor
526bdbda3c print-dev-env: stop inadvertently adding . to PATH 2023-03-12 22:40:47 +11:00
Théophane Hufschmitt
99af56cd0d
Merge pull request #8015 from tweag/progress-during-nix-copy
Display progress when running copyPaths (nix copy)
2023-03-10 16:23:22 +01:00
Eelco Dolstra
4f3a4b732c Make findRuntimeRoots() more resilient to disappearing processes
I saw this random failure in https://hydra.nixos.org/build/211811692:

  error: opening /proc/15307/fd: No such process

while running nix-collect-garbage in a readfile-context.sh. This is
because we're not handling ESRCH errors reading /proc/<pid>/fd. So
just move the read inside the try/catch where we do handle it.
2023-03-09 16:44:51 +01:00
Eelco Dolstra
8f96720b9f Fix another uninitialized variable
https://hydra.nixos.org/build/211811494
2023-03-09 15:30:53 +01:00
Alexander Bantyev
4bef2016a1
Display progress when running copyPaths (nix copy)
`nix copy` operations did not show progress. This is quite confusing.

Add a `progressSink` which displays the progress during `copyPaths`,
pretty much copied from `copyStorePath`.

Fixes https://github.com/NixOS/nix/issues/8000
2023-03-09 16:03:48 +04:00
Théophane Hufschmitt
39700c5cbe
Revert "Disable GC during coroutine execution + test" 2023-03-08 20:47:52 +01:00
Eelco Dolstra
1607ad5076 Fix uninitialized readFromStdIn variable
This was causing random failures in tests/ca/substitute.ca: 'nix copy
--file ./content-addressed.nix' wouldn't get the default installable
'.' applied in InstallablesCommand::load(), so it would do nothing.
2023-03-08 13:53:20 +01:00
Eelco Dolstra
df48040786
Merge pull request #7994 from edolstra/fix-ca-crash
Fix crash/hang with CA derivations
2023-03-08 12:15:55 +01:00
Eelco Dolstra
6b243ef2d5
Merge pull request #7993 from tweag/fix-profile-gc
Fix `nix-collect-garbage -d` with the new profile location
2023-03-08 11:10:45 +01:00
Eelco Dolstra
7bfed34367 Fix crash/hang with CA derivations
The curl download can outlive DrvOutputSubstitutionGoal (if some other
error occurs), so at shutdown setting the promise to an exception will
fail because 'this' is no longer valid in the callback. This can
manifest itself as a segfault, "corrupted double-linked list" or hang.
2023-03-08 11:09:15 +01:00
Théophane Hufschmitt
4a6244dcf7
Merge pull request #7725 from yorickvP/check-coro-gc
Disable GC during coroutine execution + test
2023-03-08 09:51:46 +01:00
Théophane Hufschmitt
2272bc6049 Fix nix-collect-garbage -d with the new profile location
Low-hanging fix for https://github.com/NixOS/nix/pull/5226#issuecomment-1454669399
2023-03-07 13:08:04 +01:00
Théophane Hufschmitt
ba0486f045
Merge pull request #7889 from sidkshatriya/sorted-fetch-paths
Print the store paths to be fetched sorted by StorePath name()
2023-03-07 11:58:10 +01:00
mupdt
762cf7f214 no-op refactor: extract outLink generation into a function 2023-03-06 10:20:21 -05:00
Eelco Dolstra
3f4dc61f7f
Merge pull request #7958 from amesgen/issue-7955
InstallableFlake: Apply nix config in `getCursors`
2023-03-06 11:08:51 +01:00
Eelco Dolstra
92771ca1be
Merge pull request #7975 from fricklerhandwerk/fix-fetchgit
remove indentation in `fetchGit` attribute listing
2023-03-06 11:00:40 +01:00
Valentin Gagarin
dfeb83cac1 output paths of store derivations are now addressed explicitly 2023-03-06 03:32:14 +01:00
Valentin Gagarin
f69d453c3e remove indentation in fetchGit attribute listing
also reword a confusing sentence and add links to Git terminology
2023-03-05 04:44:11 +01:00
Valentin Gagarin
2af9fd20c6 clarify definition of "installable"
the term was hard to discover, as its definition and explanation were in
a very long document lacking an overview section.
search did not help because it occurs so often.

- clarify wording in the definition
- add an overview of installable types
- add "installable" to glossary
- link to definition from occurrences of the term
- be more precise about where store derivation outputs are processed
- installable Nix expressions must evaluate to a derivation

Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com>
2023-03-05 01:46:17 +01:00
Valentin Gagarin
1e87d5f1ea make descriptions of each installable type an own subsection
this is easier to edit, provides anchors for free, and renders correctly
on the terminal without additional effort.
2023-03-05 01:46:17 +01:00
amesgen
77bb950566
InstallableFlake: Apply nix config in getCursors 2023-03-04 12:52:20 +01:00
Sidharth Kshatriya
427555861b Print the store paths to be fetched sorted by StorePath name() and not baseName
Presently when nix says something like:

```
these 486 paths will be fetched (511.54 MiB download, 6458.64 MiB unpacked):
 ...path1
 ...path2
 ...path3
    ...
    ...
 ...path486
```

It sorts path1, path2, path3, ..., path486 in lexicographic order of the
store path.

After this commit, nix will show path1, path2, path3, ..., path486 sorted by
StorePath name() (basically everything after the hash) rather than the store path.

This makes it easier to review what exactly is being downloaded at a glance,
especially when many paths need to be fetched.
2023-03-03 21:46:35 +05:30
Eelco Dolstra
0507462c06
Merge pull request #7918 from zimbatm/fix-empty-nix-store-env
treat empty NIX_STORE_DIR env vars as unset
2023-03-03 13:49:56 +01:00
Yorick van Pelt
2683734936
Add talkative msg for coro gc debug 2023-03-03 11:43:47 +01:00
zimbatm
dc8820c71f
fixup: use same style as getEnv 2023-03-03 11:34:36 +01:00
zimbatm
b96d9c1687
fixup: remove warning entirely
fixes 72e1e23051 (r1124211067)
2023-03-03 11:32:06 +01:00
Eelco Dolstra
f0908f592c
Merge pull request #7942 from edolstra/remove-format
Remove FormatOrString and remaining uses of format()
2023-03-03 10:22:11 +01:00
Valentin Gagarin
37c61907d7
Merge pull request #7605 from tweag/comments
add comments
2023-03-03 06:38:29 +01:00
Théophane Hufschmitt
1f394d2107
Merge branch 'master' into paths-from-stdin 2023-03-02 19:20:51 +01:00