Commit graph

8524 commits

Author SHA1 Message Date
John Ericson
eb56cb7cc7
Merge pull request #7750 from obsidiansystems/no-args-prepare
Make command infra less stateful and more regular
2023-03-15 17:23:40 -04:00
John Ericson
bc23a44c54 Make command infra less stateful and more regular
Already, we had classes like `BuiltPathsCommand` and `StorePathsCommand`
which provided alternative `run` virtual functions providing the
implementation with more arguments. This was a very nice and easy way to
make writing command; just fill in the virtual functions and it is
fairly clear what to do.

However, exception to this pattern were `Installable{,s}Command`. These
two classes instead just had a field where the installables would be
stored, and various side-effecting `prepare` and `load` machinery too
fill them in. Command would wish out those fields.

This isn't so clear to use.

What this commit does is make those command classes like the others,
with richer `run` functions.

Not only does this restore the pattern making commands easier to write,
it has a number of other benefits:

- `prepare` and `load` are gone entirely! One command just hands just
  hands off to the next.

- `useDefaultInstallables` because `defaultInstallables`. This takes
  over `prepare` for the one case that needs it, and provides enough
  flexiblity to handle `nix repl`'s idiosyncratic migration.

- We can use `ref` instead of `std::shared_ptr`. The former must be
  initialized (so it is like Rust's `Box` rather than `Option<Box>`,
  This expresses the invariant that the installable are in fact
  initialized much better.

  This is possible because since we just have local variables not
  fields, we can stop worrying about the not-yet-initialized case.

- Fewer lines of code! (Finally I have a large refactor that makes the
  number go down not up...)

- `nix repl` is now implemented in a clearer way.

The last item deserves further mention. `nix repl` is not like the other
installable commands because instead working from once-loaded
installables, it needs to be able to load them again and again.

To properly support this, we make a new superclass
`RawInstallablesCommand`. This class has the argument parsing and
completion logic, but does *not* hand off parsed installables but
instead just the raw string arguments.

This is exactly what `nix repl` needs, and allows us to instead of
having the logic awkwardly split between `prepare`,
`useDefaultInstallables,` and `load`, have everything right next to each
other. I think this will enable future simplifications of that argument
defaulting logic, but I am saving those for a future PR --- best to keep
code motion and more complicated boolean expression rewriting separate
steps.

The "diagnostic ignored `-Woverloaded-virtual`" pragma helps because C++
doesn't like our many `run` methods. In our case, we don't mind the
shadowing it all --- it is *intentional* that the derived class only
provides a `run` method, and doesn't call any of the overridden `run`
methods.

Helps with https://github.com/NixOS/rfcs/pull/134
2023-03-15 16:29:07 -04:00
John Ericson
0a140a92bb
Merge pull request #7690 from ShamrockLee/nix-hash-sri
nix-hash: support base-64 and SRI format
2023-03-15 16:02:46 -04:00
Yueh-Shun Li
c27d358abb nix-hash: support base-64 and SRI format
Add the --base64 and --sri flags for the Base64 and SRI format output.

Add the --base16 flag to explicitly specify the hexadecimal format.

Add the --to-base64 and --to-sri flag to convert a hash to the above
mentioned format.
2023-03-16 03:08:42 +08:00
figsoda
31d588db8b respect allRefs=1 when using nix flake prefetch 2023-03-15 11:29:01 -04:00
Robert Hensing
a387f46967
Merge pull request #8033 from lbodor/stop-adding-dot-to-nix-dev-env-path
`print-dev-env`: stop inadvertently adding `.` to `PATH`
2023-03-13 19:52:41 +01:00
Eelco Dolstra
fcd0b0fbd5 Fix macOS warning 2023-03-13 05:31:03 -07:00
lbodor
526bdbda3c print-dev-env: stop inadvertently adding . to PATH 2023-03-12 22:40:47 +11:00
Théophane Hufschmitt
99af56cd0d
Merge pull request #8015 from tweag/progress-during-nix-copy
Display progress when running copyPaths (nix copy)
2023-03-10 16:23:22 +01:00
Eelco Dolstra
4f3a4b732c Make findRuntimeRoots() more resilient to disappearing processes
I saw this random failure in https://hydra.nixos.org/build/211811692:

  error: opening /proc/15307/fd: No such process

while running nix-collect-garbage in a readfile-context.sh. This is
because we're not handling ESRCH errors reading /proc/<pid>/fd. So
just move the read inside the try/catch where we do handle it.
2023-03-09 16:44:51 +01:00
Eelco Dolstra
8f96720b9f Fix another uninitialized variable
https://hydra.nixos.org/build/211811494
2023-03-09 15:30:53 +01:00
Alexander Bantyev
4bef2016a1
Display progress when running copyPaths (nix copy)
`nix copy` operations did not show progress. This is quite confusing.

Add a `progressSink` which displays the progress during `copyPaths`,
pretty much copied from `copyStorePath`.

Fixes https://github.com/NixOS/nix/issues/8000
2023-03-09 16:03:48 +04:00
Théophane Hufschmitt
39700c5cbe
Revert "Disable GC during coroutine execution + test" 2023-03-08 20:47:52 +01:00
Eelco Dolstra
1607ad5076 Fix uninitialized readFromStdIn variable
This was causing random failures in tests/ca/substitute.ca: 'nix copy
--file ./content-addressed.nix' wouldn't get the default installable
'.' applied in InstallablesCommand::load(), so it would do nothing.
2023-03-08 13:53:20 +01:00
Eelco Dolstra
df48040786
Merge pull request #7994 from edolstra/fix-ca-crash
Fix crash/hang with CA derivations
2023-03-08 12:15:55 +01:00
Eelco Dolstra
6b243ef2d5
Merge pull request #7993 from tweag/fix-profile-gc
Fix `nix-collect-garbage -d` with the new profile location
2023-03-08 11:10:45 +01:00
Eelco Dolstra
7bfed34367 Fix crash/hang with CA derivations
The curl download can outlive DrvOutputSubstitutionGoal (if some other
error occurs), so at shutdown setting the promise to an exception will
fail because 'this' is no longer valid in the callback. This can
manifest itself as a segfault, "corrupted double-linked list" or hang.
2023-03-08 11:09:15 +01:00
Théophane Hufschmitt
4a6244dcf7
Merge pull request #7725 from yorickvP/check-coro-gc
Disable GC during coroutine execution + test
2023-03-08 09:51:46 +01:00
Théophane Hufschmitt
2272bc6049 Fix nix-collect-garbage -d with the new profile location
Low-hanging fix for https://github.com/NixOS/nix/pull/5226#issuecomment-1454669399
2023-03-07 13:08:04 +01:00
Théophane Hufschmitt
ba0486f045
Merge pull request #7889 from sidkshatriya/sorted-fetch-paths
Print the store paths to be fetched sorted by StorePath name()
2023-03-07 11:58:10 +01:00
Eelco Dolstra
3f4dc61f7f
Merge pull request #7958 from amesgen/issue-7955
InstallableFlake: Apply nix config in `getCursors`
2023-03-06 11:08:51 +01:00
Eelco Dolstra
92771ca1be
Merge pull request #7975 from fricklerhandwerk/fix-fetchgit
remove indentation in `fetchGit` attribute listing
2023-03-06 11:00:40 +01:00
Valentin Gagarin
dfeb83cac1 output paths of store derivations are now addressed explicitly 2023-03-06 03:32:14 +01:00
Valentin Gagarin
f69d453c3e remove indentation in fetchGit attribute listing
also reword a confusing sentence and add links to Git terminology
2023-03-05 04:44:11 +01:00
Valentin Gagarin
2af9fd20c6 clarify definition of "installable"
the term was hard to discover, as its definition and explanation were in
a very long document lacking an overview section.
search did not help because it occurs so often.

- clarify wording in the definition
- add an overview of installable types
- add "installable" to glossary
- link to definition from occurrences of the term
- be more precise about where store derivation outputs are processed
- installable Nix expressions must evaluate to a derivation

Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com>
2023-03-05 01:46:17 +01:00
Valentin Gagarin
1e87d5f1ea make descriptions of each installable type an own subsection
this is easier to edit, provides anchors for free, and renders correctly
on the terminal without additional effort.
2023-03-05 01:46:17 +01:00
amesgen
77bb950566
InstallableFlake: Apply nix config in getCursors 2023-03-04 12:52:20 +01:00
Sidharth Kshatriya
427555861b Print the store paths to be fetched sorted by StorePath name() and not baseName
Presently when nix says something like:

```
these 486 paths will be fetched (511.54 MiB download, 6458.64 MiB unpacked):
 ...path1
 ...path2
 ...path3
    ...
    ...
 ...path486
```

It sorts path1, path2, path3, ..., path486 in lexicographic order of the
store path.

After this commit, nix will show path1, path2, path3, ..., path486 sorted by
StorePath name() (basically everything after the hash) rather than the store path.

This makes it easier to review what exactly is being downloaded at a glance,
especially when many paths need to be fetched.
2023-03-03 21:46:35 +05:30
Eelco Dolstra
0507462c06
Merge pull request #7918 from zimbatm/fix-empty-nix-store-env
treat empty NIX_STORE_DIR env vars as unset
2023-03-03 13:49:56 +01:00
Yorick van Pelt
2683734936
Add talkative msg for coro gc debug 2023-03-03 11:43:47 +01:00
zimbatm
dc8820c71f
fixup: use same style as getEnv 2023-03-03 11:34:36 +01:00
zimbatm
b96d9c1687
fixup: remove warning entirely
fixes 72e1e23051 (r1124211067)
2023-03-03 11:32:06 +01:00
Eelco Dolstra
f0908f592c
Merge pull request #7942 from edolstra/remove-format
Remove FormatOrString and remaining uses of format()
2023-03-03 10:22:11 +01:00
Valentin Gagarin
37c61907d7
Merge pull request #7605 from tweag/comments
add comments
2023-03-03 06:38:29 +01:00
Théophane Hufschmitt
1f394d2107
Merge branch 'master' into paths-from-stdin 2023-03-02 19:20:51 +01:00
Eelco Dolstra
687d86f8ab
Merge pull request #7940 from tweag/fix-http-errors-encoding
Log the decompressed body in case of http errors
2023-03-02 16:25:09 +01:00
Jonas Chevalier
72e1e23051
Update src/libutil/util.cc
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-03-02 16:17:20 +01:00
Eelco Dolstra
29abc8e764 Remove FormatOrString and remaining uses of format() 2023-03-02 15:57:54 +01:00
Eelco Dolstra
b69a73a230 Get rid of some unchecked calls to std::cout 2023-03-02 15:02:24 +01:00
Théophane Hufschmitt
1a825b6919 Log the decompressed body in case of http errors
Don't show the users the raw (possibly compressed) error message as
everyone isn't able to decompress brotli in their brain.
2023-03-02 14:59:15 +01:00
Eelco Dolstra
09f5975c6a Logger::cout: Use fmt()
This ensures that in cout(s), 's' does not get interpreted as a format
string.
2023-03-02 14:52:37 +01:00
Eelco Dolstra
989b823ac5 nix store cat: Use writeFull()
Fixes #7939.
2023-03-02 14:51:14 +01:00
Eelco Dolstra
c13cbd20ab Logger::writeToStdout(): Use writeFull()
This ensures that write errors do not get ignored.
2023-03-02 14:46:28 +01:00
Eelco Dolstra
767974f411
Merge pull request #7924 from mkenigs/valid
Always set valid in path-info --json output
2023-03-02 09:58:20 +01:00
Félix Baylac Jacqué
25300c0ecd
Treat empty env var paths as unset
We make sure the env var paths are actually set (ie. not "") before
sending them to the canonicalization function. If we forget to do so,
the user will end up facing a puzzled failed assertion internal error.

We issue a non-failing warning as a stop-gap measure. We could want to
revisit this to issue a detailed failing error message in the future.
2023-03-01 20:50:07 +01:00
Théophane Hufschmitt
182129d985
Merge pull request #7932 from obsidiansystems/remove-sameMachine
Remove dead code `RemoteStore::sameMachine`
2023-03-01 20:05:39 +01:00
Valentin Gagarin
651dab55da
Merge pull request #7854 from aameen-tulip/patch-1 2023-03-01 18:24:53 +01:00
John Ericson
b7f01a82a9 Remove dead code RemoteStore::sameMachine
It has been dead code since 9747ea84b4.
2023-03-01 11:10:30 -05:00
Yorick van Pelt
176005749c
Always disable GC in a coroutine unless the patch is applied 2023-03-01 15:07:00 +01:00
Yorick van Pelt
4c73eab923
DisableGC: replace by CoroutineContext, std::shared_ptr<void> 2023-03-01 13:55:41 +01:00