Commit graph

12243 commits

Author SHA1 Message Date
Maximilian Bosch
c85467a1b6
Revert "TarArchive: Small refactoring"
This reverts commit 50a35860ee.

With this change Nix fails to open bzip2 logfiles that were created from
builds with no stdout/stderr.
2022-03-24 22:30:46 +01:00
Eelco Dolstra
d9cfd853e5
Merge pull request #6302 from edolstra/fetch-closure
Add builtins.fetchClosure
2022-03-24 22:13:57 +01:00
Eelco Dolstra
f902f3c2cb Add experimental feature 'fetch-closure' 2022-03-24 21:33:33 +01:00
Eelco Dolstra
e5f7029ba4 nix store make-content-addressed: Support --from / --to 2022-03-24 21:33:33 +01:00
Eelco Dolstra
98658ae9d2 Document fetchClosure 2022-03-24 21:33:33 +01:00
Eelco Dolstra
28186b7044 Add a test for fetchClosure and 'nix store make-content-addressed' 2022-03-24 21:33:33 +01:00
Eelco Dolstra
4120930ac1 fetchClosure: Only allow some "safe" store types 2022-03-24 21:33:33 +01:00
Eelco Dolstra
7ffda0af6e fetchClosure: Skip makeContentAddressed() if toPath is already valid 2022-03-24 21:33:33 +01:00
Eelco Dolstra
5acaf13d35 Rename 'nix store make-content-addressable' to 'nix store make-content-addressed' 2022-03-24 21:33:33 +01:00
Eelco Dolstra
f18607549c Fix makeContentAddressed() on self-references
LocalStore::addToStore() since
79ae9e4558 expects a regular NAR hash,
rather than a NAR hash modulo self-references. Fixes #6300.

Also, makeContentAddressed() now rewrites the entire closure (so 'nix
store make-content-addressable' no longer needs '-r'). See #6301.
2022-03-24 21:33:33 +01:00
Eelco Dolstra
545c2d0d8c fetchClosure: Allow a path to be rewritten to CA on the fly
The advantage is that the resulting closure doesn't need to be signed,
so you don't need to configure any binary cache keys on the client.
2022-03-24 21:33:33 +01:00
Eelco Dolstra
7f6fe8ca1d Rename 2022-03-24 21:33:33 +01:00
Eelco Dolstra
41659418cf fetchClosure: Require a CA path in pure mode 2022-03-24 21:33:33 +01:00
Eelco Dolstra
f4bafc412f Add builtins.fetchClosure
This allows closures to be imported at evaluation time, without
requiring the user to configure substituters. E.g.

  builtins.fetchClosure {
    storePath = /nix/store/f89g6yi63m1ywfxj96whv5sxsm74w5ka-python3.9-sqlparse-0.4.2;
    from = "https://cache.ngi0.nixos.org";
  }
2022-03-24 21:33:33 +01:00
Eelco Dolstra
c9148f4ece
Merge pull request #6285 from flokli/add-tmpfile
nix-daemon.conf.in: add tmpfiles file to create nix/daemon-socket directory
2022-03-24 21:24:53 +01:00
Eelco Dolstra
d67fe90375
Merge pull request #6305 from flox/genericClosure_doc
docs: genericClosure
2022-03-24 14:02:58 +01:00
Eelco Dolstra
97734fea1b
Merge pull request #6308 from NixOS/consisten-use-of-url-uri-5872
Fix flake profile use of originalUrl vs. originalUri
2022-03-24 14:02:33 +01:00
Eelco Dolstra
721481e092
Merge pull request #6307 from edolstra/cp-symlinks
install-multi-user.sh: Preserve symlinks
2022-03-24 13:22:24 +01:00
Tom Bereknyei
0736f3651d docs: genericClosure 2022-03-24 08:03:59 -04:00
Eelco Dolstra
bb0c4b9f25 install-multi-user.sh: Preserve symlinks
We need to pass -P to ensure that symlinks are copied correctly. Fixes #6303.
2022-03-24 12:48:59 +01:00
Rok Garbas
4546a007a4
Fix flake profile use of originalUrl vs. originalUri
Fixes #5872
2022-03-24 12:28:38 +01:00
Eelco Dolstra
284cb0aad7
Merge pull request #6306 from trofi/add-lexer-locations
lexer: add error location to lexer errors
2022-03-24 10:07:40 +01:00
Sergei Trofimovich
9174d884d7 lexer: add error location to lexer errors
Before the change lexter errors did not report the location:

    $ nix build -f. mc
    error: path has a trailing slash
    (use '--show-trace' to show detailed location information)

Note that it's not clear what file generates the error.

After the change location is reported:

    $ src/nix/nix --extra-experimental-features nix-command build -f ~/nm mc
    error: path has a trailing slash

           at .../pkgs/development/libraries/glib/default.nix:54:18:

               53|   };
               54|   src = /tmp/foo/;
                 |                  ^
               55|
    (use '--show-trace' to show detailed location information)

Here we see both problematic file and the string itself.
2022-03-24 08:16:14 +00:00
Florian Klink
67af5f7eda scripts/install-systemd-multi-user.sh: install /etc/tmpfiles.d/nix-daemon.conf, too
While `create_directories()` from install-multi-user.sh seems to already
create parts of the directory structure, it's marked as deprecated, and
it won't hurt also copying over the tmpfiles config and have it execute
once.
2022-03-23 13:51:38 +01:00
Domen Kožar
98ce1a21b7
Merge pull request #6144 from toonn/doc-macos-uninstall
doc: Add detailed uninstall section for macOS
2022-03-23 08:53:48 +01:00
Eelco Dolstra
d5d4d98042
Merge pull request #6298 from kayhide/nix-edit-support-kakoune
nix edit: support kakoune
2022-03-22 16:56:35 +01:00
Hideaki Kawai
3b776cb0a7 nix edit: support kakoune 2022-03-22 23:18:02 +09:00
Eelco Dolstra
8434869632
Merge pull request #6296 from edolstra/fix-profile-manifest
Don't hide repeated values while generating manifest.nix
2022-03-22 14:09:25 +01:00
Eelco Dolstra
a0259a21a4 Don't hide repeated values while generating manifest.nix
Fixes #6243.
2022-03-22 13:18:56 +01:00
Eelco Dolstra
732296ddc0 printValue(): <REPEAT> -> «repeated»
This ensures that it doesn't get parsed as a valid Nix expression.
2022-03-22 13:00:27 +01:00
Eelco Dolstra
63f564ccf2
Merge pull request #6294 from edolstra/filter-manifest
buildProfile(): Ignore manifest.{nix,json}
2022-03-22 11:28:16 +01:00
Eelco Dolstra
0b42afe027 buildProfile(): Ignore manifest.{nix,json}
If a package installs a file named manifest.json, it caused nix-env to
consider the profile a new-style profile created by 'nix
profile'. Fixes #6032.
2022-03-22 10:48:02 +01:00
Eelco Dolstra
e4ff430866
Merge pull request #6237 from obsidiansystems/store-path-string-context
Decode string context straight to using StorePaths
2022-03-22 10:29:46 +01:00
Eelco Dolstra
7ed81701ee
Merge pull request #6290 from Misterio77/fix-sourcehut-tags-refs
Fix sourcehut tags refs resolving
2022-03-21 11:23:56 +01:00
Gabriel Fontes
31544b93ff
Fix sourcehut integration test
The new implementation relies on tab separting the hash and ref (this is how sourcehut does it). This fixes the integration test to use a tab instead of a space.
2022-03-19 11:38:45 -03:00
Gabriel Fontes
9720797f69
Don't partial match sourcehut refs 2022-03-19 11:04:04 -03:00
Gabriel Fontes
345a8ee0cb
Fix sourcehut tag ref resolving 2022-03-19 10:56:13 -03:00
Théophane Hufschmitt
8ad485ea89
Merge pull request #6284 from obsidiansystems/generlized-derivation-type
Generalize DerivationType in preparation for impure derivations
2022-03-18 16:36:32 +01:00
John Ericson
4d6a3806d2 Decode string context straight to using StorePaths
I gather decoding happens on demand, so I hope don't think this should
have any perf implications one way or the other.
2022-03-18 15:36:11 +00:00
Florian Klink
d60f3cf6e9 nix-daemon.conf.in: add tmpfiles file to create nix/daemon-socket directory
nix-daemon.socket is used to socket-activate nix-daemon.service when
/nix/var/nix/daemon-socket/socket is accessed.

In container usecases, sometimes /nix/var/nix/daemon-socket is
bind-mounted read-only into the container.

In these cases, we want to skip starting nix-daemon.socket.

However, since systemd 250, `ConditionPathIsReadWrite` is also not met
if /nix/var/nix/daemon-socket doesn't exist at all. This means, a
regular NixOS system will skip starting nix-daemon.socket:

> [  237.187747] systemd[1]: Nix Daemon Socket was skipped because of a failed condition check (ConditionPathIsReadWrite=/nix/var/nix/daemon-socket).

To prevent this from happening, ship a tmpfiles file that'll cause the
directory to be created if it doesn't exist already.

In the case of NixOS, we can just add Nix to `systemd.tmpfiles.packages`
and have these files picked up automatically.
2022-03-18 16:14:09 +01:00
John Ericson
a544ed7684 Generalize DerivationType in preparation for impure derivations 2022-03-18 14:59:56 +00:00
John Ericson
049fae155a Avoid some pointless copying of drvs 2022-03-18 14:59:56 +00:00
John Ericson
8496be7def Use Deferred when building an input-addressed drv
Easier than using dummy path with input addressed.
2022-03-18 14:59:56 +00:00
Théophane Hufschmitt
e433e42c5a
Merge pull request #6283 from obsidiansystems/outputs-less-nesting
Clean up `DerivationOutput`, and headers
2022-03-18 14:56:16 +01:00
John Ericson
197feed51d Clean up DerivationOutput, and headers
1. `DerivationOutput` now as the `std::variant` as a base class. And the
   variants are given hierarchical names under `DerivationOutput`.

   In 8e0d0689be @matthewbauer and I
   didn't know a better idiom, and so we made it a field. But this sort
   of "newtype" is anoying for literals downstream.

   Since then we leaned the base class, inherit the constructors trick,
   e.g. used in `DerivedPath`. Switching to use that makes this more
   ergonomic, and consistent.

2. `store-api.hh` and `derivations.hh` are now independent.

   In bcde5456cc I swapped the dependency,
   but I now know it is better to just keep on using incomplete types as
   much as possible for faster compilation and good separation of
   concerns.
2022-03-17 22:35:53 +00:00
Eelco Dolstra
6afc361798
Merge pull request #6277 from thufschmitt/ca/nix-build-dry-run
Fix `nix build --dry-run` with CA derivations
2022-03-17 22:23:15 +01:00
Eelco Dolstra
d8f69b9a4c Merge branch 'avoid-quadratic-gc' of https://github.com/trofi/nix 2022-03-17 21:35:42 +01:00
Sergei Trofimovich
d58453f72e gc: don't visit implicit referrers on garbage collection
Before the change garbage collector was not considering
`.drv` and outputs as alive even if configuration says otherwise.
As a result `nix store gc --dry-run` could visit (and parse)
`.drv` files multiple times (worst case it's quadratic).

It happens because `alive` set was populating only runtime closure
without regard for actual configuration. The change fixes it.

Benchmark: my system has about 139MB, 40K `.drv` files.

Performance before the change:

    $ time nix store gc --dry-run
    real    4m22,148s

Performance after the change:

    $ time nix store gc --dry-run
    real    0m14,178s
2022-03-17 18:47:29 +00:00
Théophane Hufschmitt
7117053457
Merge pull request #6270 from Artturin/stdineval
nix: allow using --file - to read from stdin
2022-03-17 14:35:01 +01:00
Théophane Hufschmitt
3fc4c612fb Fix nix build --dry-run with CA derivations
Don’t try and assume that we know the output paths when we’ve just built
with `--dry-run`. Instead make `--dry-run` follow a different code path
that won’t assume the knowledge of the output paths at all.

Fix #6275
2022-03-17 11:55:19 +01:00